system

Abstract

We provide the system. [Solution] Information input method for entering the user's health status, food preferences, allergy information, and family structure, A means of obtaining inventory information of products at nearby sales facilities based on location information, A means for integrating data obtained from the aforementioned information input means with the inventory information of the aforementioned products and proposing the optimal meal plan, A means of providing the user with a selected item from the aforementioned menu via voice notification, A system that includes this.

Description

Technical Field

[0004] , , ,

[0005] , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, the method including: receiving a user utterance; adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003] <从

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

Means for Solving the Problems

[0005] This invention provides a system that generates appropriate menus based on a user profile by inputting the user's health status, family structure, food preferences, and allergy information from a terminal, and obtaining inventory information of nearby sales facilities based on location information via a server. By utilizing an AI algorithm to integrate this data and providing a means to propose the optimal menu to the user in real time, the system supports users in making meal choices that meet their needs, streamlines decision-making when shopping, and helps with health management. Furthermore, it has a function to adjust the proposed menu in response to user requests, enabling flexible responses.

[0006] A "user" refers to a person who uses the system to decide on meal menus for themselves or their family.

[0007] "Health status" refers to information about the physical health of the user and their family, including specific conditions such as high blood pressure and allergies.

[0008] "Food preferences" refers to information about the types of ingredients and dishes that users and their families tend to prefer.

[0009] "Allergy information" refers to information about the possibility of a user or their family members having an allergic reaction to a particular food.

[0010] "Family structure" refers to information that shows the number of people living with the user in their household and their relationships with those people.

[0011] A "terminal" refers to a device used by a user to input information and communicate with a system.

[0012] A "server" is a central computer system that processes data received from users, and is responsible for data storage and menu generation.

[0013] "Location information" is geographical data that indicates where a user is currently located.

[0014] "Sales facility" refers to the place where users purchase food ingredients, including supermarkets and markets.

[0015] "Inventory information" refers to data on the types and quantities of products that a sales facility can offer.

[0016] "Menu plan" refers to the plan of the menu for meals to be taken on a specific day or period.

[0017] "AI algorithm" is a type of computer program for the system to analyze data and make proposals suitable for users' needs.

[0018] "Proposal" refers to the recommended actions or options shown by the system to the user.

[0019] "Real-time" means that information processing is performed immediately, referring to a state where the user can receive immediate feedback.

[0020] "Profile" is a set of data collected on the user's eating-related information, enabling responses based on individual needs.

[0021] "System" refers to a complex technical infrastructure that integrates all these elements to support the user's shopping and menu selection.

Brief Explanation of Drawings

[0022] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying out the Invention

[0023] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0026] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0027] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0028] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0030] [First Embodiment]

[0031] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0032] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0035] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0038] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0042] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0043] The system according to the present invention aims to enable users to efficiently decide on meal menus and promote a healthy lifestyle. The system's processing will be described in detail below based on an example.

[0044] Users first use a terminal application to pre-enter details such as their own and their family's health status, food preferences, allergy information, and family structure. This information is sent from the terminal to the server and stored in a database. Once a user profile is built, it becomes possible to optimize meal plans based on individual needs.

[0045] When a user actually starts shopping at a supermarket, the device acquires the user's location information and sends it to a server. Based on this location information, the server collects inventory information for products offered at nearby retail facilities. Based on the collected data, an AI algorithm generates multiple menu suggestions that take the user's profile into consideration.

[0046] As an example, consider a family of four where the father has high blood pressure, the mother is a vegetarian, and the child prefers meat. In this case, the AI ​​can provide menu suggestions that include low-sodium vegetarian options to accommodate the high blood pressure. For the child, it would then suggest a separate menu that incorporates meat in a balanced way.

[0047] The terminal notifies the user via voice of the menu suggestions received from the server. The user can review the suggestions on the spot and, if necessary, ask questions or request changes to the menu from the AI. For example, in response to a request to adjust the recipe, such as "I'd like to add a few more vegetables to this recipe," the AI ​​will immediately regenerate and re-suggest a new recipe.

[0048] This system responds in real time and flexibly as users choose meals while shopping. This allows users to shop with less stress and provide meals that are optimal for their personal and family health.

[0049] The following describes the processing flow.

[0050] Step 1:

[0051] The user launches the application on their device and enters details such as family health status, dietary preferences, allergy information, and family composition. After completing the input, the device sends this data to the server.

[0052] Step 2:

[0053] The server stores the received user data in a database and builds a profile for each user. This establishes an information infrastructure that can be accessed in real time.

[0054] Step 3:

[0055] When a user arrives at a supermarket and begins shopping, the device acquires their current location information and sends it to the server. Based on this information, the shopping destination is identified.

[0056] Step 4:

[0057] The server queries external databases and APIs to retrieve inventory and special offer information from nearby retail stores based on the user's location. Based on this information, it creates an up-to-date product list.

[0058] Step 5:

[0059] The server uses an AI algorithm to generate optimal menu suggestions based on the user's profile and acquired inventory information. It dynamically creates recipes tailored to the user's health condition and dietary preferences, providing multiple options.

[0060] Step 6:

[0061] The device provides the user with menu suggestions received from the server via voice notification. The user can then review the suggested menu.

[0062] Step 7:

[0063] Users can ask questions or request changes to the suggested menu. User feedback is sent to the server via the terminal.

[0064] Step 8:

[0065] The server regenerates or adjusts the menu in response to adjustment requests from the user. The modified menu is then sent back to the terminal and re-proposed.

[0066] As a result, users can easily select suitable menus while shopping and streamline meal preparation.

[0067] (Example 1)

[0068] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0069] In modern life, efficiently selecting and preparing meal plans while maintaining personal and family health is a crucial challenge for many. However, efficiently shopping while considering health status, food preferences, and allergy information remains difficult. The problem lies in the lack of tools that allow users to dynamically adjust meal plans based on their health information.

[0070] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0071] In this invention, the server includes data collection means for inputting information related to the user's health, means for acquiring information on items in nearby commercial facilities using location information, and means for generating an optimal meal configuration by combining the information from the data collection means and the item information. This enables the user to efficiently select and readjust the optimal meal according to their individual health condition.

[0072] A "user" is someone who uses the system to input information related to meal plan development and receives suggestions based on that information.

[0073] A "data collection method" is an interface that has the function of inputting the user's health status, preferences, and allergy information and sending it to a server.

[0074] "Location information" refers to information that indicates the user's geographical location and is used to obtain information about items in commercial facilities.

[0075] A "commercial facility" refers to a retail environment that sells food and related goods, and is a place where users actually shop.

[0076] "Product information" refers to data that includes information such as the type of food, inventory status, and price at commercial facilities.

[0077] "Meal composition" refers to menu suggestions tailored to the user's health condition and preferences, and includes specific ingredients and cooking methods.

[0078] "Voice notification" is a method of communicating the meal plan generated by the system to the user using voice.

[0079] An "information processing system" is a set of processing mechanisms, including computing resources and software, that integrate user input information with commercial facility item information to propose an optimized meal plan.

[0080] Embodiments of the present invention are described below. First, the user launches a dedicated application using a terminal device. Through the interface of this application, the user inputs information such as their own and their family's health status (e.g., hypertension, diabetes), dietary preferences (e.g., vegetarian, carbohydrate restriction), allergy information (e.g., nut allergy), and family structure (e.g., 2-person family, 4-person family). This information is formatted and encrypted by the terminal and then sent to the server. The server stores this information in a relational database and constructs it as a user profile.

[0081] When a user begins shopping at a commercial facility, their device uses its built-in GPS module to obtain real-time location information. This location information is then sent to a server, which uses the location data to retrieve product information from nearby commercial facilities via publicly available APIs. This product information includes details such as the type of product, price, and availability.

[0082] The server provides stored user profiles and real-time item information as input data to a generating AI model, which then generates an optimal meal plan tailored to the user's health status and preferences. This AI model utilizes deep learning and other machine learning algorithms to suggest customized menus to the user.

[0083] For example, a prompt such as "Please suggest a menu for a family of four, including low-sodium options for hypertension and vegetarian options" is used with the AI ​​model to generate suggestions. The generated menu suggestions are returned to the terminal and communicated to the user via voice notification. The user can review the suggested menu and request changes according to specific needs. These requests might include something like, "I'd like to add more vegetables to this recipe," and the server quickly regenerates the menu using the AI ​​model.

[0084] This system enables users to shop efficiently and choose meals that suit their health condition and preferences.

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] The user launches a dedicated application on their terminal device and enters personal information such as health status, dietary preferences, allergy information, and family structure. This information is formatted and encrypted by the terminal. The entered data is sent to the server via an HTTP POST request. As output, the user's information is securely transmitted to the server.

[0088] Step 2:

[0089] The server stores the received user information in a relational database. Specifically, it categorizes the data by field and constructs it as a user profile. The input is personal information from the user, and the output is an organized user profile in the database.

[0090] Step 3:

[0091] When a user arrives at a commercial facility, the device's GPS function automatically acquires location information. This location information is sent from the device to the server. The input is location coordinate data, and the server obtains the user's current location as output.

[0092] Step 4:

[0093] The server uses location information to retrieve item information from nearby commercial facilities using APIs. This is done by sending API requests and parsing the data returned in JSON format. The input is the user's location information, and the output is item information data from the commercial facilities.

[0094] Step 5:

[0095] The server provides user profiles and item information as input data to a generating AI model, which then generates an optimal meal plan. The AI ​​model operates based on a deep learning algorithm, combining this information to generate suggestions. User information and item information are used as input, and an optimized menu plan is generated as output.

[0096] Step 6:

[0097] The generated menu plan is sent from the server to the terminal. The terminal uses speech synthesis technology to notify the user of the menu plan by voice. The input is the menu plan data, and the output is the voice notification to the user.

[0098] Step 7:

[0099] Users can review menu suggestions and request specific changes. For example, they can request the addition or modification of certain ingredients. This request is sent from the terminal to the server. The input is the user's additional request, and the server regenerates and outputs the revised menu suggestion.

[0100] (Application Example 1)

[0101] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0102] In modern society, it is difficult to plan meals that take into account individual health conditions and dietary preferences on a daily basis, and users face many challenges in maintaining their own health and preparing meals that suit their family structure. Furthermore, it is not easy to immediately select the optimal menu while shopping at physical stores, taking inventory information into account. In this situation, there is a need for a system that can flexibly respond to the diverse needs of users.

[0103] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0104] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and family structure; means for collecting inventory information of goods at nearby commercial facilities based on location information; means for integrating the information obtained from the data input means with the inventory information of the goods to generate an optimal meal menu; means for presenting the generated menu to the user through voice guidance; and regeneration means using a generative AI model to adjust and re-suggest the generated menu using prompt sentences based on the user's request. This allows the user to dynamically decide on a menu even while shopping, and enables the provision of optimal meals tailored to individual health needs and family needs.

[0105] A "user" refers to an information user, specifically an individual or household member who receives meal menu suggestions from the system.

[0106] "Data input means" refers to a device or software for inputting information such as the user's health status, food preferences, allergy information, and family structure.

[0107] "Location information" refers to geographical data that indicates the user's current location.

[0108] A "commercial facility" refers to a physical store that sells goods, a place where users shop.

[0109] "Inventory information for goods" refers to information including the quantity and types of goods available for sale at a commercial facility.

[0110] "Integration" is the process of combining different types of information to give them meaning and create new value.

[0111] A "menu" refers to a specific meal plan that is proposed and consists of a combination of multiple dishes and ingredients.

[0112] "Voice guidance" is a method of conveying information from a system to a user via voice.

[0113] A "prompt statement" is a text statement that describes requests or instructions for a generative AI model.

[0114] A "generative AI model" is a type of artificial intelligence that performs data analysis and logical reasoning in response to user requests to generate new menus and information.

[0115] A "regeneration mechanism" is a system function that readjusts existing menus based on additional user requests and makes new suggestions.

[0116] This system integrates information from user terminals, servers, and commercial facilities to suggest menus that support a healthy lifestyle. Users input their health status, food preferences, allergy information, and family structure via their terminals, and this information is sent to the server and stored in a database. The server uses this information to build a user profile.

[0117] When a user enters a commercial facility, their device's location information is acquired and sent to a server. Based on this location information, the server collects inventory information for items in surrounding commercial facilities. This inventory information is collected using APIs and real-time data feeds provided by the commercial facilities. The server integrates the user's profile with the commercial facilities' inventory information and generates an optimal meal menu using an AI model.

[0118] The generated menu is presented to the user via voice guidance through the terminal. The user can then make requests to the system using prompts based on the voice guidance. For example, they can request, "I'd like an option to add more vegetables to this menu." The generation AI model immediately regenerates the menu based on the prompt and provides the guidance again through the terminal.

[0119] As a concrete example, a user might ask the system to "provide vegetarian and low-calorie menu suggestions to invite a friend over." In this case, a possible prompt to the generative AI model might be something like, "Please suggest a 3-item vegetarian and low-calorie menu." Upon receiving this prompt, the generative AI model would make adjustments according to the user's request and provide the user with a new menu suggestion.

[0120] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0121] Step 1:

[0122] Users operate a terminal to input their health status, food preferences, allergy information, and family structure. This input information is sent from the terminal to the server. The server stores the received information in a database and builds a user profile. This profile is used as the basis for subsequent menu generation.

[0123] Step 2:

[0124] When a user enters a commercial facility, the terminal acquires the user's location information. This location information is transmitted to the server in real time, and the user's current location is identified. Based on this location information, the server collects inventory information for items in the surrounding commercial facilities via APIs, etc. The output of this process is a list of items actually on the shelves.

[0125] Step 3:

[0126] The server uses the user's profile and acquired inventory information to generate an optimal meal menu using a generative AI model. This process takes into account the user's health goals, preferences, and allergy information, while selecting items from the available inventory. The generated menu suggestions are sent to the terminal as server output.

[0127] Step 4:

[0128] The terminal presents the received menu suggestions to the user via voice guidance. The user listens to the voice guidance and can make additional requests or adjustments to the system using prompt messages. The prompt messages entered at this time are sent to the server through the terminal.

[0129] Step 5:

[0130] The server uses the prompt message received from the user to regenerate the menu using a generation AI model. Based on the content of the prompt message, the data is reprocessed to generate new menu suggestions. These regenerated menu suggestions are then output from the server to the terminal.

[0131] Step 6:

[0132] The terminal then provides the user with new menu suggestions via voice guidance. Based on this information, the user can proceed with their shopping and select ingredients that best suit their and their family's needs. This entire process enables the user to engage in healthy and personalized shopping.

[0133] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0134] This invention relates to a system equipped with an emotion recognition engine that recognizes the user's emotions and optimizes meal menu suggestions based on those emotions. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure into a terminal, and this information is transmitted to a server and stored in a database.

[0135] The system sends location information from the user's device to the server when they make a purchase. The server uses this location information to retrieve inventory information from nearby stores, integrates it with the user profile, and uses an AI algorithm to generate optimal menu suggestions. In addition, it is equipped with an emotion recognition engine that analyzes the user's voice and facial expressions to recognize their current emotional state.

[0136] For example, if a user looks anxious while shopping, the emotion engine will detect that emotion. The server will take this emotional information into consideration and prioritize suggesting familiar dishes or easy-to-prepare recipes to reassure the user. Conversely, if joy or excitement is detected, it can suggest new recipes or challenging dishes.

[0137] The device notifies the user of menu suggestions received from the server via voice, adjusting the tone and content of the voice based on emotional data. For example, a relaxed tone of voice can support the user, and encouraging words can be added to enhance the effectiveness of the suggestions.

[0138] Furthermore, past emotional history is recorded in a database, and the server uses this information to analyze long-term trends and utilize it to suggest menus that are a better match for the user. In this way, the system realizes flexible and accurate meal suggestions that are attentive to the user's emotions, improving the user's shopping experience and health management.

[0139] The following describes the processing flow.

[0140] Step 1:

[0141] Users use an application on their device to input their own and their family's health status, dietary preferences, allergy information, and family structure. The device then sends this information to a server and stores it in a database.

[0142] Step 2:

[0143] The device collects emotional data from the user's voice and facial expressions. Using an emotion engine, it analyzes the collected data to recognize the user's current emotional state.

[0144] Step 3:

[0145] When a user starts shopping at a supermarket, the device sends its location information to a server. Based on this location information, the server retrieves inventory information for nearby retail stores.

[0146] Step 4:

[0147] The server integrates user information stored in the database, inventory information for the current location, and emotional data recognized by the emotion engine, and uses an AI algorithm to generate the optimal menu suggestion.

[0148] Step 5:

[0149] The server sends the generated menu suggestions to the terminal. The terminal presents these suggestions to the user via voice notification. At this time, the voice tone and message content are adjusted based on emotion data.

[0150] Step 6:

[0151] The user reviews the suggested menu and provides feedback or requests for changes via their device as needed. The device then sends this request to the server.

[0152] Step 7:

[0153] The server readjusts the menu based on user change requests and feedback. If necessary, it sends the regenerated menu to the terminal and proposes it again to the user.

[0154] Step 8:

[0155] The server records emotional data collected during each shopping session in a database and analyzes past emotional history. This information is used to make future suggestions and helps provide more personalized meal recommendations to the user.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] In today's lifestyle, choosing meals that suit individual health conditions and dietary preferences is crucial, but effective meal planning is challenging. Furthermore, suggesting meals that align with a user's emotional state is more complex than simply considering nutritional balance. Additionally, systems that enable flexible, real-time meal planning during shopping outside the home are limited, and improvement is needed.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes an input means for inputting the user's health status, food preferences, allergy information, and family structure; an acquisition means for acquiring inventory information of nearby sales locations based on location information; and a suggestion means for integrating the data acquired from the input means with the inventory information and proposing an optimal meal plan using a generative model. This enables flexible menu suggestions that are optimized for each user's profile and respond to their emotional state.

[0161] "Input means" refers to a device or software for which a user inputs information such as their health status, food preferences, allergy information, and family structure.

[0162] "Acquisition method" refers to the function or process for obtaining inventory information from nearby sales locations based on the user's location information.

[0163] The "proposal method" is a function that integrates acquired data and uses a generative model to present the most suitable meal plan for the user.

[0164] "Emotion recognition means" refers to a technology or device that analyzes a user's voice and facial expression data to identify the user's emotional state.

[0165] The "notification method" refers to a function that notifies the user of the generated menu plan via voice, and adjusts the voice tone and content according to emotional data.

[0166] This invention is a system for optimizing meal menu suggestions by taking user emotions into consideration. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure using a terminal. The terminal processes the collected data and transmits it to a server. The server stores the received information in a database and manages it appropriately.

[0167] Furthermore, when a user begins shopping, the system sends location information to a server via their device. The server uses the location information to retrieve inventory information from nearby retail stores and integrates it with the provided profile information. The server then uses an emotion recognition engine to analyze the user's voice and facial expressions to recognize their current emotional state. This emotional data influences menu suggestions and forms the basis for generating suggestions that match the user's emotions.

[0168] For example, if a user appears anxious, the server will suggest comforting, familiar dishes or easy-to-make recipes. Conversely, if a user expresses joy or excitement, it will suggest new recipes or challenging dishes. The generated menu suggestions are sent to the device and communicated to the user via voice notification. The notification may include a voice tone and encouraging message tailored to the user's emotions.

[0169] The server uses historical sentiment data to analyze long-term data trends. This analysis enables the system to suggest menus that are more suitable for the user.

[0170] As a concrete example, one could input a prompt message into the AI ​​model such as, "Please suggest a healthy dinner recipe that can be prepared quickly, especially when the user is tired from work." Such a prompt would allow the system to provide suggestions tailored to the user's state.

[0171] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0172] Step 1:

[0173] The user uses a terminal to input their health status, dietary preferences, allergy information, and family structure. The input information is converted into a digital format by the terminal and prepared as data packets that can be sent to the server. The input for this process is the information entered by the user, and the output is the data packets sent to the server.

[0174] Step 2:

[0175] The terminal sends the processed data packets to the server. The server decodes the received data, verifies its contents, and then stores it in a database. The stored data is used for subsequent processing. In this step, the data transmission by the terminal is the input, and the information recorded in the database is the output.

[0176] Step 3:

[0177] When a user makes a purchase, the device's location services are activated to determine its current location. The device sends location data to a server in real time. The server receives this information and collects inventory information for nearby retail stores via an external API. The input to this process is the user's location information, and the output is the retrieved inventory information.

[0178] Step 4:

[0179] The server integrates the collected inventory information with the user's profile data. Next, a generative AI model is used to generate optimal menu suggestions based on the results of the data integration. This takes into account the user's emotional state data. The input for this step is the integrated profile and inventory information, and the output is the generated menu suggestions.

[0180] Step 5:

[0181] The server receives user voice and facial expression data from the terminal to perform emotion recognition. The server uses emotion recognition technology to analyze the user's current emotional state. The analysis results are reflected in the generated menu. In this step, the input is voice and facial expression data, and the output is the emotion recognition result.

[0182] Step 6:

[0183] The server sends the generated menu suggestions and emotion recognition results to the terminal. The terminal then uses this data to send an audio notification to the user. The notification uses a voice tone and content that is adjusted according to the user's emotions. In this process, the input is data from the server, and the output is the audio notification to the user.

[0184] Step 7:

[0185] The server performs long-term data analysis based on user feedback and past sentiment history. Based on the results of this analysis, more accurate menu suggestions become possible. The input is user feedback and historical data, and the output is improved menu suggestions.

[0186] (Application Example 2)

[0187] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0188] Existing meal plan suggestion systems offer suggestions based on the user's health condition and preferences, but they lack the flexibility to adjust menus to take the user's emotional state into account, which limits the quality of the user experience. Furthermore, in the in-store shopping experience, product recommendations and menu adjustments are not tailored to the user's emotions at the time, resulting in insufficient purchasing support.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0190] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and household composition; means for acquiring product inventory data of nearby sales facilities using location data; means for integrating the information acquired from the data input means with the product inventory data to provide an optimal meal plan; means for detecting the user's emotional state through an emotion analysis function and adjusting the menu suggestion according to that emotional state; and means for providing the user with a selected menu via voice notification, adjusting the tone and content of the voice based on the emotional data. This enables flexible and accurate meal suggestions that respond to the user's emotional state at the time, improving the in-store purchasing experience.

[0191] A "user" is an individual who uses the system to receive meal menu suggestions.

[0192] "Health status" refers to information related to the user's physical and mental health.

[0193] "Food preferences" refer to information about the ingredients, types of dishes, and seasonings that users like.

[0194] "Allergy information" refers to information about foods that may cause an allergic reaction in a user's body when consumed.

[0195] "Family structure" refers to information about the number of family members and their relationships within the user's family.

[0196] "Data input means" refers to the means of collecting information from users and incorporating it into the system.

[0197] "Location data" refers to information about the geographical location of a user or device.

[0198] "Product inventory data" refers to information regarding the inventory status of various products at sales facilities.

[0199] "Emotion analysis function" refers to technology that evaluates a user's emotional state based on their voice, facial expressions, and other factors.

[0200] "Adjusting menu suggestions" refers to the process of modifying meal plans suggested based on the user's emotional state and other information.

[0201] "Voice notification" refers to a method of informing the user of system-generated information through voice.

[0202] The system that realizes this invention is designed to enable users to efficiently plan their daily meals. It consists of a combination of hardware and software, including a server, terminal devices, and an emotion analysis engine.

[0203] The server receives individual data from the terminal device, such as the user's health status, food preferences, allergy information, and family structure. The terminal device is a portable device such as a smartphone, which is equipped with a camera and microphone. This allows for the collection of data to analyze the user's emotional state. The data is processed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text.

[0204] Location data is acquired in real time using GPS functionality, and the server can use this location information to collect product inventory data from nearby retail facilities. This information is integrated and analyzed by an algorithm called a generative AI model to create the optimal menu suggestion for the user.

[0205] The server monitors the user's emotional state through an emotion analysis engine and incorporates that data into menu suggestions. The suggested menu is sent to the terminal device and communicated to the user using voice notifications. Because the tone and content of these voice notifications are adjusted according to the user's emotional state, it is possible to provide more personalized and flexible suggestions.

[0206] For example, if a user is experiencing stress, the server will suggest a menu that includes easily prepared foods with relaxing properties. Conversely, if the server determines that the user has a positive attitude and wants to try something new, it can recommend challenging and novel recipes.

[0207] An example of a prompt is: "What is the user's current emotional state? Based on that, please suggest foods that can alleviate stress. Considering the current location, please list a few items that are easily available." This prompt is used to encourage the generative AI model to suggest foods.

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The user enters their health status, food preferences, allergy information, and household composition into a terminal device. This entered data is then sent directly to the server. The server receives this data and stores it in a database. This process creates a user profile.

[0211] Step 2:

[0212] When a user goes shopping with a terminal device, the device uses GPS functionality to acquire location data. This data is sent to a server, which then uses the location data to collect product inventory information from nearby retail facilities. This allows the server to obtain food information relevant to the user's current location.

[0213] Step 3:

[0214] The system uses the camera and microphone of the terminal device to collect the user's facial expressions and voice data. The collected data is sent to a server and analyzed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text. This allows the user's emotional state to be obtained.

[0215] Step 4:

[0216] The server integrates collected location data, inventory information, user profiles, and emotional data, and uses a generative AI model to generate optimal menu suggestions. This data processing and calculation selects menus that are appropriate for the user's health condition and current emotions.

[0217] Step 5:

[0218] The server sends the generated menu suggestions to the terminal device, which then notifies the user via voice notification. The tone and content of the voice are adjusted to match the user's emotional state, resulting in more personalized suggestions.

[0219] Step 6:

[0220] The user reviews the suggested menu and requests adjustments if necessary. This request is sent to the server, which then uses the generated AI model to propose a new menu. This step allows for flexible menu changes tailored to the user's intentions.

[0221] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0222] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0223] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0224] [Second Embodiment]

[0225] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0226] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0227] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0228] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0229] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0231] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0232] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0233] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0234] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0235] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0236] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0237] The system according to the present invention aims to enable users to efficiently decide on meal menus and promote a healthy lifestyle. The system's processing will be described in detail below based on an example.

[0238] Users first use a terminal application to pre-enter details such as their own and their family's health status, food preferences, allergy information, and family structure. This information is sent from the terminal to the server and stored in a database. Once a user profile is built, it becomes possible to optimize meal plans based on individual needs.

[0239] When a user actually starts shopping at a supermarket, the device acquires the user's location information and sends it to a server. Based on this location information, the server collects inventory information for products offered at nearby retail facilities. Based on the collected data, an AI algorithm generates multiple menu suggestions that take the user's profile into consideration.

[0240] As an example, consider a family of four where the father has high blood pressure, the mother is a vegetarian, and the child prefers meat. In this case, the AI ​​can provide menu suggestions that include low-sodium vegetarian options to accommodate the high blood pressure. For the child, it would then suggest a separate menu that incorporates meat in a balanced way.

[0241] The terminal notifies the user via voice of the menu suggestions received from the server. The user can review the suggestions on the spot and, if necessary, ask questions or request changes to the menu from the AI. For example, in response to a request to adjust the recipe, such as "I'd like to add a few more vegetables to this recipe," the AI ​​will immediately regenerate and re-suggest a new recipe.

[0242] This system responds in real time and flexibly as users choose meals while shopping. This allows users to shop with less stress and provide meals that are optimal for their personal and family health.

[0243] The following describes the processing flow.

[0244] Step 1:

[0245] The user launches the application on their device and enters details such as family health status, dietary preferences, allergy information, and family composition. After completing the input, the device sends this data to the server.

[0246] Step 2:

[0247] The server stores the received user data in a database and builds a profile for each user. This establishes an information infrastructure that can be accessed in real time.

[0248] Step 3:

[0249] When a user arrives at a supermarket and begins shopping, the device acquires their current location information and sends it to the server. Based on this information, the shopping destination is identified.

[0250] Step 4:

[0251] The server queries external databases and APIs to retrieve inventory and special offer information from nearby retail stores based on the user's location. Based on this information, it creates an up-to-date product list.

[0252] Step 5:

[0253] The server uses an AI algorithm to generate optimal menu suggestions based on the user's profile and acquired inventory information. It dynamically creates recipes tailored to the user's health condition and dietary preferences, providing multiple options.

[0254] Step 6:

[0255] The device provides the user with menu suggestions received from the server via voice notification. The user can then review the suggested menu.

[0256] Step 7:

[0257] Users can ask questions or request changes to the suggested menu. User feedback is sent to the server via the terminal.

[0258] Step 8:

[0259] The server regenerates or adjusts the menu in response to adjustment requests from the user. The modified menu is then sent back to the terminal and re-proposed.

[0260] As a result, users can easily select suitable menus while shopping and streamline meal preparation.

[0261] (Example 1)

[0262] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0263] In modern life, efficiently selecting and preparing meal plans while maintaining personal and family health is a crucial challenge for many. However, efficiently shopping while considering health status, food preferences, and allergy information remains difficult. The problem lies in the lack of tools that allow users to dynamically adjust meal plans based on their health information.

[0264] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0265] In this invention, the server includes data collection means for inputting information related to the user's health, means for acquiring information on items in nearby commercial facilities using location information, and means for generating an optimal meal configuration by combining the information from the data collection means and the item information. This enables the user to efficiently select and readjust the optimal meal according to their individual health condition.

[0266] A "user" is someone who uses the system to input information related to meal plan development and receives suggestions based on that information.

[0267] A "data collection method" is an interface that has the function of inputting the user's health status, preferences, and allergy information and sending it to a server.

[0268] "Location information" refers to information that indicates the user's geographical location and is used to obtain information about items in commercial facilities.

[0269] A "commercial facility" refers to a retail environment that sells food and related goods, and is a place where users actually shop.

[0270] "Product information" refers to data that includes information such as the type of food, inventory status, and price at commercial facilities.

[0271] "Meal composition" refers to menu suggestions tailored to the user's health condition and preferences, and includes specific ingredients and cooking methods.

[0272] "Voice notification" is a method of communicating the meal plan generated by the system to the user using voice.

[0273] An "information processing system" is a set of processing mechanisms, including computing resources and software, that integrate user input information with commercial facility item information to propose an optimized meal plan.

[0274] Embodiments of the present invention are described below. First, the user launches a dedicated application using a terminal device. Through the interface of this application, the user inputs information such as their own and their family's health status (e.g., hypertension, diabetes), dietary preferences (e.g., vegetarian, carbohydrate restriction), allergy information (e.g., nut allergy), and family structure (e.g., 2-person family, 4-person family). This information is formatted and encrypted by the terminal and then sent to the server. The server stores this information in a relational database and constructs it as a user profile.

[0275] When a user begins shopping at a commercial facility, their device uses its built-in GPS module to obtain real-time location information. This location information is then sent to a server, which uses the location data to retrieve product information from nearby commercial facilities via publicly available APIs. This product information includes details such as the type of product, price, and availability.

[0276] The server provides stored user profiles and real-time item information as input data to a generating AI model, which then generates an optimal meal plan tailored to the user's health status and preferences. This AI model utilizes deep learning and other machine learning algorithms to suggest customized menus to the user.

[0277] For example, a prompt such as "Please suggest a menu for a family of four, including low-sodium options for hypertension and vegetarian options" is used with the AI ​​model to generate suggestions. The generated menu suggestions are returned to the terminal and communicated to the user via voice notification. The user can review the suggested menu and request changes according to specific needs. These requests might include something like, "I'd like to add more vegetables to this recipe," and the server quickly regenerates the menu using the AI ​​model.

[0278] This system enables users to shop efficiently and choose meals that suit their health condition and preferences.

[0279] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0280] Step 1:

[0281] The user launches a dedicated application on their terminal device and enters personal information such as health status, dietary preferences, allergy information, and family structure. This information is formatted and encrypted by the terminal. The entered data is sent to the server via an HTTP POST request. As output, the user's information is securely transmitted to the server.

[0282] Step 2:

[0283] The server stores the received user information in a relational database. Specifically, it classifies the data for each field and constructs it as a user profile. The input is the personal information from the user, and the output is the organized user profile in the database.

[0284] Step 3:

[0285] When the user arrives at a commercial facility, the GPS function of the terminal automatically obtains the location information. This location information is transmitted from the terminal to the server. The input is the data of the location coordinates, and as the output, the server obtains the current location of the user.

[0286] Step 4:

[0287] Based on the location information, the server obtains item information from nearby commercial facilities using an API. This is done by sending an API request and analyzing the data returned in JSON format. The input is the location information of the user, and the output is the item information data of the commercial facility.

[0288] Step 5:

[0289] The server provides the user profile and item information as input data to the AI model generated based on them, and generates an optimal diet composition. The AI model operates based on deep learning algorithms and combines this information to generate a proposal. The user information and item information are used as input, and an optimized diet plan is generated as the output.

[0290] Step 6:

[0291] The generated diet plan is transmitted from the server to the terminal. The terminal uses voice synthesis technology to notify the user of the diet plan by voice. The input is the data of the diet plan, and the output is the voice notification to the user.

[0292] Step 7:

[0293] Users can review menu suggestions and request specific changes. For example, they can request the addition or modification of certain ingredients. This request is sent from the terminal to the server. The input is the user's additional request, and the server regenerates and outputs the revised menu suggestion.

[0294] (Application Example 1)

[0295] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0296] In modern society, it is difficult to plan meals that take into account individual health conditions and dietary preferences on a daily basis, and users face many challenges in maintaining their own health and preparing meals that suit their family structure. Furthermore, it is not easy to immediately select the optimal menu while shopping at physical stores, taking inventory information into account. In this situation, there is a need for a system that can flexibly respond to the diverse needs of users.

[0297] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0298] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and family structure; means for collecting inventory information of goods at nearby commercial facilities based on location information; means for integrating the information obtained from the data input means with the inventory information of the goods to generate an optimal meal menu; means for presenting the generated menu to the user through voice guidance; and regeneration means using a generative AI model to adjust and re-suggest the generated menu using prompt sentences based on the user's request. This allows the user to dynamically decide on a menu even while shopping, and enables the provision of optimal meals tailored to individual health needs and family needs.

[0299] A "user" refers to an information user, specifically an individual or household member who receives meal menu suggestions from the system.

[0300] The "data input means" is a device or software for inputting the user's health status, food preferences, allergy information, family composition, etc.

[0301] The "location information" is geographical data indicating the location where the user is currently located.

[0302] The "commercial facility" refers to a physical store that sells goods and is a place where users go shopping.

[0303] The "inventory information of goods" is information including the quantity and type of goods that can be sold in a commercial facility.

[0304] "Integration" is a process of combining different types of information to give meaning and create new value.

[0305] The "menu" refers to a specific meal plan proposed and is composed of a combination of multiple dishes and ingredients.

[0306] The "voice guidance" is a means for the system to convey information to the user by voice. The "prompt text" is a text that describes requests or instructions for the generation AI model.

[0307]

[0308] The "generation AI model" is a type of artificial intelligence that performs data analysis and logical inference in response to user requests and generates new menus and information.

[0309] The "regeneration means" is a function within the system for readjusting an existing menu based on additional user requests and making new proposals.

[0310] This system integrates information from user terminals, servers, and commercial facilities to suggest menus that support a healthy lifestyle. Users input their health status, food preferences, allergy information, and family structure via their terminals, and this information is sent to the server and stored in a database. The server uses this information to build a user profile.

[0311] When a user enters a commercial facility, their device's location information is acquired and sent to a server. Based on this location information, the server collects inventory information for items in surrounding commercial facilities. This inventory information is collected using APIs and real-time data feeds provided by the commercial facilities. The server integrates the user's profile with the commercial facilities' inventory information and generates an optimal meal menu using an AI model.

[0312] The generated menu is presented to the user via voice guidance through the terminal. The user can then make requests to the system using prompts based on the voice guidance. For example, they can request, "I'd like an option to add more vegetables to this menu." The generation AI model immediately regenerates the menu based on the prompt and provides the guidance again through the terminal.

[0313] As a concrete example, a user might ask the system to "provide vegetarian and low-calorie menu suggestions to invite a friend over." In this case, a possible prompt to the generative AI model might be something like, "Please suggest a 3-item vegetarian and low-calorie menu." Upon receiving this prompt, the generative AI model would make adjustments according to the user's request and provide the user with a new menu suggestion.

[0314] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0315] Step 1:

[0316] Users operate a terminal to input their health status, food preferences, allergy information, and family structure. This input information is sent from the terminal to the server. The server stores the received information in a database and builds a user profile. This profile is used as the basis for subsequent menu generation.

[0317] Step 2:

[0318] When a user enters a commercial facility, the terminal acquires the user's location information. This location information is transmitted to the server in real time, and the user's current location is identified. Based on this location information, the server collects inventory information for items in the surrounding commercial facilities via APIs, etc. The output of this process is a list of items actually on the shelves.

[0319] Step 3:

[0320] The server uses the user's profile and acquired inventory information to generate an optimal meal menu using a generative AI model. This process takes into account the user's health goals, preferences, and allergy information, while selecting items from the available inventory. The generated menu suggestions are sent to the terminal as server output.

[0321] Step 4:

[0322] The terminal presents the received menu suggestions to the user via voice guidance. The user listens to the voice guidance and can make additional requests or adjustments to the system using prompt messages. The prompt messages entered at this time are sent to the server through the terminal.

[0323] Step 5:

[0324] The server uses the prompt message received from the user to regenerate the menu using a generation AI model. Based on the content of the prompt message, the data is reprocessed to generate new menu suggestions. These regenerated menu suggestions are then output from the server to the terminal.

[0325] Step 6:

[0326] The terminal then provides the user with new menu suggestions via voice guidance. Based on this information, the user can proceed with their shopping and select ingredients that best suit their and their family's needs. This entire process enables the user to engage in healthy and personalized shopping.

[0327] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0328] This invention relates to a system equipped with an emotion recognition engine that recognizes the user's emotions and optimizes meal menu suggestions based on those emotions. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure into a terminal, and this information is transmitted to a server and stored in a database.

[0329] The system sends location information from the user's device to the server when they make a purchase. The server uses this location information to retrieve inventory information from nearby stores, integrates it with the user profile, and uses an AI algorithm to generate optimal menu suggestions. In addition, it is equipped with an emotion recognition engine that analyzes the user's voice and facial expressions to recognize their current emotional state.

[0330] For example, if a user looks anxious while shopping, the emotion engine will detect that emotion. The server will take this emotional information into consideration and prioritize suggesting familiar dishes or easy-to-prepare recipes to reassure the user. Conversely, if joy or excitement is detected, it can suggest new recipes or challenging dishes.

[0331] The device notifies the user of menu suggestions received from the server via voice, adjusting the tone and content of the voice based on emotional data. For example, a relaxed tone of voice can support the user, and encouraging words can be added to enhance the effectiveness of the suggestions.

[0332] Furthermore, past emotional history is recorded in a database, and the server uses this information to analyze long-term trends and utilize it to suggest menus that are a better match for the user. In this way, the system realizes flexible and accurate meal suggestions that are attentive to the user's emotions, improving the user's shopping experience and health management.

[0333] The following describes the processing flow.

[0334] Step 1:

[0335] Users use an application on their device to input their own and their family's health status, dietary preferences, allergy information, and family structure. The device then sends this information to a server and stores it in a database.

[0336] Step 2:

[0337] The device collects emotional data from the user's voice and facial expressions. Using an emotion engine, it analyzes the collected data to recognize the user's current emotional state.

[0338] Step 3:

[0339] When a user starts shopping at a supermarket, the device sends its location information to a server. Based on this location information, the server retrieves inventory information for nearby retail stores.

[0340] Step 4:

[0341] The server integrates user information stored in the database, inventory information for the current location, and emotional data recognized by the emotion engine, and uses an AI algorithm to generate the optimal menu suggestion.

[0342] Step 5:

[0343] The server sends the generated menu suggestions to the terminal. The terminal presents these suggestions to the user via voice notification. At this time, the voice tone and message content are adjusted based on emotion data.

[0344] Step 6:

[0345] The user reviews the suggested menu and provides feedback or requests for changes via their device as needed. The device then sends this request to the server.

[0346] Step 7:

[0347] The server readjusts the menu based on user change requests and feedback. If necessary, it sends the regenerated menu to the terminal and proposes it again to the user.

[0348] Step 8:

[0349] The server records emotional data collected during each shopping session in a database and analyzes past emotional history. This information is used to make future suggestions and helps provide more personalized meal recommendations to the user.

[0350] (Example 2)

[0351] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0352] In today's lifestyle, choosing meals that suit individual health conditions and dietary preferences is crucial, but effective meal planning is challenging. Furthermore, suggesting meals that align with a user's emotional state is more complex than simply considering nutritional balance. Additionally, systems that enable flexible, real-time meal planning during shopping outside the home are limited, and improvement is needed.

[0353] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0354] In this invention, the server includes an input means for inputting the user's health status, food preferences, allergy information, and family structure; an acquisition means for acquiring inventory information of nearby sales locations based on location information; and a suggestion means for integrating the data acquired from the input means with the inventory information and proposing an optimal meal plan using a generative model. This enables flexible menu suggestions that are optimized for each user's profile and respond to their emotional state.

[0355] "Input means" refers to a device or software for which a user inputs information such as their health status, food preferences, allergy information, and family structure.

[0356] "Acquisition method" refers to the function or process for obtaining inventory information from nearby sales locations based on the user's location information.

[0357] The "proposal method" is a function that integrates acquired data and uses a generative model to present the most suitable meal plan for the user.

[0358] "Emotion recognition means" refers to a technology or device that analyzes a user's voice and facial expression data to identify the user's emotional state.

[0359] The "notification method" refers to a function that notifies the user of the generated menu plan via voice, and adjusts the voice tone and content according to emotional data.

[0360] This invention is a system for optimizing meal menu suggestions by taking user emotions into consideration. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure using a terminal. The terminal processes the collected data and transmits it to a server. The server stores the received information in a database and manages it appropriately.

[0361] Furthermore, when a user begins shopping, the system sends location information to a server via their device. The server uses the location information to retrieve inventory information from nearby retail stores and integrates it with the provided profile information. The server then uses an emotion recognition engine to analyze the user's voice and facial expressions to recognize their current emotional state. This emotional data influences menu suggestions and forms the basis for generating suggestions that match the user's emotions.

[0362] For example, if a user appears anxious, the server will suggest comforting, familiar dishes or easy-to-make recipes. Conversely, if a user expresses joy or excitement, it will suggest new recipes or challenging dishes. The generated menu suggestions are sent to the device and communicated to the user via voice notification. The notification may include a voice tone and encouraging message tailored to the user's emotions.

[0363] The server uses historical sentiment data to analyze long-term data trends. This analysis enables the system to suggest menus that are more suitable for the user.

[0364] As a concrete example, one could input a prompt message into the AI ​​model such as, "Please suggest a healthy dinner recipe that can be prepared quickly, especially when the user is tired from work." Such a prompt would allow the system to provide suggestions tailored to the user's state.

[0365] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0366] Step 1:

[0367] The user uses a terminal to input their health status, dietary preferences, allergy information, and family structure. The input information is converted into a digital format by the terminal and prepared as data packets that can be sent to the server. The input for this process is the information entered by the user, and the output is the data packets sent to the server.

[0368] Step 2:

[0369] The terminal sends the processed data packets to the server. The server decodes the received data, verifies its contents, and then stores it in a database. The stored data is used for subsequent processing. In this step, the data transmission by the terminal is the input, and the information recorded in the database is the output.

[0370] Step 3:

[0371] When a user makes a purchase, the device's location services are activated to determine its current location. The device sends location data to a server in real time. The server receives this information and collects inventory information for nearby retail stores via an external API. The input to this process is the user's location information, and the output is the retrieved inventory information.

[0372] Step 4:

[0373] The server integrates the collected inventory information with the user's profile data. Next, a generative AI model is used to generate optimal menu suggestions based on the results of the data integration. This takes into account the user's emotional state data. The input for this step is the integrated profile and inventory information, and the output is the generated menu suggestions.

[0374] Step 5:

[0375] The server receives user voice and facial expression data from the terminal to perform emotion recognition. The server uses emotion recognition technology to analyze the user's current emotional state. The analysis results are reflected in the generated menu. In this step, the input is voice and facial expression data, and the output is the emotion recognition result.

[0376] Step 6:

[0377] The server sends the generated menu suggestions and emotion recognition results to the terminal. The terminal then uses this data to send an audio notification to the user. The notification uses a voice tone and content that is adjusted according to the user's emotions. In this process, the input is data from the server, and the output is the audio notification to the user.

[0378] Step 7:

[0379] The server performs long-term data analysis based on user feedback and past sentiment history. Based on the results of this analysis, more accurate menu suggestions become possible. The input is user feedback and historical data, and the output is improved menu suggestions.

[0380] (Application Example 2)

[0381] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0382] Existing meal plan suggestion systems offer suggestions based on the user's health condition and preferences, but they lack the flexibility to adjust menus to take the user's emotional state into account, which limits the quality of the user experience. Furthermore, in the in-store shopping experience, product recommendations and menu adjustments are not tailored to the user's emotions at the time, resulting in insufficient purchasing support.

[0383] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0384] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and household composition; means for acquiring product inventory data of nearby sales facilities using location data; means for integrating the information acquired from the data input means with the product inventory data to provide an optimal meal plan; means for detecting the user's emotional state through an emotion analysis function and adjusting the menu suggestion according to that emotional state; and means for providing the user with a selected menu via voice notification, adjusting the tone and content of the voice based on the emotional data. This enables flexible and accurate meal suggestions that respond to the user's emotional state at the time, improving the in-store purchasing experience.

[0385] A "user" is an individual who uses the system to receive meal menu suggestions.

[0386] "Health status" refers to information related to the user's physical and mental health.

[0387] "Food preferences" refer to information about the ingredients, types of dishes, and seasonings that users like.

[0388] "Allergy information" refers to information about foods that may cause an allergic reaction in a user's body when consumed.

[0389] "Family structure" refers to information about the number of family members and their relationships within the user's family.

[0390] "Data input means" refers to the means of collecting information from users and incorporating it into the system.

[0391] "Location data" refers to information about the geographical location of a user or device.

[0392] "Product inventory data" refers to information regarding the inventory status of various products at sales facilities.

[0393] "Emotion analysis function" refers to technology that evaluates a user's emotional state based on their voice, facial expressions, and other factors.

[0394] "Adjusting menu suggestions" refers to the process of modifying meal plans suggested based on the user's emotional state and other information.

[0395] "Voice notification" refers to a method of informing the user of system-generated information through voice.

[0396] The system that realizes this invention is designed to enable users to efficiently plan their daily meals. It consists of a combination of hardware and software, including a server, terminal devices, and an emotion analysis engine.

[0397] The server receives individual data from the terminal device, such as the user's health status, food preferences, allergy information, and family structure. The terminal device is a portable device such as a smartphone, equipped with a camera and microphone. This allows for the collection of data to analyze the user's emotional state. The data is processed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text.

[0398] Location data is acquired in real time using GPS functionality, and the server can use this location information to collect product inventory data from nearby retail facilities. This information is integrated and analyzed by an algorithm called a generative AI model to create the optimal menu suggestion for the user.

[0399] The server monitors the user's emotional state through an emotion analysis engine and incorporates that data into menu suggestions. The suggested menu is sent to the terminal device and communicated to the user using voice notifications. Because the tone and content of these voice notifications are adjusted according to the user's emotional state, it is possible to provide more personalized and flexible suggestions.

[0400] For example, if a user is experiencing stress, the server will suggest a menu that includes easily prepared foods with relaxing properties. Conversely, if the server determines that the user has a positive attitude and wants to try something new, it can recommend challenging and novel recipes.

[0401] An example of a prompt is: "What is the user's current emotional state? Based on that, please suggest foods that can alleviate stress. Considering the current location, please list a few items that are easily available." This prompt is used to encourage the generative AI model to suggest foods.

[0402] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0403] Step 1:

[0404] The user enters their health status, food preferences, allergy information, and household composition into a terminal device. This entered data is then sent directly to the server. The server receives this data and stores it in a database. This process creates a user profile.

[0405] Step 2:

[0406] When a user goes shopping with a terminal device, the device uses GPS functionality to acquire location data. This data is sent to a server, which then uses the location data to collect product inventory information from nearby retail facilities. This allows the server to obtain food information relevant to the user's current location.

[0407] Step 3:

[0408] The system uses the camera and microphone of the terminal device to collect the user's facial expressions and voice data. The collected data is sent to a server and analyzed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text. This allows the user's emotional state to be obtained.

[0409] Step 4:

[0410] The server integrates collected location data, inventory information, user profiles, and emotional data, and uses a generative AI model to generate optimal menu suggestions. This data processing and calculation selects menus that are appropriate for the user's health condition and current emotions.

[0411] Step 5:

[0412] The server sends the generated menu suggestions to the terminal device, which then notifies the user via voice notification. The tone and content of the voice are adjusted to match the user's emotional state, resulting in more personalized suggestions.

[0413] Step 6:

[0414] The user reviews the suggested menu and requests adjustments if necessary. This request is sent to the server, which then uses the generated AI model to propose a new menu. This step allows for flexible menu changes tailored to the user's intentions.

[0415] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0416] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0417] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0418] [Third Embodiment]

[0419] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0420] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0421] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0422] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0423] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0424] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0425] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0426] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0427] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0428] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0429] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0430] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0431] The system according to the present invention aims to enable users to efficiently decide on meal menus and promote a healthy lifestyle. The system's processing will be described in detail below based on an example.

[0432] Users first use a terminal application to pre-enter details such as their own and their family's health status, food preferences, allergy information, and family structure. This information is sent from the terminal to the server and stored in a database. Once a user profile is built, it becomes possible to optimize meal plans based on individual needs.

[0433] When a user actually starts shopping at a supermarket, the device acquires the user's location information and sends it to a server. Based on this location information, the server collects inventory information for products offered at nearby retail facilities. Based on the collected data, an AI algorithm generates multiple menu suggestions that take the user's profile into consideration.

[0434] As an example, consider a family of four where the father has high blood pressure, the mother is a vegetarian, and the child prefers meat. In this case, the AI ​​can provide menu suggestions that include low-sodium vegetarian options to accommodate the high blood pressure. For the child, it would then suggest a separate menu that incorporates meat in a balanced way.

[0435] The terminal notifies the user via voice of the menu suggestions received from the server. The user can review the suggestions on the spot and, if necessary, ask questions or request changes to the menu from the AI. For example, in response to a request to adjust the recipe, such as "I'd like to add a few more vegetables to this recipe," the AI ​​will immediately regenerate and re-suggest a new recipe.

[0436] This system responds in real time and flexibly as users choose meals while shopping. This allows users to shop with less stress and provide meals that are optimal for their personal and family health.

[0437] The following describes the processing flow.

[0438] Step 1:

[0439] The user launches the application on their device and enters details such as family health status, dietary preferences, allergy information, and family composition. After completing the input, the device sends this data to the server.

[0440] Step 2:

[0441] The server stores the received user data in a database and builds a profile for each user. This establishes an information infrastructure that can be accessed in real time.

[0442] Step 3:

[0443] When a user arrives at a supermarket and begins shopping, the device acquires their current location information and sends it to the server. Based on this information, the shopping destination is identified.

[0444] Step 4:

[0445] The server queries external databases and APIs to retrieve inventory and special offer information from nearby retail stores based on the user's location. Based on this information, it creates an up-to-date product list.

[0446] Step 5:

[0447] The server uses an AI algorithm to generate optimal menu suggestions based on the user's profile and acquired inventory information. It dynamically creates recipes tailored to the user's health condition and dietary preferences, providing multiple options.

[0448] Step 6:

[0449] The device provides the user with menu suggestions received from the server via voice notification. The user can then review the suggested menu.

[0450] Step 7:

[0451] Users can ask questions or request changes to the suggested menu. User feedback is sent to the server via the terminal.

[0452] Step 8:

[0453] The server regenerates or adjusts the menu in response to adjustment requests from the user. The modified menu is then sent back to the terminal and re-proposed.

[0454] As a result, users can easily select suitable menus while shopping and streamline meal preparation.

[0455] (Example 1)

[0456] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0457] In modern life, efficiently selecting and preparing meal plans while maintaining personal and family health is a crucial challenge for many. However, efficiently shopping while considering health status, food preferences, and allergy information remains difficult. The problem lies in the lack of tools that allow users to dynamically adjust meal plans based on their health information.

[0458] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0459] In this invention, the server includes data collection means for inputting information related to the user's health, means for acquiring information on items in nearby commercial facilities using location information, and means for generating an optimal meal configuration by combining the information from the data collection means and the item information. This enables the user to efficiently select and readjust the optimal meal according to their individual health condition.

[0460] A "user" is someone who uses the system to input information related to meal plan development and receives suggestions based on that information.

[0461] A "data collection method" is an interface that has the function of inputting the user's health status, preferences, and allergy information and sending it to a server.

[0462] "Location information" refers to information that indicates the user's geographical location and is used to obtain information about items in commercial facilities.

[0463] A "commercial facility" refers to a retail environment that sells food and related goods, and is a place where users actually shop.

[0464] "Product information" refers to data that includes information such as the type of food, inventory status, and price at commercial facilities.

[0465] "Meal composition" refers to menu suggestions tailored to the user's health condition and preferences, and includes specific ingredients and cooking methods.

[0466] "Voice notification" is a method of communicating the meal plan generated by the system to the user using voice.

[0467] An "information processing system" is a set of processing mechanisms, including computing resources and software, that integrate user input information with commercial facility item information to propose an optimized meal plan.

[0468] Embodiments of the present invention are described below. First, the user launches a dedicated application using a terminal device. Through the interface of this application, the user inputs information such as their own and their family's health status (e.g., hypertension, diabetes), dietary preferences (e.g., vegetarian, carbohydrate restriction), allergy information (e.g., nut allergy), and family structure (e.g., 2-person family, 4-person family). This information is formatted and encrypted by the terminal and then sent to the server. The server stores this information in a relational database and constructs it as a user profile.

[0469] When a user begins shopping at a commercial facility, their device uses its built-in GPS module to obtain real-time location information. This location information is then sent to a server, which uses the location data to retrieve product information from nearby commercial facilities via publicly available APIs. This product information includes details such as the type of product, price, and availability.

[0470] The server provides stored user profiles and real-time item information as input data to a generating AI model, which then generates an optimal meal plan tailored to the user's health status and preferences. This AI model utilizes deep learning and other machine learning algorithms to suggest customized menus to the user.

[0471] For example, a prompt such as "Please suggest a menu for a family of four, including low-sodium options for hypertension and vegetarian options" is used with the AI ​​model to generate suggestions. The generated menu suggestions are returned to the terminal and communicated to the user via voice notification. The user can review the suggested menu and request changes according to specific needs. These requests might include something like, "I'd like to add more vegetables to this recipe," and the server quickly regenerates the menu using the AI ​​model.

[0472] This system enables users to shop efficiently and choose meals that suit their health condition and preferences.

[0473] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0474] Step 1:

[0475] The user launches a dedicated application on their terminal device and enters personal information such as health status, dietary preferences, allergy information, and family structure. This information is formatted and encrypted by the terminal. The entered data is sent to the server via an HTTP POST request. As output, the user's information is securely transmitted to the server.

[0476] Step 2:

[0477] The server stores the received user information in a relational database. Specifically, it categorizes the data by field and constructs it as a user profile. The input is personal information from the user, and the output is an organized user profile in the database.

[0478] Step 3:

[0479] When a user arrives at a commercial facility, the device's GPS function automatically acquires location information. This location information is sent from the device to the server. The input is location coordinate data, and the server obtains the user's current location as output.

[0480] Step 4:

[0481] The server uses location information to retrieve item information from nearby commercial facilities using APIs. This is done by sending API requests and parsing the data returned in JSON format. The input is the user's location information, and the output is item information data from the commercial facilities.

[0482] Step 5:

[0483] The server provides user profiles and item information as input data to a generating AI model, which then generates an optimal meal plan. The AI ​​model operates based on a deep learning algorithm, combining this information to generate suggestions. User information and item information are used as input, and an optimized menu plan is generated as output.

[0484] Step 6:

[0485] The generated menu plan is sent from the server to the terminal. The terminal uses speech synthesis technology to notify the user of the menu plan by voice. The input is the menu plan data, and the output is the voice notification to the user.

[0486] Step 7:

[0487] Users can review menu suggestions and request specific changes. For example, they can request the addition or modification of certain ingredients. This request is sent from the terminal to the server. The input is the user's additional request, and the server regenerates and outputs the revised menu suggestion.

[0488] (Application Example 1)

[0489] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0490] In modern society, it is difficult to plan meals that take into account individual health conditions and dietary preferences on a daily basis, and users face many challenges in maintaining their own health and preparing meals that suit their family structure. Furthermore, it is not easy to immediately select the optimal menu while shopping at physical stores, taking inventory information into account. In this situation, there is a need for a system that can flexibly respond to the diverse needs of users.

[0491] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0492] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and family structure; means for collecting inventory information of goods at nearby commercial facilities based on location information; means for integrating the information obtained from the data input means with the inventory information of the goods to generate an optimal meal menu; means for presenting the generated menu to the user through voice guidance; and regeneration means using a generative AI model to adjust and re-suggest the generated menu using prompt sentences based on the user's request. This allows the user to dynamically decide on a menu even while shopping, and enables the provision of optimal meals tailored to individual health needs and family needs.

[0493] A "user" refers to an information user, specifically an individual or household member who receives meal menu suggestions from the system.

[0494] "Data input means" refers to a device or software for inputting information such as the user's health status, food preferences, allergy information, and family structure.

[0495] "Location information" refers to geographical data that indicates the user's current location.

[0496] A "commercial facility" refers to a physical store that sells goods, a place where users shop.

[0497] "Inventory information for goods" refers to information including the quantity and types of goods available for sale at a commercial facility.

[0498] "Integration" is the process of combining different types of information to give them meaning and create new value.

[0499] A "menu" refers to a specific meal plan that is proposed and consists of a combination of multiple dishes and ingredients.

[0500] "Voice guidance" is a method of conveying information from a system to a user via voice.

[0501] A "prompt statement" is a text statement that describes requests or instructions for a generative AI model.

[0502] A "generative AI model" is a type of artificial intelligence that performs data analysis and logical reasoning in response to user requests to generate new menus and information.

[0503] A "regeneration mechanism" is a system function that readjusts existing menus based on additional user requests and makes new suggestions.

[0504] This system integrates information from user terminals, servers, and commercial facilities to suggest menus that support a healthy lifestyle. Users input their health status, food preferences, allergy information, and family structure via their terminals, and this information is sent to the server and stored in a database. The server uses this information to build a user profile.

[0505] When a user enters a commercial facility, their device's location information is acquired and sent to a server. Based on this location information, the server collects inventory information for items in surrounding commercial facilities. This inventory information is collected using APIs and real-time data feeds provided by the commercial facilities. The server integrates the user's profile with the commercial facilities' inventory information and generates an optimal meal menu using an AI model.

[0506] The generated menu is presented to the user via voice guidance through the terminal. The user can then make requests to the system using prompts based on the voice guidance. For example, they can request, "I'd like an option to add more vegetables to this menu." The generation AI model immediately regenerates the menu based on the prompt and provides the guidance again through the terminal.

[0507] As a concrete example, a user might ask the system to "provide vegetarian and low-calorie menu suggestions to invite a friend over." In this case, a possible prompt to the generative AI model might be something like, "Please suggest a 3-item vegetarian and low-calorie menu." Upon receiving this prompt, the generative AI model would make adjustments according to the user's request and provide the user with a new menu suggestion.

[0508] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0509] Step 1:

[0510] Users operate a terminal to input their health status, food preferences, allergy information, and family structure. This input information is sent from the terminal to the server. The server stores the received information in a database and builds a user profile. This profile is used as the basis for subsequent menu generation.

[0511] Step 2:

[0512] When a user enters a commercial facility, the terminal acquires the user's location information. This location information is transmitted to the server in real time, and the user's current location is identified. Based on this location information, the server collects inventory information for items in the surrounding commercial facilities via APIs, etc. The output of this process is a list of items actually on the shelves.

[0513] Step 3:

[0514] The server uses the user's profile and acquired inventory information to generate an optimal meal menu using a generative AI model. This process takes into account the user's health goals, preferences, and allergy information, while selecting items from the available inventory. The generated menu suggestions are sent to the terminal as server output.

[0515] Step 4:

[0516] The terminal presents the received menu suggestions to the user via voice guidance. The user listens to the voice guidance and can make additional requests or adjustments to the system using prompt messages. The prompt messages entered at this time are sent to the server through the terminal.

[0517] Step 5:

[0518] The server uses the prompt message received from the user to regenerate the menu using a generation AI model. Based on the content of the prompt message, the data is reprocessed to generate new menu suggestions. These regenerated menu suggestions are then output from the server to the terminal.

[0519] Step 6:

[0520] The terminal then provides the user with new menu suggestions via voice guidance. Based on this information, the user can proceed with their shopping and select ingredients that best suit their and their family's needs. This entire process enables the user to engage in healthy and personalized shopping.

[0521] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0522] This invention relates to a system equipped with an emotion recognition engine that recognizes the user's emotions and optimizes meal menu suggestions based on those emotions. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure into a terminal, and this information is transmitted to a server and stored in a database.

[0523] The system sends location information from the user's device to the server when they make a purchase. The server uses this location information to retrieve inventory information from nearby stores, integrates it with the user profile, and uses an AI algorithm to generate optimal menu suggestions. In addition, it is equipped with an emotion recognition engine that analyzes the user's voice and facial expressions to recognize their current emotional state.

[0524] For example, if a user looks anxious while shopping, the emotion engine will detect that emotion. The server will take this emotional information into consideration and prioritize suggesting familiar dishes or easy-to-prepare recipes to reassure the user. Conversely, if joy or excitement is detected, it can suggest new recipes or challenging dishes.

[0525] The device notifies the user of menu suggestions received from the server via voice, adjusting the tone and content of the voice based on emotional data. For example, a relaxed tone of voice can support the user, and encouraging words can be added to enhance the effectiveness of the suggestions.

[0526] Furthermore, past emotional history is recorded in a database, and the server uses this information to analyze long-term trends and utilize it to suggest menus that are a better match for the user. In this way, the system realizes flexible and accurate meal suggestions that are attentive to the user's emotions, improving the user's shopping experience and health management.

[0527] The following describes the processing flow.

[0528] Step 1:

[0529] Users use an application on their device to input their own and their family's health status, dietary preferences, allergy information, and family structure. The device then sends this information to a server and stores it in a database.

[0530] Step 2:

[0531] The device collects emotional data from the user's voice and facial expressions. Using an emotion engine, it analyzes the collected data to recognize the user's current emotional state.

[0532] Step 3:

[0533] When a user starts shopping at a supermarket, the device sends its location information to a server. Based on this location information, the server retrieves inventory information for nearby retail stores.

[0534] Step 4:

[0535] The server integrates user information stored in the database, inventory information for the current location, and emotional data recognized by the emotion engine, and uses an AI algorithm to generate the optimal menu suggestion.

[0536] Step 5:

[0537] The server sends the generated menu suggestions to the terminal. The terminal presents these suggestions to the user via voice notification. At this time, the voice tone and message content are adjusted based on emotion data.

[0538] Step 6:

[0539] The user reviews the suggested menu and provides feedback or requests for changes via their device as needed. The device then sends this request to the server.

[0540] Step 7:

[0541] The server readjusts the menu based on user change requests and feedback. If necessary, it sends the regenerated menu to the terminal and proposes it again to the user.

[0542] Step 8:

[0543] The server records emotional data collected during each shopping session in a database and analyzes past emotional history. This information is used to make future suggestions and helps provide more personalized meal recommendations to the user.

[0544] (Example 2)

[0545] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0546] In today's lifestyle, choosing meals that suit individual health conditions and dietary preferences is crucial, but effective meal planning is challenging. Furthermore, suggesting meals that align with a user's emotional state is more complex than simply considering nutritional balance. Additionally, systems that enable flexible, real-time meal planning during shopping outside the home are limited, and improvement is needed.

[0547] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0548] In this invention, the server includes an input means for inputting the user's health status, food preferences, allergy information, and family structure; an acquisition means for acquiring inventory information of nearby sales locations based on location information; and a suggestion means for integrating the data acquired from the input means with the inventory information and proposing an optimal meal plan using a generative model. This enables flexible menu suggestions that are optimized for each user's profile and respond to their emotional state.

[0549] "Input means" refers to a device or software for which a user inputs information such as their health status, food preferences, allergy information, and family structure.

[0550] "Acquisition method" refers to the function or process for obtaining inventory information from nearby sales locations based on the user's location information.

[0551] The "proposal method" is a function that integrates acquired data and uses a generative model to present the most suitable meal plan for the user.

[0552] "Emotion recognition means" refers to a technology or device that analyzes a user's voice and facial expression data to identify the user's emotional state.

[0553] The "notification method" refers to a function that notifies the user of the generated menu plan via voice, and adjusts the voice tone and content according to emotional data.

[0554] This invention is a system for optimizing meal menu suggestions by taking user emotions into consideration. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure using a terminal. The terminal processes the collected data and transmits it to a server. The server stores the received information in a database and manages it appropriately.

[0555] Furthermore, when a user begins shopping, the system sends location information to a server via their device. The server uses the location information to retrieve inventory information from nearby retail stores and integrates it with the provided profile information. The server then uses an emotion recognition engine to analyze the user's voice and facial expressions to recognize their current emotional state. This emotional data influences menu suggestions and forms the basis for generating suggestions that match the user's emotions.

[0556] For example, if a user appears anxious, the server will suggest comforting, familiar dishes or easy-to-make recipes. Conversely, if a user expresses joy or excitement, it will suggest new recipes or challenging dishes. The generated menu suggestions are sent to the device and communicated to the user via voice notification. The notification may include a voice tone and encouraging message tailored to the user's emotions.

[0557] The server uses historical sentiment data to analyze long-term data trends. This analysis enables the system to suggest menus that are more suitable for the user.

[0558] As a concrete example, one could input a prompt message into the AI ​​model such as, "Please suggest a healthy dinner recipe that can be prepared quickly, especially when the user is tired from work." Such a prompt would allow the system to provide suggestions tailored to the user's state.

[0559] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0560] Step 1:

[0561] The user uses a terminal to input their health status, dietary preferences, allergy information, and family structure. The input information is converted into a digital format by the terminal and prepared as data packets that can be sent to the server. The input for this process is the information entered by the user, and the output is the data packets sent to the server.

[0562] Step 2:

[0563] The terminal sends the processed data packets to the server. The server decodes the received data, verifies its contents, and then stores it in a database. The stored data is used for subsequent processing. In this step, the data transmission by the terminal is the input, and the information recorded in the database is the output.

[0564] Step 3:

[0565] When a user makes a purchase, the device's location services are activated to determine its current location. The device sends location data to a server in real time. The server receives this information and collects inventory information for nearby retail stores via an external API. The input to this process is the user's location information, and the output is the retrieved inventory information.

[0566] Step 4:

[0567] The server integrates the collected inventory information with the user's profile data. Next, a generative AI model is used to generate optimal menu suggestions based on the results of the data integration. This takes into account the user's emotional state data. The input for this step is the integrated profile and inventory information, and the output is the generated menu suggestions.

[0568] Step 5:

[0569] The server receives user voice and facial expression data from the terminal to perform emotion recognition. The server uses emotion recognition technology to analyze the user's current emotional state. The analysis results are reflected in the generated menu. In this step, the input is voice and facial expression data, and the output is the emotion recognition result.

[0570] Step 6:

[0571] The server sends the generated menu suggestions and emotion recognition results to the terminal. The terminal then uses this data to send an audio notification to the user. The notification uses a voice tone and content that is adjusted according to the user's emotions. In this process, the input is data from the server, and the output is the audio notification to the user.

[0572] Step 7:

[0573] The server performs long-term data analysis based on user feedback and past sentiment history. Based on the results of this analysis, more accurate menu suggestions become possible. The input is user feedback and historical data, and the output is improved menu suggestions.

[0574] (Application Example 2)

[0575] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0576] Existing meal plan suggestion systems offer suggestions based on the user's health condition and preferences, but they lack the flexibility to adjust menus to take the user's emotional state into account, which limits the quality of the user experience. Furthermore, in the in-store shopping experience, product recommendations and menu adjustments are not tailored to the user's emotions at the time, resulting in insufficient purchasing support.

[0577] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0578] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and household composition; means for acquiring product inventory data of nearby sales facilities using location data; means for integrating the information acquired from the data input means with the product inventory data to provide an optimal meal plan; means for detecting the user's emotional state through an emotion analysis function and adjusting the menu suggestion according to that emotional state; and means for providing the user with a selected menu via voice notification, adjusting the tone and content of the voice based on the emotional data. This enables flexible and accurate meal suggestions that respond to the user's emotional state at the time, improving the in-store purchasing experience.

[0579] A "user" is an individual who uses the system to receive meal menu suggestions.

[0580] "Health status" refers to information related to the user's physical and mental health.

[0581] "Food preferences" refer to information about the ingredients, types of dishes, and seasonings that users like.

[0582] "Allergy information" refers to information about foods that may cause an allergic reaction in a user's body when consumed.

[0583] "Family structure" refers to information about the number of family members and their relationships within the user's family.

[0584] "Data input means" refers to the means of collecting information from users and incorporating it into the system.

[0585] "Location data" refers to information about the geographical location of a user or device.

[0586] "Product inventory data" refers to information regarding the inventory status of various products at sales facilities.

[0587] "Emotion analysis function" refers to technology that evaluates a user's emotional state based on their voice, facial expressions, and other factors.

[0588] "Adjusting menu suggestions" refers to the process of modifying meal plans suggested based on the user's emotional state and other information.

[0589] "Voice notification" refers to a method of informing the user of system-generated information through voice.

[0590] The system that realizes this invention is designed to enable users to efficiently plan their daily meals. It consists of a combination of hardware and software, including a server, terminal devices, and an emotion analysis engine.

[0591] The server receives individual data from the terminal device, such as the user's health status, food preferences, allergy information, and family structure. The terminal device is a portable device such as a smartphone, equipped with a camera and microphone. This allows for the collection of data to analyze the user's emotional state. The data is processed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text.

[0592] Location data is acquired in real time using GPS functionality, and the server can use this location information to collect product inventory data from nearby retail facilities. This information is integrated and analyzed by an algorithm called a generative AI model to create the optimal menu suggestion for the user.

[0593] The server monitors the user's emotional state through an emotion analysis engine and incorporates that data into menu suggestions. The suggested menu is sent to the terminal device and communicated to the user using voice notifications. Because the tone and content of these voice notifications are adjusted according to the user's emotional state, it is possible to provide more personalized and flexible suggestions.

[0594] For example, if a user is experiencing stress, the server will suggest a menu that includes easily prepared foods with relaxing properties. Conversely, if the server determines that the user has a positive attitude and wants to try something new, it can recommend challenging and novel recipes.

[0595] An example of a prompt is: "What is the user's current emotional state? Based on that, please suggest foods that can alleviate stress. Considering the current location, please list a few items that are easily available." This prompt is used to encourage the generative AI model to suggest foods.

[0596] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0597] Step 1:

[0598] The user enters their health status, food preferences, allergy information, and household composition into a terminal device. This entered data is then sent directly to the server. The server receives this data and stores it in a database. This process creates a user profile.

[0599] Step 2:

[0600] When a user goes shopping with a terminal device, the device uses GPS functionality to acquire location data. This data is sent to a server, which then uses the location data to collect product inventory information from nearby retail facilities. This allows the server to obtain food information relevant to the user's current location.

[0601] Step 3:

[0602] The system uses the camera and microphone of the terminal device to collect the user's facial expressions and voice data. The collected data is sent to a server and analyzed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text. This allows the user's emotional state to be obtained.

[0603] Step 4:

[0604] The server integrates collected location data, inventory information, user profiles, and emotional data, and uses a generative AI model to generate optimal menu suggestions. This data processing and calculation selects menus that are appropriate for the user's health condition and current emotions.

[0605] Step 5:

[0606] The server sends the generated menu suggestions to the terminal device, which then notifies the user via voice notification. The tone and content of the voice are adjusted to match the user's emotional state, resulting in more personalized suggestions.

[0607] Step 6:

[0608] The user reviews the suggested menu and requests adjustments if necessary. This request is sent to the server, which then uses the generated AI model to propose a new menu. This step allows for flexible menu changes tailored to the user's intentions.

[0609] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0610] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0611] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0612] [Fourth Embodiment]

[0613] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0614] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0615] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0616] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0617] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0619] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0620] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0621] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0622] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0623] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0624] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0625] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0626] The system according to the present invention aims to enable users to efficiently decide on meal menus and promote a healthy lifestyle. The system's processing will be described in detail below based on an example.

[0627] Users first use a terminal application to pre-enter details such as their own and their family's health status, food preferences, allergy information, and family structure. This information is sent from the terminal to the server and stored in a database. Once a user profile is built, it becomes possible to optimize meal plans based on individual needs.

[0628] When a user actually starts shopping at a supermarket, the device acquires the user's location information and sends it to a server. Based on this location information, the server collects inventory information for products offered at nearby retail facilities. Based on the collected data, an AI algorithm generates multiple menu suggestions that take the user's profile into consideration.

[0629] As an example, consider a family of four where the father has high blood pressure, the mother is a vegetarian, and the child prefers meat. In this case, the AI ​​can provide menu suggestions that include low-sodium vegetarian options to accommodate the high blood pressure. For the child, it would then suggest a separate menu that incorporates meat in a balanced way.

[0630] The terminal notifies the user via voice of the menu suggestions received from the server. The user can review the suggestions on the spot and, if necessary, ask questions or request changes to the menu from the AI. For example, in response to a request to adjust the recipe, such as "I'd like to add a few more vegetables to this recipe," the AI ​​will immediately regenerate and re-suggest a new recipe.

[0631] This system responds in real time and flexibly as users choose meals while shopping. This allows users to shop with less stress and provide meals that are optimal for their personal and family health.

[0632] The following describes the processing flow.

[0633] Step 1:

[0634] The user launches the application on their device and enters details such as family health status, dietary preferences, allergy information, and family composition. After completing the input, the device sends this data to the server.

[0635] Step 2:

[0636] The server stores the received user data in a database and builds a profile for each user. This establishes an information infrastructure that can be accessed in real time.

[0637] Step 3:

[0638] When a user arrives at a supermarket and begins shopping, the device acquires their current location information and sends it to the server. Based on this information, the shopping destination is identified.

[0639] Step 4:

[0640] The server queries external databases and APIs to retrieve inventory and special offer information from nearby retail stores based on the user's location. Based on this information, it creates an up-to-date product list.

[0641] Step 5:

[0642] The server uses an AI algorithm to generate optimal menu suggestions based on the user's profile and acquired inventory information. It dynamically creates recipes tailored to the user's health condition and dietary preferences, providing multiple options.

[0643] Step 6:

[0644] The device provides the user with menu suggestions received from the server via voice notification. The user can then review the suggested menu.

[0645] Step 7:

[0646] Users can ask questions or request changes to the suggested menu. User feedback is sent to the server via the terminal.

[0647] Step 8:

[0648] The server regenerates or adjusts the menu in response to adjustment requests from the user. The modified menu is then sent back to the terminal and re-proposed.

[0649] As a result, users can easily select suitable menus while shopping and streamline meal preparation.

[0650] (Example 1)

[0651] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0652] In modern life, efficiently selecting and preparing meal plans while maintaining personal and family health is a crucial challenge for many. However, efficiently shopping while considering health status, food preferences, and allergy information remains difficult. The problem lies in the lack of tools that allow users to dynamically adjust meal plans based on their health information.

[0653] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0654] In this invention, the server includes data collection means for inputting information related to the user's health, means for acquiring information on items in nearby commercial facilities using location information, and means for generating an optimal meal configuration by combining the information from the data collection means and the item information. This enables the user to efficiently select and readjust the optimal meal according to their individual health condition.

[0655] A "user" is someone who uses the system to input information related to meal plan development and receives suggestions based on that information.

[0656] A "data collection method" is an interface that has the function of inputting the user's health status, preferences, and allergy information and sending it to a server.

[0657] "Location information" refers to information that indicates the user's geographical location and is used to obtain information about items in commercial facilities.

[0658] A "commercial facility" refers to a retail environment that sells food and related goods, and is a place where users actually shop.

[0659] "Product information" refers to data that includes information such as the type of food, inventory status, and price at commercial facilities.

[0660] "Meal composition" refers to menu suggestions tailored to the user's health condition and preferences, and includes specific ingredients and cooking methods.

[0661] "Voice notification" is a method of communicating the meal plan generated by the system to the user using voice.

[0662] An "information processing system" is a set of processing mechanisms, including computing resources and software, that integrate user input information with commercial facility item information to propose an optimized meal plan.

[0663] Embodiments of the present invention are described below. First, the user launches a dedicated application using a terminal device. Through the interface of this application, the user inputs information such as their own and their family's health status (e.g., hypertension, diabetes), dietary preferences (e.g., vegetarian, carbohydrate restriction), allergy information (e.g., nut allergy), and family structure (e.g., 2-person family, 4-person family). This information is formatted and encrypted by the terminal and then sent to the server. The server stores this information in a relational database and constructs it as a user profile.

[0664] When a user begins shopping at a commercial facility, their device uses its built-in GPS module to obtain real-time location information. This location information is then sent to a server, which uses the location data to retrieve product information from nearby commercial facilities via publicly available APIs. This product information includes details such as the type of product, price, and availability.

[0665] The server provides stored user profiles and real-time item information as input data to a generating AI model, which then generates an optimal meal plan tailored to the user's health status and preferences. This AI model utilizes deep learning and other machine learning algorithms to suggest customized menus to the user.

[0666] For example, a prompt such as "Please suggest a menu for a family of four, including low-sodium options for hypertension and vegetarian options" is used with the AI ​​model to generate suggestions. The generated menu suggestions are returned to the terminal and communicated to the user via voice notification. The user can review the suggested menu and request changes according to specific needs. These requests might include something like, "I'd like to add more vegetables to this recipe," and the server quickly regenerates the menu using the AI ​​model.

[0667] This system enables users to shop efficiently and choose meals that suit their health condition and preferences.

[0668] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0669] Step 1:

[0670] The user launches a dedicated application on their terminal device and enters personal information such as health status, dietary preferences, allergy information, and family structure. This information is formatted and encrypted by the terminal. The entered data is sent to the server via an HTTP POST request. As output, the user's information is securely transmitted to the server.

[0671] Step 2:

[0672] The server stores the received user information in a relational database. Specifically, it categorizes the data by field and constructs it as a user profile. The input is personal information from the user, and the output is an organized user profile in the database.

[0673] Step 3:

[0674] When a user arrives at a commercial facility, the device's GPS function automatically acquires location information. This location information is sent from the device to the server. The input is location coordinate data, and the server obtains the user's current location as output.

[0675] Step 4:

[0676] The server uses location information to retrieve item information from nearby commercial facilities using APIs. This is done by sending API requests and parsing the data returned in JSON format. The input is the user's location information, and the output is item information data from the commercial facilities.

[0677] Step 5:

[0678] The server provides user profiles and item information as input data to a generating AI model, which then generates an optimal meal plan. The AI ​​model operates based on a deep learning algorithm, combining this information to generate suggestions. User information and item information are used as input, and an optimized menu plan is generated as output.

[0679] Step 6:

[0680] The generated menu plan is sent from the server to the terminal. The terminal uses speech synthesis technology to notify the user of the menu plan by voice. The input is the menu plan data, and the output is the voice notification to the user.

[0681] Step 7:

[0682] Users can review menu suggestions and request specific changes. For example, they can request the addition or modification of certain ingredients. This request is sent from the terminal to the server. The input is the user's additional request, and the server regenerates and outputs the revised menu suggestion.

[0683] (Application Example 1)

[0684] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0685] In modern society, it is difficult to plan meals that take into account individual health conditions and dietary preferences on a daily basis, and users face many challenges in maintaining their own health and preparing meals that suit their family structure. Furthermore, it is not easy to immediately select the optimal menu while shopping at physical stores, taking inventory information into account. In this situation, there is a need for a system that can flexibly respond to the diverse needs of users.

[0686] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0687] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and family structure; means for collecting inventory information of goods at nearby commercial facilities based on location information; means for integrating the information obtained from the data input means with the inventory information of the goods to generate an optimal meal menu; means for presenting the generated menu to the user through voice guidance; and regeneration means using a generative AI model to adjust and re-suggest the generated menu using prompt sentences based on the user's request. This allows the user to dynamically decide on a menu even while shopping, and enables the provision of optimal meals tailored to individual health needs and family needs.

[0688] A "user" refers to an information user, specifically an individual or household member who receives meal menu suggestions from the system.

[0689] "Data input means" refers to a device or software for inputting information such as the user's health status, food preferences, allergy information, and family structure.

[0690] "Location information" refers to geographical data that indicates the user's current location.

[0691] A "commercial facility" refers to a physical store that sells goods, a place where users shop.

[0692] "Inventory information for goods" refers to information including the quantity and types of goods available for sale at a commercial facility.

[0693] "Integration" is the process of combining different types of information to give them meaning and create new value.

[0694] A "menu" refers to a specific meal plan that is proposed and consists of a combination of multiple dishes and ingredients.

[0695] "Voice guidance" is a method of conveying information from a system to a user via voice.

[0696] A "prompt statement" is a text statement that describes requests or instructions for a generative AI model.

[0697] A "generative AI model" is a type of artificial intelligence that performs data analysis and logical reasoning in response to user requests to generate new menus and information.

[0698] A "regeneration mechanism" is a system function that readjusts existing menus based on additional user requests and makes new suggestions.

[0699] This system integrates information from user terminals, servers, and commercial facilities to suggest menus that support a healthy lifestyle. Users input their health status, food preferences, allergy information, and family structure via their terminals, and this information is sent to the server and stored in a database. The server uses this information to build a user profile.

[0700] When a user enters a commercial facility, their device's location information is acquired and sent to a server. Based on this location information, the server collects inventory information for items in surrounding commercial facilities. This inventory information is collected using APIs and real-time data feeds provided by the commercial facilities. The server integrates the user's profile with the commercial facilities' inventory information and generates an optimal meal menu using an AI model.

[0701] The generated menu is presented to the user via voice guidance through the terminal. The user can then make requests to the system using prompts based on the voice guidance. For example, they can request, "I'd like an option to add more vegetables to this menu." The generation AI model immediately regenerates the menu based on the prompt and provides the guidance again through the terminal.

[0702] As a concrete example, a user might ask the system to "provide vegetarian and low-calorie menu suggestions to invite a friend over." In this case, a possible prompt to the generative AI model might be something like, "Please suggest a 3-item vegetarian and low-calorie menu." Upon receiving this prompt, the generative AI model would make adjustments according to the user's request and provide the user with a new menu suggestion.

[0703] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0704] Step 1:

[0705] Users operate a terminal to input their health status, food preferences, allergy information, and family structure. This input information is sent from the terminal to the server. The server stores the received information in a database and builds a user profile. This profile is used as the basis for subsequent menu generation.

[0706] Step 2:

[0707] When a user enters a commercial facility, the terminal acquires the user's location information. This location information is transmitted to the server in real time, and the user's current location is identified. Based on this location information, the server collects inventory information for items in the surrounding commercial facilities via APIs, etc. The output of this process is a list of items actually on the shelves.

[0708] Step 3:

[0709] The server uses the user's profile and acquired inventory information to generate an optimal meal menu using a generative AI model. This process takes into account the user's health goals, preferences, and allergy information, while selecting items from the available inventory. The generated menu suggestions are sent to the terminal as server output.

[0710] Step 4:

[0711] The terminal presents the received menu suggestions to the user via voice guidance. The user listens to the voice guidance and can make additional requests or adjustments to the system using prompt messages. The prompt messages entered at this time are sent to the server through the terminal.

[0712] Step 5:

[0713] The server uses the prompt message received from the user to regenerate the menu using a generation AI model. Based on the content of the prompt message, the data is reprocessed to generate new menu suggestions. These regenerated menu suggestions are then output from the server to the terminal.

[0714] Step 6:

[0715] The terminal then provides the user with new menu suggestions via voice guidance. Based on this information, the user can proceed with their shopping and select ingredients that best suit their and their family's needs. This entire process enables the user to engage in healthy and personalized shopping.

[0716] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0717] This invention relates to a system equipped with an emotion recognition engine that recognizes the user's emotions and optimizes meal menu suggestions based on those emotions. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure into a terminal, and this information is transmitted to a server and stored in a database.

[0718] The system sends location information from the user's device to the server when they make a purchase. The server uses this location information to retrieve inventory information from nearby stores, integrates it with the user profile, and uses an AI algorithm to generate optimal menu suggestions. In addition, it is equipped with an emotion recognition engine that analyzes the user's voice and facial expressions to recognize their current emotional state.

[0719] For example, if a user looks anxious while shopping, the emotion engine will detect that emotion. The server will take this emotional information into consideration and prioritize suggesting familiar dishes or easy-to-prepare recipes to reassure the user. Conversely, if joy or excitement is detected, it can suggest new recipes or challenging dishes.

[0720] The device notifies the user of menu suggestions received from the server via voice, adjusting the tone and content of the voice based on emotional data. For example, a relaxed tone of voice can support the user, and encouraging words can be added to enhance the effectiveness of the suggestions.

[0721] Furthermore, past emotional history is recorded in a database, and the server uses this information to analyze long-term trends and utilize it to suggest menus that are a better match for the user. In this way, the system realizes flexible and accurate meal suggestions that are attentive to the user's emotions, improving the user's shopping experience and health management.

[0722] The following describes the processing flow.

[0723] Step 1:

[0724] Users use an application on their device to input their own and their family's health status, dietary preferences, allergy information, and family structure. The device then sends this information to a server and stores it in a database.

[0725] Step 2:

[0726] The device collects emotional data from the user's voice and facial expressions. Using an emotion engine, it analyzes the collected data to recognize the user's current emotional state.

[0727] Step 3:

[0728] When a user starts shopping at a supermarket, the device sends its location information to a server. Based on this location information, the server retrieves inventory information for nearby retail stores.

[0729] Step 4:

[0730] The server integrates user information stored in the database, inventory information for the current location, and emotional data recognized by the emotion engine, and uses an AI algorithm to generate the optimal menu suggestion.

[0731] Step 5:

[0732] The server sends the generated menu suggestions to the terminal. The terminal presents these suggestions to the user via voice notification. At this time, the voice tone and message content are adjusted based on emotion data.

[0733] Step 6:

[0734] The user reviews the suggested menu and provides feedback or requests for changes via their device as needed. The device then sends this request to the server.

[0735] Step 7:

[0736] The server readjusts the menu based on user change requests and feedback. If necessary, it sends the regenerated menu to the terminal and proposes it again to the user.

[0737] Step 8:

[0738] The server records emotional data collected during each shopping session in a database and analyzes past emotional history. This information is used to make future suggestions and helps provide more personalized meal recommendations to the user.

[0739] (Example 2)

[0740] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0741] In today's lifestyle, choosing meals that suit individual health conditions and dietary preferences is crucial, but effective meal planning is challenging. Furthermore, suggesting meals that align with a user's emotional state is more complex than simply considering nutritional balance. Additionally, systems that enable flexible, real-time meal planning during shopping outside the home are limited, and improvement is needed.

[0742] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0743] In this invention, the server includes an input means for inputting the user's health status, food preferences, allergy information, and family structure; an acquisition means for acquiring inventory information of nearby sales locations based on location information; and a suggestion means for integrating the data acquired from the input means with the inventory information and proposing an optimal meal plan using a generative model. This enables flexible menu suggestions that are optimized for each user's profile and respond to their emotional state.

[0744] "Input means" refers to a device or software for which a user inputs information such as their health status, food preferences, allergy information, and family structure.

[0745] "Acquisition method" refers to the function or process for obtaining inventory information from nearby sales locations based on the user's location information.

[0746] The "proposal method" is a function that integrates acquired data and uses a generative model to present the most suitable meal plan for the user.

[0747] "Emotion recognition means" refers to a technology or device that analyzes a user's voice and facial expression data to identify the user's emotional state.

[0748] The "notification method" refers to a function that notifies the user of the generated menu plan via voice, and adjusts the voice tone and content according to emotional data.

[0749] This invention is a system for optimizing meal menu suggestions by taking user emotions into consideration. The system begins with the user inputting their individual health status, food preferences, allergy information, and family structure using a terminal. The terminal processes the collected data and transmits it to a server. The server stores the received information in a database and manages it appropriately.

[0750] Furthermore, when a user begins shopping, the system sends location information to a server via their device. The server uses the location information to retrieve inventory information from nearby retail stores and integrates it with the provided profile information. The server then uses an emotion recognition engine to analyze the user's voice and facial expressions to recognize their current emotional state. This emotional data influences menu suggestions and forms the basis for generating suggestions that match the user's emotions.

[0751] For example, if a user appears anxious, the server will suggest comforting, familiar dishes or easy-to-make recipes. Conversely, if a user expresses joy or excitement, it will suggest new recipes or challenging dishes. The generated menu suggestions are sent to the device and communicated to the user via voice notification. The notification may include a voice tone and encouraging message tailored to the user's emotions.

[0752] The server uses historical sentiment data to analyze long-term data trends. This analysis enables the system to suggest menus that are more suitable for the user.

[0753] As a concrete example, one could input a prompt message into the AI ​​model such as, "Please suggest a healthy dinner recipe that can be prepared quickly, especially when the user is tired from work." Such a prompt would allow the system to provide suggestions tailored to the user's state.

[0754] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0755] Step 1:

[0756] The user uses a terminal to input their health status, dietary preferences, allergy information, and family structure. The input information is converted into a digital format by the terminal and prepared as data packets that can be sent to the server. The input for this process is the information entered by the user, and the output is the data packets sent to the server.

[0757] Step 2:

[0758] The terminal sends the processed data packets to the server. The server decodes the received data, verifies its contents, and then stores it in a database. The stored data is used for subsequent processing. In this step, the data transmission by the terminal is the input, and the information recorded in the database is the output.

[0759] Step 3:

[0760] When a user makes a purchase, the device's location services are activated to determine its current location. The device sends location data to a server in real time. The server receives this information and collects inventory information for nearby retail stores via an external API. The input to this process is the user's location information, and the output is the retrieved inventory information.

[0761] Step 4:

[0762] The server integrates the collected inventory information with the user's profile data. Next, a generative AI model is used to generate optimal menu suggestions based on the results of the data integration. This takes into account the user's emotional state data. The input for this step is the integrated profile and inventory information, and the output is the generated menu suggestions.

[0763] Step 5:

[0764] The server receives user voice and facial expression data from the terminal to perform emotion recognition. The server uses emotion recognition technology to analyze the user's current emotional state. The analysis results are reflected in the generated menu. In this step, the input is voice and facial expression data, and the output is the emotion recognition result.

[0765] Step 6:

[0766] The server sends the generated menu suggestions and emotion recognition results to the terminal. The terminal then uses this data to send an audio notification to the user. The notification uses a voice tone and content that is adjusted according to the user's emotions. In this process, the input is data from the server, and the output is the audio notification to the user.

[0767] Step 7:

[0768] The server performs long-term data analysis based on user feedback and past sentiment history. Based on the results of this analysis, more accurate menu suggestions become possible. The input is user feedback and historical data, and the output is improved menu suggestions.

[0769] (Application Example 2)

[0770] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0771] Existing meal plan suggestion systems offer suggestions based on the user's health condition and preferences, but they lack the flexibility to adjust menus to take the user's emotional state into account, which limits the quality of the user experience. Furthermore, in the in-store shopping experience, product recommendations and menu adjustments are not tailored to the user's emotions at the time, resulting in insufficient purchasing support.

[0772] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0773] In this invention, the server includes data input means for inputting the user's health status, food preferences, allergy information, and household composition; means for acquiring product inventory data of nearby sales facilities using location data; means for integrating the information acquired from the data input means with the product inventory data to provide an optimal meal plan; means for detecting the user's emotional state through an emotion analysis function and adjusting the menu suggestion according to that emotional state; and means for providing the user with a selected menu via voice notification, adjusting the tone and content of the voice based on the emotional data. This enables flexible and accurate meal suggestions that respond to the user's emotional state at the time, improving the in-store purchasing experience.

[0774] A "user" is an individual who uses the system to receive meal menu suggestions.

[0775] "Health status" refers to information related to the user's physical and mental health.

[0776] "Food preferences" refer to information about the ingredients, types of dishes, and seasonings that users like.

[0777] "Allergy information" refers to information about foods that may cause an allergic reaction in a user's body when consumed.

[0778] "Family structure" refers to information about the number of family members and their relationships within the user's family.

[0779] "Data input means" refers to the means of collecting information from users and incorporating it into the system.

[0780] "Location data" refers to information about the geographical location of a user or device.

[0781] "Product inventory data" refers to information regarding the inventory status of various products at sales facilities.

[0782] "Emotion analysis function" refers to technology that evaluates a user's emotional state based on their voice, facial expressions, and other factors.

[0783] "Adjusting menu suggestions" refers to the process of modifying meal plans suggested based on the user's emotional state and other information.

[0784] "Voice notification" refers to a method of informing the user of system-generated information through voice.

[0785] The system that realizes this invention is designed to enable users to efficiently plan their daily meals. It consists of a combination of hardware and software, including a server, terminal devices, and an emotion analysis engine.

[0786] The server receives individual data from the terminal device, such as the user's health status, food preferences, allergy information, and family structure. The terminal device is a portable device such as a smartphone, equipped with a camera and microphone. This allows for the collection of data to analyze the user's emotional state. The data is processed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text.

[0787] Location data is acquired in real time using GPS functionality, and the server can use this location information to collect product inventory data from nearby retail facilities. This information is integrated and analyzed by an algorithm called a generative AI model to create the optimal menu suggestion for the user.

[0788] The server monitors the user's emotional state through an emotion analysis engine and incorporates that data into menu suggestions. The suggested menu is sent to the terminal device and communicated to the user using voice notifications. Because the tone and content of these voice notifications are adjusted according to the user's emotional state, it is possible to provide more personalized and flexible suggestions.

[0789] For example, if a user is experiencing stress, the server will suggest a menu that includes easily prepared foods with relaxing properties. Conversely, if the server determines that the user has a positive attitude and wants to try something new, it can recommend challenging and novel recipes.

[0790] An example of a prompt is: "What is the user's current emotional state? Based on that, please suggest foods that can alleviate stress. Considering the current location, please list a few items that are easily available." This prompt is used to encourage the generative AI model to suggest foods.

[0791] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0792] Step 1:

[0793] The user enters their health status, food preferences, allergy information, and household composition into a terminal device. This entered data is then sent directly to the server. The server receives this data and stores it in a database. This process creates a user profile.

[0794] Step 2:

[0795] When a user goes shopping with a terminal device, the device uses GPS functionality to acquire location data. This data is sent to a server, which then uses the location data to collect product inventory information from nearby retail facilities. This allows the server to obtain food information relevant to the user's current location.

[0796] Step 3:

[0797] The system uses the camera and microphone of the terminal device to collect the user's facial expressions and voice data. The collected data is sent to a server and analyzed using facial expression analysis with Amazon Rekognition and speech recognition with Google Cloud Speech-to-Text. This allows the user's emotional state to be obtained.

[0798] Step 4:

[0799] The server integrates collected location data, inventory information, user profiles, and emotional data, and uses a generative AI model to generate optimal menu suggestions. This data processing and calculation selects menus that are appropriate for the user's health condition and current emotions.

[0800] Step 5:

[0801] The server sends the generated menu suggestions to the terminal device, which then notifies the user via voice notification. The tone and content of the voice are adjusted to match the user's emotional state, resulting in more personalized suggestions.

[0802] Step 6:

[0803] The user reviews the suggested menu and requests adjustments if necessary. This request is sent to the server, which then uses the generated AI model to propose a new menu. This step allows for flexible menu changes tailored to the user's intentions.

[0804] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0805] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0806] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0807] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0808] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0809] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0810] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0811] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0812] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0813] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0814] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0815] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0816] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0817] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0818] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0819] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0820] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0821] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0822] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0823] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0824] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0825] The following is further disclosed regarding the embodiments described above.

[0826] (Claim 1)

[0827] Information input method for entering the user's health status, food preferences, allergy information, and family structure,

[0828] A means of obtaining inventory information of products at nearby sales facilities based on location information,

[0829] A means for integrating data obtained from the aforementioned information input means with the inventory information of the aforementioned products and proposing the optimal meal plan,

[0830] A means of providing the user with a selected item from the aforementioned menu via voice notification,

[0831] A system that includes this.

[0832] (Claim 2)

[0833] The system according to claim 1, further comprising means for readjusting the proposed menu in response to user selections or requests.

[0834] (Claim 3)

[0835] The system according to claim 1, wherein the information input means transmits information entered by a user using a terminal device to a server, and the server stores the information in a database.

[0836] "Example 1"

[0837] (Claim 1)

[0838] A data collection method for inputting information related to the user's health,

[0839] A means of obtaining information about goods in nearby commercial facilities using location information,

[0840] A means for generating an optimal meal composition by combining information from the data collection means and the item information,

[0841] A means for notifying the user of the selected meal configuration by voice,

[0842] An information processing system that includes this.

[0843] (Claim 2)

[0844] The information processing system according to claim 1, further comprising means for regenerating the meal configuration according to user selections or preferences.

[0845] (Claim 3)

[0846] The information processing system according to claim 1, wherein the data collection means transmits information entered by a user to an information processing device via a communication device, and the information processing device stores the information in a database.

[0847] "Application Example 1"

[0848] (Claim 1)

[0849] A data entry method for inputting the user's health status, food preferences, allergy information, and family structure,

[0850] A means of collecting inventory information of goods at nearby commercial facilities based on location information,

[0851] A means for integrating information obtained from the data input means with inventory information of the items to generate an optimal meal menu,

[0852] A means for presenting the generated menu to the user through voice guidance,

[0853] A regeneration means using a generative AI model that adjusts and re-suggests the generated menu using prompt statements based on user requests,

[0854] A system that includes this.

[0855] (Claim 2)

[0856] The system according to claim 1, further comprising the process of transmitting information entered by a user using a terminal device to a server, and the server storing the information using information storage means.

[0857] (Claim 3)

[0858] The system according to claim 1, further comprising means for using the user's location information to obtain information on the provision of goods based on the current location.

[0859] "Example 2 of combining an emotion engine"

[0860] (Claim 1)

[0861] Input methods for entering the user's health status, food preferences, allergy information, and family structure,

[0862] A means of acquiring inventory information for nearby sales locations based on location information,

[0863] A proposal means that integrates the data obtained from the input means with the inventory information and proposes an optimal meal plan using a generative model,

[0864] An emotion recognition means that analyzes the user's voice and facial expressions to recognize their emotional state and reflect it in the proposed means,

[0865] A notification means that adjusts the tone and content of the voice based on emotional data when notifying the user of the selected item from the aforementioned menu,

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1, further comprising means for readjusting the proposed menu in response to user selections or requests and optimizing the suggestions by utilizing past sentiment history data.

[0869] (Claim 3)

[0870] The system according to claim 1, wherein the input means transmits information entered by a user using a terminal device to a central device, and the central device stores the information in a storage area.

[0871] "Application example 2 when combining with an emotional engine"

[0872] (Claim 1)

[0873] A data entry method for inputting the user's health status, food preferences, allergy information, and household composition,

[0874] A means of obtaining product inventory data of nearby sales facilities using location data,

[0875] A means for integrating information obtained from the aforementioned data input means with the inventory data of the aforementioned products to provide an optimal meal plan,

[0876] A means for detecting the user's emotional state through an emotion analysis function and adjusting menu suggestions according to that emotional state,

[0877] A means of providing the user with a selected item from the aforementioned menu via voice notification, and adjusting the tone and content of the voice based on emotional data,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, further comprising means for recommending products according to the user's emotional state.

[0881] (Claim 3)

[0882] The system according to claim 1, wherein the data input means transmits information entered by a user using a terminal device to a network device, and the network device stores the information in an information warehouse. [Explanation of Symbols]

[0883] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Information input method for entering the user's health status, food preferences, allergy information, and family structure, A means of obtaining inventory information of products at nearby sales facilities based on location information, A means for integrating data obtained from the aforementioned information input means with the inventory information of the aforementioned products and proposing the optimal meal plan, A means of providing the user with a selected item from the aforementioned menu via voice notification, A system that includes this.

2. The system according to claim 1, further comprising means for readjusting the proposed menu in response to user selections or requests.

3. The system according to claim 1, wherein the information input means transmits information entered by a user using a terminal device to a server, and the server stores the information in a database.

Images