Recipe suggestion system

Abstract

To provide efficient and streamlined recipe suggestions, while also offering a user-friendly and user-friendly system. [Solution] A recipe suggestion system comprising: a camera unit equipped on a user's mobile device for taking pictures of the expiration dates of food stored in the refrigerator; a recognition unit for recognizing the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit for displaying the expiration date information and suggestions related to the food items; a question unit for asking questions about the date and time based on the date and time information; and a suggestion unit for suggesting recipes using the food items based on the user's or mobile device's response information corresponding to the suggestion and the expiration date information.

Description

Technical Field

[0001] The present invention relates to a system that recognizes expiration date information of food and proposes recipes.

Background Art

[0002] The present invention reads expiration date information of food with a photographing unit, processes it with a recognition unit, and acquires information on foods in ascending order of short expiration dates. Then, characters are displayed on a display unit, questions regarding date and time are asked in a question unit, and based on the answer information and the expiration date information, a recipe using the food can be proposed in a proposal unit. Thus, it is a system that utilizes the expiration date information of food using a mobile terminal owned by the user and proposes recipes.

Prior Art Documents

Patent Documents

[0003] Japanese Unexamined Patent Application Publication No. 2022-076493

Summary of the Invention

Problems to be Solved by the Invention

[0004] [[ID=�1]] An object of the present invention is to utilize the expiration date information of food and propose recipes efficiently and without waste using a mobile terminal owned by the user. Specifically, it is to provide a system that acquires information on foods in ascending order of short expiration dates and proposes recipes using the food based on the answer information and the expiration date information. Also, by having a character propose recipes, it is also an object to provide a system that is more familiar and easier to use for the user.

Means for Solving the Problems

[0005] This invention reads the expiration date information of food products using a camera unit, processes it using a recognition unit, and obtains information on food products in order of shortest expiration date. Then, it displays a character using a display unit, asks questions about the date and time using a question unit, and based on the answer information and expiration date information, it can suggest recipes using the food products using a suggestion unit. In this way, the system utilizes the expiration date information of food products and suggests recipes using the user's mobile device. Another objective is to provide a more user-friendly and user-friendly system by having a character suggest recipes.

[0006] Furthermore, the suggested recipes are determined from a recipe database pre-registered by the system, based on the information provided by the user and the expiration date information. This invention makes it possible to provide more efficient and waste-free recipe suggestions by utilizing the expiration date information of food products. In addition, by having a character suggest recipes, the system can be made more user-friendly and easy to use. [Effects of the Invention]

[0007] This invention utilizes food expiration date information to provide more efficient and waste-free recipe suggestions. Furthermore, by having a character suggest recipes, it offers a more user-friendly and accessible system. The suggested recipes are determined from a pre-registered recipe database based on the information provided by the user and the expiration date information. [Modes for carrying out the invention]

[0008] This invention reads the expiration date information of food products using a camera unit, processes it using a recognition unit, and obtains information on food products in order of shortest expiration date. Then, it displays a character using a display unit, asks questions about the date and time using a question unit, and based on the answer information and expiration date information, it can suggest recipes using the food products using a suggestion unit. In this way, the system utilizes the expiration date information of food products and suggests recipes using the user's mobile device. Another objective is to provide a more user-friendly and user-friendly system by having a character suggest recipes.

[0009] Furthermore, the suggested recipes are determined from a recipe database pre-registered by the system, based on the information provided by the user and the expiration date information. This invention makes it possible to provide more efficient and waste-free recipe suggestions by utilizing the expiration date information of food products. In addition, by having a character suggest recipes, the system can be made more user-friendly and easy to use. The suggested recipes are determined from a recipe database pre-registered by the system, based on the information provided by the user and the expiration date information. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram of the recipe suggestion system of the present invention.

[0011] The definitions of keywords in the claims are as follows:

[0012] Food expiration date information refers to the expiration date information printed on food products.

[0013] The recognition unit is the part that reads and processes the expiration date information of food products using the imaging unit.

[0014] The suggestion section proposes recipes using food items, based on information about foods listed in order of shortest shelf life and the responses received.

[0015] Usage example

[0016] The recipe suggestion system of the present invention can provide efficient and waste-free recipe suggestions, for example, when ingredients are left over at home and cannot be used up, or when purchased ingredients are nearing their expiration date. For example, it can acquire information on foods in order of shortest expiration date, and based on the response information and expiration date information, the suggestion unit can propose recipes using those foods. For example, it can suggest recipes for soups or stews using vegetables that are nearing their expiration date.

[0017] As shown in Figure 1, the recipe suggestion system of the present invention reads the expiration date information of food products using a photography unit, processes it using a recognition unit, and obtains information on food products in order of shortest expiration date. Then, it displays a character using a display unit, asks questions about the date and time using a questioning unit, and suggests recipes using the food products using the answer information and expiration date information using a suggestion unit. The suggested recipes are determined from a recipe database pre-registered by the system based on the information answered by the user and the expiration date information. Furthermore, the system provides a more user-friendly and user-friendly system by having a character suggest recipes.

[0018] The recipe suggestion system of the present invention can be implemented on devices such as mobile terminals and smartphones, and can be used in homes, restaurants, and other establishments.

[0019] The following describes a recipe suggestion system that recognizes the expiration date information of food products according to this embodiment.

[0020] The present invention relates to a recipe proposal system comprising: a photographing unit provided in a portable terminal owned by a user for photographing the expiration date of food stored in a refrigerator as shown in FIG. 1; a recognition unit for recognizing expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit for displaying the expiration date information and making proposals regarding the food; a questioning unit for asking questions regarding the date and time based on the date and time information; and a proposal unit for proposing a recipe using the food based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information. The present invention can recognize the expiration date information of food in the refrigerator and propose a recipe based on that information. Further, when the user answers a question regarding the date and time, a recipe is proposed taking into account the answer result, enabling a more appropriate recipe proposal.

[0021] When the user performs an operation of designating a transfer destination and instructing transmission to the transfer destination (e.g., terminal device) using the response text data displayed on the display screen and a transfer application such as an email or messaging app, the refrigerator management app unit instructs the transmission app unit to transfer the response text data selected based on the operation received by the operation reception unit to another device, enabling the recipe proposal system to be used on another device.

[0022] A portable terminal application is provided for photographing the expiration date of food stored in a refrigerator. This application recognizes the expiration date from the photographed image and displays the food in ascending order of the expiration date. The object of photographing is the part containing information regarding the expiration date of the food, which may also be the receipt at the time of purchasing the food. Further, questions regarding the date and time can be asked based on the date and time information. A recipe using the food can be proposed based on the information answered by the user and the expiration date information. The proposed recipe is presented in a user-friendly manner with characters displayed.

[0023] Provided is a mobile terminal that photographs the expiration dates of foodstuffs stored in a refrigerator. This mobile terminal recognizes the expiration date information of foods in ascending order of the shortness of the expiration date based on the expiration date. Also, based on the date and time information, it asks questions regarding the date and time. Furthermore, based on the response information corresponding to the proposal and the expiration date information, it proposes recipes using foods.

[0024] When the user gives an instruction to sort (rearrange) the response text data displayed on the display screen in ascending order of the expiration date or consumption date, the refrigerator management application unit rearranges and displays the pork, chicken, pudding from Company A, and onions included in the response text data in ascending order of the expiration date or consumption date. Also, it is possible to search for and display recipes from the foodstuffs selected by the user.

[0025] Also, in this embodiment, a customer purchases goods worth more than a predetermined amount per month from the seller, and when the seller's monthly purchase amount by the customer exceeds the predetermined amount, the seller performs a discount settlement from the total amount according to the excess amount, and a contract is made such that a predetermined product price discount point (point) and coupon are added. In this case, in this embodiment, the seller confirms that the customer's monthly purchase amount is more than the predetermined amount, and in that case, can provide the recipe proposal system to the customer for free. In such a form, the seller can not only enhance the sales promotion effect of its own products, but also the convenience can be improved because the customer can use the recipe proposal system including products other than its own products for free.

[0026] This is a recipe proposal system in which the control unit of the refrigerator performs voice recognition processing and transmits the analysis result of voice data including an inquiry to the server system. (Modification 4) In the above description, it was assumed that the expiration date information of foods is recognized by the mobile terminal held by the user. Instead of this, the expiration date information of foods may be recognized by a camera provided in the refrigerator. This is a recipe proposal system that recognizes the expiration date information of foods by a camera provided in the refrigerator.

[0027] This recipe suggestion system indicates that, if there is a terminal installed in relation to the refrigerator door, power would be wasted if the display unit were constantly running. Therefore, the display function is designed to activate, for example, when the refrigerator door is opened.

[0028] The recipe suggestion system according to the present invention includes a refrigerator management application unit that displays a list of inquiries (for example, "I want to make something," "Do you have any recommendations?", etc.), a recipe suggestion unit that proposes a recipe according to the inquiry selected by the user from the list, displays the ingredients and procedures of the recipe selected by the user, the refrigerator management application unit that automatically checks the ingredients and procedures of the recipe selected by the user, notifies the user if any ingredients are missing, and can also display a list of necessary ingredients. A recipe suggestion system as described in claim 1, characterized in that the recipe suggestion is made by a character, the character is an animated character, and the animated character is capable of voice output by speech synthesis technology. For example, if the inquiry is selected as "I want to make something," the voice output will say something like, "Is there a particular genre or type you prefer?"

[0029] Furthermore, a key feature of the recipe suggestion system of the present invention is that when the total value of the food in the refrigerator is displayed, the amount of food already stored in the refrigerator is deducted.

[0030] The refrigerator management app unit receives, stores, and displays response text data, including ingredient lists and recipes, in response to inquiries such as checking the ingredients stored in the refrigerator, identifying ingredients nearing their expiration or best-before date, and checking recipes that can be made with the ingredients stored in the refrigerator. It also uses the camera function on the user's mobile device to photograph the expiration dates of the food stored in the refrigerator and recognizes the expiration date information of the food items in order from shortest to oldest based on the expiration dates. Furthermore, it displays the expiration date information and a character that makes suggestions regarding the food, and asks questions about the date and time based on the date and time information. Finally, it suggests recipes using the food based on the user's or mobile device's response information and the expiration date information corresponding to the suggestions.

[0031] The food information recording means is configured to record at least the following: information about the name of food A (an example of a food product), information about the quantity of food in stock, information about the best-before date of food, information about the expiration date of food, information about the price (list price) of food, information about the food purchaser, information about the number of food items held by the purchaser (in other words, the number of food items purchased by the purchaser but not yet consumed), and information about the food producer.

[0032] The food expiration date management device comprises a menu generation unit for food expiration date management and processing, menu display control means, input means, food item storage unit, food item display control means, food item selection control means, selected food item storage unit, calendar generation unit, calendar display control means, expiration date setting control means, set expiration date storage unit, expiration date warning date and time item generation means, expiration date warning date and time setting display control means, warning date and time storage unit, food storage status calculation means, food inventory list display control means, data erasure means, and a timing unit and liquid crystal display unit. This device can set and manage the expiration dates of food. It can also issue warnings when the expiration date is approaching. Furthermore, it can check the inventory status of stored food. This makes food management in the home easier. This recipe suggestion system comprises a shooting unit that photographs the expiration dates of food stored in a refrigerator, a classification unit that classifies the food in the refrigerator based on the expiration dates, a suggestion unit that proposes recipes for the classified food, a listing unit that lists the ingredients required for the suggested recipe, a notification unit that notifies the user of any missing ingredients from the list, and an ordering unit that automatically orders the selected ingredients when the user selects the notified ingredients. This invention relates to a method for managing ingredients in a refrigerator and a recipe suggestion system based on this method, enabling the automatic classification of ingredients in the refrigerator under specific conditions, recipe suggestions, and automatic ordering of missing ingredients.

[0033] The recipe suggestion method according to the present invention involves recognizing the expiration date information of food stored in a refrigerator, calculating the number of days elapsed from that information, determining the freshness of the food from that number of days elapsed, deciding on the suggested recipe content based on that freshness, and displaying the suggested recipe content to the user. A refrigerator management system that acquires information such as the expiration dates and inventory levels of food items in the refrigerator, and automatically creates and presents menus, shopping lists, and other information to the user based on this data. The term "food ingredients" includes "vegetables," "fruits," "meat," "seafood," "eggs and dairy products," and "processed foods." This invention is a refrigerator management system that acquires information such as the expiration dates and inventory levels of food items in the refrigerator, and automatically creates and presents menus and shopping lists to the user based on this information. In this invention, information such as food names and expiration dates is automatically extracted from image data captured by a camera installed inside the refrigerator and stored in a database, where various analysis processes are performed. For example, menus and shopping lists are automatically created and presented to the user based on items that are nearing their expiration date or have low inventory levels. Furthermore, this invention is accessible from smartphones and other devices, allowing users to check what they need inside the refrigerator and view shopping lists even when they are away from home. The above example shows how to obtain information such as food names and expiration dates from image data captured by a camera installed inside a refrigerator. However, the camera does not necessarily need to be installed inside the refrigerator. The subject of the image should include information about the food's expiration date, and a receipt from when the food was purchased would also suffice.

[0034] The present invention aims to provide a recipe suggestion system for managing the expiration dates of food products. According to the present invention, a camera unit equipped in a mobile terminal owned by the user photographs the expiration dates of food products stored in a refrigerator, and a recognition unit recognizes the expiration date information. The display unit then displays the expiration date information of the food products in order from shortest to oldest, along with a character that makes suggestions related to the food products. Furthermore, a question unit is provided that asks questions about the date and time based on the date and time information. When answer information is obtained from the user or mobile terminal, the suggestion unit proposes a recipe using the food product based on the answer information and the expiration date information.

[0035] The food expiration date management device according to the present invention is formed to have external dimensions of, for example, 80 mm x 80 mm x 10 mm and is configured to be detachably attached to a refrigerator door or the like by magnets. The food expiration date management device according to the present invention also comprises a shooting unit, a recognition unit, a display unit, a questioning unit, and a suggestion unit. The shooting unit photographs the expiration dates of the food stored in the refrigerator. The recognition unit recognizes the expiration date information of the food in order of shortest expiration date, based on the expiration date. The display unit displays the expiration date information and a character that makes suggestions regarding the food. The questioning unit asks questions regarding the date and time based on the date and time information. The suggestion unit, based on the answer information from the user or the mobile terminal corresponding to the suggestion and the expiration date information, has the character suggest a recipe using the food.

[0036] The food expiration date management device in this embodiment is specifically composed of a computer as shown in the figure. In this embodiment, the computer includes a shooting unit, a recognition unit, a display unit, a questioning unit, and a suggestion unit. The shooting unit is a shooting unit provided in a mobile terminal owned by the user. The recognition unit is a recognition unit that recognizes the expiration date information of food products in order of shortest expiration date, based on the expiration date. The display unit is a display unit that displays the expiration date information and a character that makes suggestions regarding the food products. The questioning unit is a questioning unit that asks questions regarding the date and time based on the date and time information. The suggestion unit is a suggestion unit in which the character suggests recipes using the food products based on the answer information from the user or the mobile terminal corresponding to the suggestion and the expiration date information.

[0037] Furthermore, the terminal device according to the present invention includes: an expiration date information acquisition means for acquiring expiration date information of products corresponding to tags received from a detection device from a server device; an expiration date information storage means for storing the acquired expiration date information in association with the tag and product ID; a display means for displaying product information identified by the product corresponding to the tag for tags whose expiration date information stored in the expiration date information storage means satisfies predetermined conditions; a recipe storage means for storing recipe information which is information recording the dish item, the type of product used, and the quantity; and a recipe suggestion means for referring to the tag information and acquiring and suggesting recipe information corresponding to products corresponding to tags whose expiration date information has not reached a predetermined expiration date from the recipe storage means. Embodiments of the present invention are as described above.

[0038] The recipe suggestion system according to the present invention comprises: a camera unit equipped in a mobile terminal owned by the user; a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests a recipe using the food items based on the user's or mobile terminal's response information corresponding to the suggestion and the expiration date information. The user takes a picture of the expiration date of food items stored in the refrigerator, the recognition unit suggests a recipe from the recognized expiration date information and date and time information, and the system suggests the optimal recipe based on the user's response.

[0039] This application provides a mobile device that takes pictures of the expiration dates of food stored in a refrigerator. The application recognizes the expiration dates from the captured images and displays the expiration date information of the food items in order from shortest to oldest. It also includes a question section that asks questions about the date and time based on the date and time information, and suggests recipes using the food items based on the answers from the user or the mobile device corresponding to the suggestions and the expiration date information. This application may use inventory information entered by the user, inventory information determined from images taken inside the refrigerator, or inventory information determined from changes in the weight of food items stored in a predetermined location.

[0040] This recipe suggestion system uses a refrigerator's control unit to perform voice recognition processing and send the analysis results of voice data, including inquiries, to a server system. While the system uses a user's mobile device to recognize the expiration date of food items, it could also be implemented using a camera installed inside the refrigerator. This recipe suggestion system uses a camera installed inside the refrigerator to recognize the expiration date information of food items.

[0041] The recipe suggestion system according to the present invention comprises: a camera unit equipped in a mobile terminal owned by the user; a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions regarding the date and time based on the date and time information; and a suggestion unit in which the character suggests a recipe using the food items based on the user's or mobile terminal's response information corresponding to the suggestion and the expiration date information. The user takes a picture of the expiration date of food items stored in the refrigerator, the recognition unit recognizes the information and displays it on the display unit. The user can also select and actually make the recipe suggested by the character.

[0042] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a block diagram of a recipe suggestion system according to one embodiment of the present invention. In this embodiment, a camera unit equipped in a mobile terminal owned by the user photographs the expiration dates of food stored in the refrigerator and transmits this photographic data to an information acquisition unit. Based on this photographic data, the information acquisition unit recognizes the expiration date information of the food items in order from shortest to oldest and transmits this information to a display unit. The display unit displays this expiration date information and a character that makes suggestions regarding the food. In addition, a question unit is provided that asks questions about the date and time based on the date and time information, and a suggestion unit is provided that proposes recipes using the food based on the answer information from the user or mobile terminal and the expiration date information. The present invention includes a camera unit in a user's mobile device that photographs the expiration dates of food stored in the refrigerator, a recognition unit that recognizes the expiration date information of the food in order from shortest to oldest based on the expiration date, a display unit that displays the expiration date information and a character that makes suggestions regarding the food, a question unit that asks questions about the date and time based on the date and time information, and a recipe that uses the food suggested based on the user's or mobile device's response information corresponding to the suggestion and the expiration date information. If a recipe is selected, the system displays a message indicating that the stock of the missing ingredients is insufficient, notifies the user that the missing ingredients are needed separately, and if the missing ingredients are needed separately, it displays information such as the name of the store where the missing ingredients can be obtained and how to obtain them. The present invention also covers "surplus food in the refrigerator" and "vegetables and fruits that were not used in the household."

[0043] The recipe suggestion system according to the present invention comprises: a camera unit provided in a user's mobile terminal; a recognition unit that recognizes the expiration date information of food products in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes suggestions regarding the food products; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests a recipe using the food products based on the user's or mobile terminal's response information corresponding to the suggestion and the expiration date information. The user terminal then uses a recipe suggestion means implemented by running an application to refer to tag information in a recipe storage means and extract product information for all tags whose completion status is not "completed" and whose expiration date indicated by the expiration date information has not been reached. The system then suggests and displays a recipe from the extracted product information.

[0044] The recipe suggestion system according to the present invention comprises: a camera unit equipped in a mobile terminal owned by the user; a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests a recipe using the food items based on the answer information from the user or the mobile terminal corresponding to the suggestion and the expiration date information. The user takes a picture of the expiration date of food items stored in the refrigerator, the recognition unit recognizes the expiration date information from the captured image, and the display unit displays a character to make suggestions. The user can answer questions based on the date and time information and cook a meal referring to the suggested recipe.

[0045] Furthermore, users can, for example, display a shopping list stored in the server's memory on the terminal device's display to check which ingredients are missing and how much they need, thus preventing them from forgetting to buy necessary ingredients.

[0046] When identifying the above-mentioned recipe information, this recipe suggestion unit prioritizes selecting ingredients that the user currently possesses as surplus ingredients after cooking and that are nearing their expiration date, using ingredient management, and then identifies recipes that use those ingredients from the recipes. Furthermore, the present invention is characterized by the interaction of the above-mentioned components.

[0047] Furthermore, the system displays dish information from the menu search unit on the display unit and transmits one or more selected dish information to the menu storage unit via the communication control unit.

[0048] Furthermore, the recommended store search unit may search for the names and locations of recommended stores based on the purchased food information from the purchased food search unit and the food inventory information from multiple food inventory information storage devices, and notify the mobile terminal device. Moreover, the present invention has the above-described technical features and is a recipe suggestion system that enables food management in the refrigerator and recipe suggestions.

[0049] The recipe suggestion system according to the present invention, as shown in Figure 1, comprises a mobile terminal owned by the user, a refrigerator, and a server. The mobile terminal includes a shooting unit, a recognition unit, a display unit, a questioning unit, and a suggestion unit. The refrigerator includes a storage unit for storing food and an output unit for outputting expiration date information of the food. The server includes a storage unit for storing recipe information and a generation unit for generating suggestion information based on the recipe information. The shooting unit photographs the expiration date information of the food, the recognition unit recognizes the expiration date information of the food based on the expiration date information, and the display unit has the character suggest a recipe using the food based on the response information from the user or the mobile terminal corresponding to the suggestion and the expiration date information. The object of the shooting unit is the part of the food that includes information about the expiration date, and may also be a receipt when the food was purchased. The questioning unit asks questions about the date and time based on the date and time information. The suggestion unit suggests a recipe based on the suggestion information.

[0050] The refrigerator portable terminal device transmits an inventory information acquisition request to the food inventory information management device of the refrigerator's food inventory information management device via a cellular network, the internet, a home network, and a communication control device, requesting the acquisition of inventory information.

[0051] The mobile terminal comprises a communication unit, a display unit, an input unit, and a control unit. The control unit extracts expiration date information from image data acquired from the shooting unit, transmits it to the recognition unit, receives the recognition result, transmits it to the display unit, and controls the display so that a character is shown. It also determines the content of a question from the date and time information, transmits it to the display unit, and controls the display so that a character is shown. Furthermore, it determines recipe candidates from the answer content, transmits them to the suggestion unit, and controls the display so that a character is shown.

[0052] When a mobile device acquires information about ingredients, it will suggest recipes based on that information. For example, if the ingredient "eggs" is entered, recipes such as "egg dishes" will be suggested.

[0053] The portable device is equipped with a storage device that stores information notified by the expiration date sorting unit. Because this recipe suggestion system is on the user's portable device, it can be used even when the user is away from home. Furthermore, it makes managing food in the refrigerator easier, leading to a reduction in food waste.

[0054] Furthermore, the mobile terminal device can send and receive information to and from the refrigerator's food inventory and cooking information management device via a cellular network, the internet, and a home network.

[0055] The recipe suggestion system according to the present invention is equipped with a storage device that stores information notified from a food inventory cooking information management device. This storage device stores information such as the expiration date of food, the inventory status of food, and recipe information. The recipe suggestion system according to the present invention is equipped with a shooting unit on a mobile terminal owned by the user, a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date, a display unit that displays the expiration date information and a character that makes suggestions regarding the food, a questioning unit that asks questions about the date and time based on the date and time information, and a suggestion unit in which the character suggests a recipe using the food based on the answer information from the user or the mobile terminal corresponding to the suggestion and the expiration date information.

[0056] Furthermore, the mobile terminal device is equipped with a storage device that stores information notified by the recommended food purchase search unit.

[0057] The shooting unit outputs video data as a result of the shooting to the camera module control unit. The camera module control unit generates still image data from the video data, extracts expiration date information from the still image data, and outputs it to the recognition unit. The recognition unit then extracts the food name and expiration date from the extracted expiration date information and outputs it to the display unit for display. Furthermore, the suggestion unit generates recipe candidates from the response information and expiration date information, and a character suggests and displays them.

[0058] The camera unit outputs the shooting results from the refrigerator compartment camera and the drawer camera to the camera module control unit. The camera module control unit extracts the food name and expiration date information from the shooting results and sends it to the recognition unit for recognition processing. After the recognition processing, the display unit displays a character and a recipe suggestion screen, and the suggestion unit suggests a recipe on the recipe suggestion screen based on the user's responses.

[0059] The refrigerator is equipped with a camera for the refrigerator compartment and a camera for the drawer. The user takes a picture of the expiration dates of the food inside the refrigerator using a mobile device. The recognition unit recognizes the expiration date information and displays the expiration date information of the food items in order from shortest to oldest. The display unit displays the expiration date information and a character that makes suggestions regarding the food items. The questioning unit asks questions about the date and time based on the date and time information. The suggestion unit suggests a recipe using the food items based on the user's or mobile device's response information and the expiration date information corresponding to the suggestion. Although this example shows taking a picture of the expiration dates of food items inside the refrigerator, the subject of the camera's photography is not limited to items inside the refrigerator; it could also be a receipt from when the food was purchased, as long as it contains information about the expiration date of the food items.

[0060] The present invention relates to a method and apparatus for managing inventory in a refrigerator and providing recipes, and aims to provide technical means that enable users to easily, efficiently, and enjoyably manage inventory in a refrigerator and provide recipes under specific conditions.

[0061] Based on the purchase date and time information provided by the user, the expiration date of the food item is calculated, and the name of the food item and its expiration date are displayed. In addition, recipes are suggested for the food item selected by the user.

[0062] The food information recording means is configured to record at least the following: information on the name of food A (an example of a food product), information on the number of food items in stock, information on the food's best-before date, information on the food's expiration date, information on the food's price (list price), information on the food's purchaser, information on the number of food items owned by the purchaser (in other words, the number of food items purchased by the purchaser but not yet consumed), and information on the food's producer. The recipe suggestion system also includes: a shooting unit that photographs the best-before dates of food items stored in the refrigerator; a recognition unit that recognizes the best-before date information of food items in order of shortest best-before date based on the best-before date; a display unit that displays the best-before date information and a character that makes suggestions regarding the food items; a questioning unit that asks questions regarding the date and time based on the date and time information; and a suggestion unit in which the character suggests a recipe using the food items based on the user's or mobile terminal's response information corresponding to the suggestion and the best-before date information. Furthermore, because the present invention has the configuration described above, it is possible to automatically determine various data such as "the expiration date of the product" and "the price of the product" from "the product stored in the refrigerator" and "the expiration date of the product", and to provide a recipe suggestion service optimized for situations such as "the product stored in the refrigerator".

[0063] The food expiration date management device comprises a menu generation unit for food expiration date management and processing, menu display control means, input means, food item storage unit, food item display control means, food item selection control means, selected food item storage unit, calendar generation unit, calendar display control means, expiration date setting control means, set expiration date storage unit, expiration date warning date and time item generation means, expiration date warning date and time setting display control means, warning date and time storage unit, food storage status calculation means, food inventory list display control means, data erasure means, and a timing unit and liquid crystal display unit. This device can set and manage the expiration dates of food. It can also issue warnings when the expiration date is approaching. Furthermore, it can check the inventory status of stored food. This makes food management in the home easier. This recipe suggestion system comprises a shooting unit that photographs the expiration dates of food stored in a refrigerator, a classification unit that classifies the food in the refrigerator based on the expiration dates, a suggestion unit that proposes recipes for the classified food, a listing unit that lists the ingredients required for the suggested recipe, a notification unit that notifies the user of any missing ingredients from the list, and an ordering unit that automatically orders the selected ingredients when the user selects the notified ingredients. This invention relates to a method for managing ingredients in a refrigerator and a recipe suggestion system based on this method, which enables the automatic classification of ingredients in the refrigerator under specific conditions, and allows for recipe suggestions and automatic ordering of missing ingredients.

[0064] As shown in Figure 1, the recipe suggestion system according to the present invention comprises: a shooting unit equipped on a user's mobile terminal for photographing the expiration dates of food stored in the refrigerator; a recognition unit for recognizing the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit for displaying the expiration date information and a character that makes suggestions regarding the food items; a questioning unit for asking questions about the date and time based on the date and time information; and a suggestion unit for having the character suggest a recipe using the food items based on the user's or mobile terminal's response information corresponding to the suggestion and the expiration date information. Furthermore, the recipe suggestion system according to the present invention may have the configuration shown in Figure 1, and can automatically determine which food items are nearing their expiration date based on the type and quantity of food items in the refrigerator and suggest a recipe using that food item. Moreover, the recipe suggestion system according to the present invention may have the configuration shown in Figure 1, and can automatically determine which food items are nearing their expiration date based on the type and quantity of food items in the user's refrigerator and suggest a recipe using that food item, automatically check for missing ingredients from the ingredient list of the recipe selected by the user, create a list of missing ingredients, display only the missing ingredients from that list, and save it as a "shopping list". The present invention has the configuration described above and has the following effects. (1) Recipe suggestions can be automatically generated based on the expiration date information of the food in the refrigerator, allowing users to select recipes easily and quickly without any hassle. (2) After selecting a recipe, you can check only the necessary ingredients from the list of missing ingredients and save it as a "shopping list," saving time and effort when creating a shopping list. (3) This invention not only manages food in the refrigerator but also allows for the creation of shopping lists, making it effective in a wide range of areas such as "improving household efficiency" and "saving energy".

[0065] This recipe suggestion system can perform the following actions, for example: The user takes a picture of the expiration date of food in their refrigerator, and after receiving that information on their mobile device, questions such as "What do you want to eat today?" are displayed. Based on the user's answers and the information of the food they photographed, a recipe is suggested. In addition to questions such as "What do you want to eat today?", questions such as "How many servings are you making today?" may also be displayed. Claim 1 describes a recipe suggestion system characterized in that the recipe suggestion is made by a character, and the character is an animated character. Specifically, "animated character" refers to an "animated character" that is known as an "animated character."

[0066] When the CPU switches to the expiration date input operation mode, coordinate data on the touch panel corresponding to "expiration date input" is received by the CPU from the operation control unit as a command, causing the food item display control means to operate and food item data such as milk, eggs, fish, pork, beef, vegetables, fruits, and bread to be read from the food item storage unit. A food selection screen including various food items and icon marks for scrolling the food item screen is displayed on the liquid crystal display unit in a predetermined pattern. By combining these elements, the recipe suggestion system of the present invention is realized. This refrigerator inventory management device can detect the inventory status of products inside the refrigerator, automatically create a shopping list based on the inventory status, and notify the user's terminal. The refrigerator inventory management system can detect the inventory status of items inside the refrigerator, automatically create a shopping list, and notify the user's terminal. The specific configuration is as follows: The refrigerator inventory management system uses sensors installed inside the refrigerator to detect the inventory status of items and sends the database information to a server. The server automatically generates a shopping list based on the product name, quantity, etc., and notifies the user's terminal. An application is also provided that allows users to check the shopping list and product information from their terminals.

[0067] This system operates on the user's mobile device and works by the user taking a picture of the expiration date of food in their refrigerator and sending that information to the system. The system recognizes information such as the food name and expiration date from the captured image and suggests recipes based on that information. It also answers questions the user may have regarding dates and times, and these answers are also reflected in the recipe suggestions.

[0068] Furthermore, the recipe suggestion system of the present invention can suggest recipes based on the expiration date information of food items in the refrigerator, thereby contributing to the reduction of food waste. [2. When a purchaser of a food product requests information on the best-before date or expiration date] [2-1. In the case of best-before date] When a purchaser of a food product requests information on the best-before date of the purchased product, the food information confirmation means confirms the best-before date information recorded in the food information recording means 4. The present invention can then suggest recipes from the best-before date information of the food in the refrigerator, contributing to the reduction of food waste. This home appliance is capable of automatically tracking, managing, and displaying the remaining quantity and expiration date of food and cooked dishes stored inside the refrigerator. It is characterized by comprising an image processing device that can automatically determine and display the remaining quantity and expiration date from image data captured by a camera installed inside the refrigerator, and a communication device that can be wirelessly transmitted from inside or outside the refrigerator. The present invention is a home appliance comprising an image processing device that can automatically determine and display the remaining amount, expiration date, etc., from image data captured by a camera installed inside the refrigerator, and a communication device that can be wirelessly transmitted from inside or outside the refrigerator.

[0069] Furthermore, the recipe suggestion method according to the present invention involves taking a photograph of the expiration date of food stored in a refrigerator and suggesting a recipe based on that information. The recipe suggestion device comprises: means for photographing the expiration dates of food stored in a refrigerator; means for recognizing the expiration date information of food items in order of shortest expiration date based on the expiration date information; means for displaying the expiration date information and a character that makes suggestions regarding the food items; means for asking questions about the date and time based on the date and time information; and means for suggesting recipes using the food items based on the user's or mobile device's response information corresponding to the suggestion and the expiration date information. This invention relates to a method and apparatus for managing food items in a refrigerator. The specific method is as follows: First, a photograph of the food items in the refrigerator and data such as their expiration dates (hereinafter referred to as "data") are taken with a smartphone or the like. Next, the data is analyzed to identify items that are nearing their expiration date, items that have expired, etc. If items that are nearing their expiration date, etc., are identified, then "dishes that can be made with this ingredient," etc., are displayed on the smartphone or the like. These consist of "dish name," "ingredients," "how to make it," etc.

[0070] If the recognition unit determines that the recognized food item is one that has been stored in the refrigerator, it updates the information corresponding to the food name of the recognized food item in the identified record for any food items that have been stored there in the past.

[0071] If this system determines that a food item has been taken out of a refrigerator, it updates the corresponding information in the identified record, including the food name of the recognized food item, and can operate on the user's mobile device.

[0072] Furthermore, if the food recognition unit determines that the recognized food item is one that has been stored in the refrigerator, it will associate new information with the user of the identified record for any food items that have not been stored there before.

[0073] Furthermore, the food recognition unit determines whether the recognized food item was placed in the refrigerator or removed from the refrigerator based on the in / out determination result information included in the received food management request information, and displays the result on the display unit.

[0074] Furthermore, the food recognition unit obtains the expiration date based on the date and time information included in the food management request information through predetermined methods such as calculations and queries, and performs processing to change the expiration date indicated by the expiration date information to the obtained expiration date.

[0075] Then, when the food recognition unit determines that it has received food management request information, it performs food recognition processing based on the cropped food image indicated by the image data included in the food management request information.

[0076] The recipe suggestion system according to the present invention transmits food images included in food management request information to a device that stores a trained model, and obtains the food identification result from this device. The system then recognizes the expiration date information with a recognition unit and obtains the expiration date information of the foods in order of shortest expiration date. Furthermore, it includes a questioning unit that asks questions about the date and time based on the date and time information, and a suggestion unit in which the character suggests a recipe using the food based on the answer information from the user or the mobile terminal corresponding to the suggestion and the expiration date information.

[0077] A recipe suggestion system comprising: a camera unit equipped on a user's mobile device to photograph the expiration dates of food stored in the refrigerator; a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date information; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests recipes using the food items based on the user's or mobile device's response information corresponding to the suggestion and the expiration date information. The food recognition unit recognizes the food item indicated by the acquired specific result as the food item indicated by the cropped food image.

[0078] This system works, for example, through the following steps: First, the user takes a picture of the expiration date of food in their refrigerator and sends the image data to their mobile device. Next, the recognition unit obtains the food name information from the image data and then obtains the expiration date information for that food from that food name information. The display unit then displays the expiration date information for the food along with a character, and the suggestion unit then suggests a recipe using that food. The user can respond if they like the recipe, or request a different recipe if they don't. A recipe suggestion system as described in claim 1, characterized in that the recipe suggestion is made by a character, a voice recognition function is provided on the screen on which the character is displayed, and response information is generated based on the content voice-input by the user. It is equipped with a voice recognition function, and answer information is generated from what the user inputs by voice.

[0079] This system is equipped with a voice recognition function, which generates response information from the user's voice input. For example, in response to a question such as "Are there any simpler dishes?", it will suggest recipes for simple dishes such as "omelet rice" or "oyakodon." It can also accept requests such as "I want to eat a meat dish today." In this case as well, the voice recognition function is used, and a recipe for a meat dish will be suggested based on the keyword "meat dish." In the recipes, essential ingredients and ingredients that are not essential but can be used to add or change the flavor are displayed separately. Recipes may combine multiple menus, such as a main dish and side dishes, rice, and a main dish and miso soup, like a course meal, and in such cases, a combination of recipes for multiple menus will be suggested. The recipe suggestion system according to the present invention can also photograph food in the pantry and freezer in addition to the food in the refrigerator, and can manage inventory including these items. With this function, users can receive recipe suggestions that combine essential ingredients in the refrigerator with food stored in the pantry and freezer. For example, a recipe might suggest chicken stored in the refrigerator as an essential ingredient for the main dish, while canned tomatoes in the pantry or frozen vegetables in the freezer are displayed as optional ingredients. This allows users to focus on the essential ingredients in their refrigerator and then use ingredients from their pantry and freezer to add or change flavors or enhance side dishes. Furthermore, the suggested recipes include a feature that visually distinguishes between essential and optional ingredients. Specifically, essential ingredients are highlighted as indispensable for the main cooking steps of the recipe, while optional ingredients are displayed with different colors or marks as elements that can be added or changed according to the user's preferences or ingredient availability. For example, in a recipe for chicken curry, a main dish, chicken, onions, and spices are listed as essential ingredients, while tomatoes and eggplants are listed as optional ingredients to "add as you like." This distinction makes it easy for users to customize recipes to their liking and also helps prevent food waste. This system allows users to maximize the use of food in their refrigerator, pantry, and freezer, and receive more flexible and diverse meal suggestions. This system allows users to receive recipe suggestions based on specific foods, not only for single dishes but also for combinations of multiple menu items. For example, combinations such as a main dish and a side dish, rice, a side dish and miso soup, or curry and lassi are considered. These combinations of multiple menu items make it easy to plan a balanced meal. Furthermore, even if a user requests a combination of dishes based on a specific theme or preference, the system will respond flexibly to that request and suggest relevant recipes.

[0080] When the food management server receives food management request information, it transmits it to the food recognition unit via the server communication unit. The food recognition unit then retrieves the necessary recipe data from the recipe database based on the received food information and user information. From the retrieved recipe data, the server extracts the necessary ingredients and procedures. A recipe screen is generated and displayed from the extracted ingredients and procedures. If the user makes a selection, a detailed screen for the selected recipe is generated and displayed. A refrigerator food management device that can capture and acquire expiration date information of food inside the refrigerator, and then suggest the types and quantities of food that need to be consumed as soon as possible based on that information and the date and time information, comprises a recognition unit that automatically recognizes information such as the types and quantities of food inside the refrigerator from image data captured by a camera installed inside the refrigerator, a display unit that displays suggestions for the types and quantities of food that need to be consumed as soon as possible based on the recognized information such as the types and quantities of food inside the refrigerator and the date and time information, and, if selected by the user, generates and displays a detailed screen of the selected recipe. Referring to the flowchart, the server control unit determines whether the request received by the server communication unit was sent from the food management device inside the refrigerator.

[0081] A recipe suggestion system comprising: a camera unit equipped on a user's mobile device to photograph the expiration dates of food stored in the refrigerator; a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests recipes using the food items based on the user's or mobile device's response information and the expiration date information corresponding to the suggestion. This system determines the expiration date of food items from the expiration date information of food items photographed with the user's mobile device, displays the food items in order of their expiration date, and suggests recipes using those food items. It also suggests recipes taking into account the user's responses and date and time information. Furthermore, this system acquires and manages data such as the date and time of purchase and the place of purchase from the food data photographed with the user's mobile device. The recipe suggestion system according to claim 1, characterized in that the recipe suggestion is made by a character, and the character is an animated character. A recipe suggestion system comprising: a camera unit equipped on a user's mobile device for photographing the expiration dates of food stored in the refrigerator; a recognition unit for recognizing the expiration date information of food items in order of shortest expiration date based on the expiration date information; a display unit for displaying the expiration date information and an animated character that makes suggestions regarding the food items; a question unit for asking questions about the date and time based on the date and time information; and a suggestion unit for which the animated character suggests recipes using the food items based on the user's or mobile device's response information and the expiration date information corresponding to the suggestion. This system is the same as the one described in claim 1, but with the addition of an "animated character". This animated character moves around cutely while suggesting recipes and confirming date and time information. This makes cooking more enjoyable for the user.

[0082] Then, when the user selects one candidate recipe from the candidate recipes displayed on the terminal device's display unit using the control unit, and the control unit receives this operation, the recipe suggestion unit transmits the selected candidate recipe from the communication unit to the terminal device, suggesting detailed information to the user about the candidate recipe selected by the user, such as the name of the dish, how to make it, and the ingredients used in the dish.

[0083] The recipe suggestion system works as follows: After the suggestion unit extracts candidate recipes, it processes them to select the best recipe from the candidates and displays it to the user. It extracts candidate recipes from combinations of available ingredients and creates a candidate recipe list. From the candidate recipe list, it creates a list of the top N candidate recipes with the highest evaluation scores, and then creates and displays the best recipe list from these top N candidate recipes with the highest evaluation scores. From the best recipe selected by the user, it creates and displays a list of necessary ingredients and necessary cooking utensils. From the best recipe selected by the user, it creates and displays a shopping list from the necessary ingredients and necessary cooking utensils. The system calculates and displays an evaluation score for budget and nutritional balance based on the ingredients and cooking utensils the user has gathered from the shopping list, which helps the user decide on their shopping and cooking plans. A recipe suggestion system as described in claim 1, wherein the camera unit is capable of photographing not only food in the refrigerator but also food in the pantry, freezer, etc. The recipe suggestion system according to the present invention can take photos not only inside the refrigerator but also inside the pantry, freezer, and other areas.

[0084] The recipe suggestion department identifies and suggests recipes in order of the fewest additional ingredients the user needs to purchase, based on the information on surplus ingredients (i.e., ingredients currently owned) held for the user in the ingredient management system and the information on ingredients used in each record held in the recipes.

[0085] The recipe suggestion system proposes recommended recipes to the user based on the weight ratio of the ingredients in stock, which is obtained by the weight ratio acquisition unit, and the recipe information stored in the recipe database.

[0086] When identifying the above-mentioned recipe information, the recipe suggestion unit prioritizes selecting ingredients that the user currently possesses as surplus ingredients after cooking and that are nearing their expiration date, using ingredient management, and then identifies recipes that use those ingredients from the recipes. Furthermore, the present invention is characterized by the interaction of the above-mentioned components.

[0087] Furthermore, the recipe suggestion unit identifies recipes that can be made with the surplus ingredients, based on the information on surplus ingredients held by the ingredient management system described above, from among the recipe information within the recipes, and outputs that recipe information to the user terminal.

[0088] The recipe suggestion system, specifically the recipe suggestion unit, distributes and displays the recipe information identified in the above process to the user's terminal, and then terminates its processing.

[0089] By taking a photo of the expiration date of food stored in the refrigerator and having it recognized by a mobile device, the system can obtain the expiration date information of the stored food and display recipes suggested by a character. Users can also input information such as the planned date and purpose of use of the food in the refrigerator, and the system will suggest recipes based on this information. The recipe suggestion system according to claim 1, characterized in that the recipe suggestion is made by a character, and the character is an animated character. This recipe suggestion system displays animated characters, allowing users to enjoyably acquire cooking ideas. Furthermore, the animated characters also teach cooking procedures and tips, making it easy for even beginners to cook. The recipe suggestion system according to the present invention comprises: a shooting unit provided in a mobile terminal owned by the user; a recognition unit that recognizes the expiration date information of food products in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and an animated character; a questioning unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the animated character suggests a recipe using the food product based on the answer information from the user or the mobile terminal corresponding to the suggestion and the expiration date information.

[0090] In this recipe suggestion system, the combination of an equipment adapter installed in the refrigerator and a terminal device is registered (paired) with the server, enabling the refrigerator, server, and terminal device to communicate with each other.

[0091] The recipe suggestion system generates recipe candidates from selected ingredients in stock, and can randomly select one recipe from the candidates generated from the ingredients selected by the user and display it on the display unit. It can also randomly select one recipe from the candidates generated from the ingredients selected by the user and display it on the display unit.

[0092] A recipe suggestion system comprising: a camera unit equipped on a user's mobile device to photograph the expiration dates of food stored in the refrigerator; a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date information; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests recipes using the food items based on the user's or mobile device's response information and the expiration date information corresponding to the suggestion. The display unit can display multiple candidate recipes, or it can display only the names of the dishes.

[0093] This system is envisioned to be integrated into household appliances such as refrigerators and freezers. It can also be operated on mobile devices such as smartphones and tablets. Claim 1 describes a recipe suggestion system characterized in that a character suggests recipes, and the recipes suggested by the character take into consideration information such as cooking utensils and seasonings owned by the user. The recipe suggestion system of the present invention can suggest recipes while also taking into account information such as the cooking utensils and seasonings that the user owns. For example, in response to a user's request, "I want to eat a hot pot dish today," the system can suggest "a hot pot dish that can be made with the pot you own." Also, in response to a user's request, "I want to eat a vegetable-centered dish today," the system can suggest "a dish that can be made with the vegetables you own." In this way, the recipe suggestion system of the present invention can suggest recipes while also taking into account information such as the cooking utensils and seasonings that the user owns, so it can present recipes that are easy for the user to actually make and that turn out delicious. Claim 1 describes a recipe suggestion system characterized in that the recipe suggestion is made by a character, and the character is an animated character. The recipe suggestion system of this invention displays animated characters. These animated characters have a cute design, making them approachable. Furthermore, the animated characters also exhibit movement.

[0094] Furthermore, the recipe suggestion method according to the present invention involves taking a photograph of the expiration date of food stored in a refrigerator, recognizing the expiration date information of the food items in order from shortest to oldest from the image taken with the user's mobile device, displaying the expiration date information and a character that makes suggestions regarding the food items, asking questions about the date and time based on the date and time information, and suggesting a recipe based on the answer information and the expiration date information.

[0095] This recipe suggestion system comprises: a camera unit equipped on a user's mobile device to photograph the expiration dates of food stored in the refrigerator; a recognition unit that recognizes the expiration date information of the food items in order from shortest to oldest based on the expiration dates; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit that displays only cooking recipes that allow the user to make the desired amount of food using the available ingredients, based on the user's or mobile device's response information and the expiration date information, and in which the character suggests recipes using the food items.

[0096] At this time, the LCD screen displays the current date, day of the week, and time measured by the timing unit, as well as the ambient temperature detected by the temperature sensor.

[0097] In this recipe suggestion system, the image displayed on the display unit is the same as the image displayed on the mobile device's display unit (Figure 1).

[0098] The display unit shows a recipe button for each food item, and determines whether or not the corresponding recipe button has been pressed. Furthermore, the recipe suggestion method according to the present invention is implemented using the recipe suggestion system described above.

[0099] Furthermore, the recipe suggestion method according to the present invention is implemented using the recipe suggestion system described above.

[0100] A recipe suggestion system comprising: a camera unit on a user's mobile device that photographs the expiration dates of food stored in the refrigerator; a recognition unit that recognizes the expiration date information of the food in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes suggestions regarding the food; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests a recipe using the food based on the answer information from the user or the mobile device corresponding to the suggestion and the expiration date information. This recipe suggestion system operates, for example, in the following procedure: First, the user photographs the expiration dates of the food in the refrigerator and has the mobile device recognize them. Next, a question such as "What would you like for dinner tonight?" is displayed on the screen where the character is displayed. The user selects and answers from among them. After that, a recipe suggested by the character is displayed. In this way, the user can easily select a recipe from the ingredients in the refrigerator.

[0101] Claim 1 is a recipe suggestion system characterized in that the character is an animated character. The display of animated characters makes recipe selection more enjoyable for the user. This recipe suggestion system comprises: a camera unit equipped on a user's mobile device that photographs the expiration dates of food stored in the refrigerator; a recognition unit that recognizes the expiration date information of the food items in order from shortest to oldest based on the expiration dates; a display unit that displays the expiration date information and an animated character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the animated character suggests recipes using the food items based on the user's or mobile device's response information and the expiration date information corresponding to the suggestion.

[0102] In this system, the input unit may be a physical keyboard or other switch-based input device, or it may be integrated with the display unit and configured as a touch panel.

[0103] The recipe suggestion system according to the present invention may be implemented in the following forms, for example. (1) Take a picture of the expiration date of the food inside the refrigerator and have the user recognize it on their mobile device. (2) Based on the recognized expiration date information, create and display a list of foods in order of shortest expiration date. (3) When a user asks "I want to cook something" regarding a food item selected from the displayed list, when a user answers "I want to make a specific dish", or when a user answers "Please tell me if there are any ingredients or seasonings", the system will suggest an appropriate recipe. (4) Depending on whether a user responds to a proposed recipe with "I like this recipe" or "I don't like this recipe," the content of future proposals will be changed accordingly.

[0104] As shown in Figure 1, the recipe suggestion system according to the present invention comprises: a shooting unit provided in a user's mobile terminal for photographing the expiration dates of food stored in a refrigerator; a recognition unit for recognizing the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit for displaying the expiration date information and a character that makes suggestions regarding the food items; a questioning unit for asking questions about the date and time based on the date and time information; and a suggestion unit for having the character suggest a recipe using the food items based on the user's or mobile terminal's response information corresponding to the suggestion and the expiration date information. Furthermore, the expiration date warning date and time setting display control means displays the warning dates and times of "1 week ago," "3 days ago," "2 days ago," and "the day before" generated by the expiration date warning date and time item generation means on the liquid crystal display unit, and sets the expiration date and time of the food item by touching one of the warning dates and times with the input means corresponding to the food item to be stored.

[0105] In this case, the display area for each food item will show a pig illustration, text, and a warning date and time for the expiration date, as shown in the enlarged display area for pork. The warning date and time will be displayed in a color-coded or flashing manner according to the setting.

[0106] In this case as well, the information may be the date and time obtained from the server device's internal clock, or it may be the date and time sent by the user terminal along with the tag.

[0107] The recipe suggestion method according to the present invention involves taking a photograph of the expiration date of food stored in a refrigerator, recognizing the expiration date information of the food items in order from shortest to oldest from the image taken with the user's mobile device, displaying the expiration date information and a character that makes suggestions regarding the food items, asking questions about the date and time based on the date and time information, and suggesting a recipe based on the answer information and the expiration date information.

[0108] A recipe suggestion system comprising: a camera unit equipped on a user's mobile device to photograph the expiration dates of food stored in the refrigerator; a recognition unit that recognizes the expiration date information of food items in order of shortest expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes suggestions regarding the food items; a question unit that asks questions about the date and time based on the date and time information; and a suggestion unit in which the character suggests a recipe using the food items based on the user's or mobile device's response information corresponding to the suggestion and the expiration date information. The expiration date warning date and time item generation means generates a format in which a predetermined period before the food expires is divided into multiple stages as a warning date and time, for example, "1 week ago," "3 days ago," "2 days ago," and "the day before."

[0109] The recipe suggestion system according to the present invention may be implemented in the following forms, for example. (1) Take a picture of the expiration date of the food inside the refrigerator and have the user recognize it on their mobile device. (2) Based on the recognized expiration date information, a list of foods in order of shortest expiration date is created and displayed. (3) If a user asks "I want to cook something" about a food item selected from the displayed list, the system will continue to suggest appropriate recipes while asking additional questions such as "Are there any conditions?". (4) If the user is missing ingredients for a recipe they have selected, a shopping list or similar will be created to help the user with their shopping. (5) It is also possible to obtain user profile information such as "Is today a special day?" and suggest recipes, shopping lists, etc., taking into account the date and time.

[0110] Furthermore, the recipe suggestion method according to the present invention is implemented using the recipe suggestion system described above.

[0111] The present invention provides a portable terminal that photographs the expiration dates of food stored in a refrigerator. This portable terminal comprises a recognition unit that recognizes expiration date information, a display unit that displays a character that makes suggestions about the food, a question unit that asks questions based on date and time information, and a suggestion unit that suggests recipes based on the answer information and expiration date information. The recipe display system comprises a food recognition unit that acquires the type of food stored in the refrigerator from an image of the refrigerator taken from the front and above of the refrigerator, and a terminal device that displays recipe information corresponding to the food recognized by the food recognition unit on a touch panel (display unit).

[0112] Then, when the user selects one candidate recipe from the candidate recipes displayed on the terminal device's display unit using the control unit, and the control unit receives this operation, the recipe suggestion unit transmits the selected candidate recipe from the communication unit to the terminal device, suggesting detailed information to the user about the candidate recipe selected by the user, such as the name of the dish, how to make it, and the ingredients used in the dish.

[0113] As shown in Figure 1, the recipe suggestion system according to the present invention comprises a mobile terminal held by the user, a shooting unit that photographs the expiration dates of food stored in a refrigerator, a recognition unit that recognizes the expiration date information of the food in order of shortest expiration date based on the expiration date, a display unit that displays the expiration date information and a character that makes suggestions regarding the food, a questioning unit that asks questions about the date and time based on the date and time information, and a suggestion unit in which the character suggests a recipe using the food based on the answer information from the user or the mobile terminal corresponding to the suggestion and the expiration date information. Furthermore, the recipe suggestion method according to the present invention is carried out using the recipe suggestion system described above.

[0114] The recipe suggestion system, after the suggestion unit extracts candidate recipes, performs the following processing to select the optimal recipe from the candidate recipes and displays and provides the optimal recipe to the user. Candidate recipes are extracted from combinations of available ingredients to create a candidate recipe list. A list of the top N candidate recipes with the highest evaluation scores is created from the candidate recipe list, and the optimal recipe list is created and displayed from the top N candidate recipes with the highest evaluation scores.

[0115] The system creates and displays a list of necessary ingredients and cooking utensils from the user's selected optimal recipe list. It also creates and displays a shopping list based on the user's selected ingredients and cooking utensils. Finally, it calculates and displays shopping budget and nutritional balance information based on the user's shopping list and the necessary ingredients and cooking utensils. Furthermore, the recipe suggestion method according to the present invention involves taking a photograph of the expiration date of food stored in a refrigerator and suggesting a recipe based on that information.

[0116] A recipe suggestion system according to claim 1, comprising: a filtering unit that narrows down the recipes suggested by the character on a screen in which the character is displayed, based on search conditions consisting of the type of ingredients and cooking utensils selected by the user and conditions such as cooking time; and a display unit that displays one or more recipes randomly selected from the narrowed-down recipes. For example, you can search using criteria such as "chicken," "frying pan," and "within 30 minutes," and recipes like "fried chicken" or "stir-fried chicken and vegetables" will be displayed.

[0117] The recipe suggestion system according to the present invention includes a filtering unit that photographs the expiration dates of food stored in a refrigerator, suggests recipes based on that information, and performs a narrowing process based on search conditions consisting of the type of ingredients and cooking utensils and cooking time selected by the user on the screen displayed at that time, and a display unit that displays one or more recipes randomly selected from the narrowed-down recipes. The recipe suggestion method according to the present invention is a method of searching based on specified conditions from ingredients and cooking utensils in the refrigerator and randomly selecting and displaying one or more recipes from the results.

Claims

[Claim 1] A recipe suggestion system comprising: a camera unit equipped on a user's mobile device for photographing the expiration dates of food stored in the refrigerator; a recognition unit for recognizing the expiration date information of food items in order of shortest expiration date based on the expiration date information; a display unit for displaying the expiration date information and suggestions related to the food items; a question unit for asking questions about the date and time based on the date and time information; and a suggestion unit for suggesting recipes using the food items based on the user's or mobile device's response information corresponding to the suggestion and the expiration date information.

Images