Recipe Proposal System

Abstract

To propose an efficient and waste-free recipe and provide a system that is more familiar and user-friendly for the user. 【Solution means】 A recipe proposal system comprising: a photographing unit provided in a mobile terminal owned by a user that photographs the expiration date of food stored in a refrigerator; a recognition unit that recognizes the expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit that displays the expiration date information and a proposal regarding the food; a question unit that asks a question regarding the date and time based on the date and time information; and a proposal unit that proposes a recipe using the food based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information.

Description

Technical Field

[0001] The present invention relates to a system that recognizes expiration date information of food and proposes recipes.

Background Art

[0002] The present invention reads expiration date information of food with a photographing unit, processes it with a recognition unit, and acquires information on foods in ascending order of the expiration date. Then, characters are displayed on a display unit, questions regarding date and time are asked in a question unit, and based on the response information and the expiration date information, a recipe using the food can be proposed in a proposal unit. In this way, it is a system that utilizes the expiration date information of food using a portable terminal owned by the user and proposes recipes.

Prior Art Documents

Patent Documents

[0003] Japanese Patent Application Laid-Open No. 2022-076493

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to utilize the expiration date information of food and propose recipes efficiently and without waste using a portable terminal owned by the user. Specifically, it is to provide a system that acquires information on foods in ascending order of the expiration date and proposes a recipe using the food based on the response information and the expiration date information. It is also an object to provide a system that is more familiar and easier to use for the user by having a character propose a recipe.

Means for Solving the Problems

[0005] The present invention reads the expiration date information of food with a photographing unit, processes it with a recognition unit, and acquires information on foods in ascending order of the expiration date. Then, it can display characters on a display unit, ask questions regarding the date and time at a question unit, and propose recipes using the foods at a proposal unit based on the response information and the expiration date information. In this way, it is a system that utilizes the expiration date information of foods and proposes recipes using a portable terminal owned by the user. Also, by having a character propose recipes, it aims to provide a system that is more familiar and easier to use for the user.

[0006] Furthermore, the proposed recipe is determined from a recipe database pre-registered in the system based on the information answered by the user and the expiration date information. The present invention can utilize the expiration date information of foods to propose recipes more efficiently and without waste. Also, by having a character propose recipes, it can provide a system that is more familiar and easier to use for the user.

Effects of the Invention

[0007] The present invention can utilize the expiration date information of foods to propose recipes more efficiently and without waste. Also, by having a character propose recipes, it can provide a system that is more familiar and easier to use for the user. The proposed recipe is determined from a recipe database pre-registered in the system based on the information answered by the user and the expiration date information.

Modes for Carrying Out the Invention

[0008] The present invention reads the expiration date information of food with a photographing unit, processes it with a recognition unit, and acquires information on food in ascending order of the expiration date. Then, it can display characters on a display unit, ask questions regarding the date and time at a question unit, and propose recipes using the food at a proposal unit based on the response information and the expiration date information. In this way, it is a system that utilizes the expiration date information of food and proposes recipes using a portable terminal owned by the user. Also, by having a character propose recipes, it aims to provide a system that is more familiar and easier to use for the user.

[0009] Furthermore, the proposed recipe is determined from a recipe database pre-registered in the system based on the information answered by the user and the expiration date information. The present invention can utilize the expiration date information of food to propose recipes more efficiently and without waste. Also, by having a character propose recipes, it can provide a system that is more familiar and easier to use for the user. The proposed recipe is determined from a recipe database pre-registered in the system based on the information answered by the user and the expiration date information.

Brief Description of the Drawings

[0010]

Figure 1

[0011] The definitions of the keywords in the claims are as follows.

[0012] The expiration date information of food is the information on the expiration date described on the food.

[0013] The recognition unit is the part that reads and processes the expiration date information of food with a photographing unit.

[0014] The proposal unit is the part that proposes a recipe using the food based on the information on the food in ascending order of the expiration date and the response information.

[0015] Usage Example

[0016] The recipe proposal system of the present invention can, for example, efficiently and waste - free propose recipes when ingredients are left over without being used up at home or when the expiration date of purchased ingredients is approaching. For example, it can obtain information on foods in the order of shortest expiration date and, based on the response information and expiration date information, the proposal section can propose recipes using the foods. For example, it can propose recipes for soups or stews using vegetables whose expiration date is approaching.

[0017] The recipe proposal system of the present invention reads the expiration date information of foods by the photographing section as shown in FIG. 1, processes it by the recognition section, and obtains information on foods in the order of shortest expiration date. Then, it can display characters on the display section, ask questions regarding the date and time in the question section, and based on the response information and expiration date information, the proposal section can propose recipes using the foods. The proposed recipes are determined from the recipe database pre - registered in the system based on the information answered by the user and the expiration date information. Also, by having a character propose recipes, the system provides a system that is more familiar and easier to use for the user.

[0018] The recipe proposal system of the present invention can be implemented on devices such as mobile terminals and smartphones and can be used at home, in restaurants, etc.

[0019] Hereinafter, a recipe proposal system for recognizing the expiration date information of foods according to the present embodiment will be described.

[0020] The present invention relates to a recipe proposal system comprising: a photographing unit provided in a portable terminal owned by a user for photographing the expiration date of food stored in a refrigerator as shown in FIG. 1; a recognition unit for recognizing, based on the expiration date, the expiration date information of foods in ascending order of the expiration date; a display unit for displaying the expiration date information and making proposals regarding the food; a questioning unit for asking questions regarding the date and time based on the date and time information; and a proposal unit for proposing a recipe using the food based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information. The present invention can recognize the expiration date information of foods in the refrigerator and propose a recipe based on that information. Further, when the user answers a question regarding the date and time, a recipe is proposed taking into account the answer result, so that a more appropriate recipe can be proposed.

[0021] When the user performs an operation of designating a transfer destination and instructing transmission to the transfer destination (for example, a terminal device) using the response text data displayed on the display screen and a transfer application such as an email or messaging app, the refrigerator management app unit instructs the transmission app unit to transfer the response text data selected based on the operation received by the operation reception unit to another device so that the recipe proposal system can be used on the other device.

[0022] A portable terminal application is provided for photographing the expiration date of food stored in a refrigerator. This application recognizes the expiration date from the photographed image and displays the foods in ascending order of the expiration date. The object of photographing is the part containing information regarding the expiration date of the food, which may also be the receipt at the time of purchasing the food. Further, based on the date and time information, questions regarding the date and time can be asked. Based on the information answered by the user and the expiration date information, a recipe using the food can be proposed. The proposed recipe is presented in an easy-to-understand manner to the user by displaying characters.

[0023] A portable terminal is provided that photographs the expiration dates of food ingredients stored in a refrigerator. This portable terminal recognizes the expiration date information of foods in ascending order of the expiration date based on the expiration date. Also, based on the date and time information, it asks questions regarding the date and time. Furthermore, based on the response information corresponding to the proposal and the expiration date information, it proposes recipes using the foods.

[0024] When the user instructs to sort (rearrange) the response text data displayed on the display screen in ascending order of the expiration date or the consumption date, the refrigerator management application unit rearranges and displays the pork, chicken, pudding from Company A, and onions included in the response text data in ascending order of the expiration date or the consumption date. Also, it is possible to search for and display recipes from the food ingredients selected by the user.

[0025] Also, in this embodiment, a customer purchases goods worth more than a predetermined amount per month from the seller, and when the seller's monthly purchase amount by the customer exceeds the predetermined amount, the seller conducts a discounted settlement from the total amount according to the excess amount, and a contract is made such that a predetermined product price discount point (point) and a coupon are added. In this case, in this embodiment, the seller confirms that the customer's monthly purchase amount is more than the predetermined amount, and in that case, can provide the recipe proposal system to the customer for free. In such a form, the seller can not only enhance the sales promotion effect of its own products, but also the customer can use the recipe proposal system including products other than its own products for free, so the convenience can be improved.

[0026] A recipe proposal system in which the control unit of the refrigerator performs voice recognition processing and transmits the analysis result of voice data including an inquiry to the server system. (Modification Example 4) In the above description, the expiration date information of foods is recognized by the portable terminal held by the user, but instead, the expiration date information of foods may be recognized by a camera provided in the refrigerator. A recipe proposal system that recognizes the expiration date information of foods by a camera provided in the refrigerator.

[0027] When there is a terminal provided in relation to the refrigerator door, if the display unit is constantly operating, power is wasted. Therefore, for example, when the refrigerator door is opened, the display function is shown to operate.

[0028] The recipe proposal system according to the present invention is such that the refrigerator management application section displays a list of inquiry contents (for example, "want to make something", "is there any recommendation", etc.), and the recipe proposal section proposes a recipe according to the inquiry content selected by the user, displays the ingredients and procedures of the recipe selected by the user, the refrigerator management application section automatically checks the ingredients and procedures of the recipe selected by the user, notifies the user if there are insufficient ingredients, etc., and can also display a list of necessary ingredients, etc. The recipe proposal system described in claim 1, wherein the proposal of the recipe is made by a character, the character is an animation character, and the animation character is capable of voice output by voice synthesis technology. For example, when the inquiry content of "want to make something" is selected, it is voice-output such as "Do you have a preference for a specific genre or type?"

[0029] And the recipe proposal system of the present invention is characterized in that when the total amount of food in the refrigerator is displayed, the food already stored in the refrigerator is subtracted.

[0030] The refrigerator management app section receives, stores, and displays response text data including food lists and recipes in response to inquiries such as checking the ingredients stored in the refrigerator, checking those ingredients with approaching expiration dates or consumption dates, and checking recipes that can be made with the ingredients stored in the refrigerator. Also, using the camera unit provided in the mobile terminal owned by the user, it photographs the expiration date of the food stored in the refrigerator, and based on the expiration date, recognizes the expiration date information of the food in ascending order of the short expiration date. Furthermore, it displays the expiration date information and a character that proposes a suggestion regarding the food, and asks a question regarding the date and time based on the date and time information. Then, based on the response information from the user or the mobile terminal corresponding to the suggestion and the expiration date information, it proposes a recipe using the food.

[0031] The food information recording means is configured to be able to record at least information on the name of Food A (an example of a food product), information on the inventory quantity of the food, information on the expiration date of the food, information on the consumption date of the food, information on the price (list price) of the food, information on the purchaser of the food, information on the quantity of the food held by the purchaser who purchased the food (in other words, the number of foods that the purchaser has purchased but has not yet consumed), and information on the producer of the food.

[0032] The food expiration date management device includes a food expiration date management / processing menu generation section, menu display control means, input means, a food item storage section, food item display control means, food item selection control means, a selected food item storage section, a calendar generation section, calendar display control means, expiration date setting control means, a set expiration date storage section, expiration date warning date and time item generation means, expiration date warning date and time setting and display control means, a warning date and time storage section, food storage status calculation means, food inventory list display control means, data deletion means, and a timing section and a liquid crystal display section. This device can set and manage the expiration date of food. Also, it can issue a warning when the expiration date approaches. Furthermore, it can check the inventory status of the stored food. Thereby, the management of food within the home is made easy. A recipe proposal system comprising: a photographing unit that photographs the expiration date of food stored in a refrigerator; a classification unit that classifies the food in the refrigerator based on the expiration date; a proposal unit that proposes a recipe for the classified food; a listing unit that lists up the ingredients required for the proposed recipe; a notification unit that notifies the user of the lacking ingredients from the listed ingredients; and an ordering unit that automatically orders the selected ingredients when the user selects the notified ingredients. The present invention relates to a method for managing food ingredients in a refrigerator and a recipe proposal system based on that method, which enables automatic classification of food ingredients in the refrigerator under specific conditions, and performs functions such as recipe proposal and automatic ordering of lacking ingredients. The present invention relates to a method for managing food ingredients in a refrigerator and a recipe proposal system based on that method, which enables automatic classification of food ingredients in the refrigerator under specific conditions, and performs functions such as recipe proposal and automatic ordering of lacking ingredients.

[0033] The recipe proposal method according to the present invention is to recognize the expiration date information of food stored in a refrigerator, calculate the number of elapsed days from that information, judge the freshness state of the food from the number of elapsed days, determine the content of the proposed recipe from the freshness state, and display the content of the proposed recipe to the user. A refrigerator management system that can acquire information such as the expiration date and inventory quantity of food ingredients in the refrigerator, and automatically create a menu, shopping list, etc. from this information and present it to the user. Note that "food ingredients" include "vegetables", "fruits", "meat", "seafood", "eggs and dairy products", "processed foods", etc. The present invention is a refrigerator management system that can acquire information such as the expiration date and inventory quantity of food in the refrigerator, and automatically create a menu, shopping list, etc. from this information and present it to the user. In the present invention, information such as the name of the food and the expiration date is automatically extracted from the image data captured by a camera or the like installed inside the refrigerator, and stored in a database. Various analysis processes and the like are performed on this database. For example, a menu, shopping list, etc. are automatically created from items whose expiration date is approaching or whose inventory quantity is decreasing, and presented to the user. Also, in the present invention, it is accessible from a smartphone or the like, and it is possible to check what is currently needed inside the refrigerator even when away from home, and to check the shopping list and the like. In addition, in the above, an example was given of acquiring information such as the name of the food and the expiration date from the image data captured by a camera or the like installed inside the refrigerator, but it is not necessary for the camera to be installed inside the refrigerator. The object of shooting is the part including information regarding the expiration date of the food, and it may be the receipt when purchasing the food.

[0034] An object of the present invention is to provide a recipe proposal system for managing the expiration date of food. According to the present invention, a shooting unit provided in a portable terminal owned by a user shoots the expiration date of food stored in the refrigerator, and a recognition unit recognizes the expiration date information. Then, a display unit displays the expiration date information of the food in ascending order of the expiration date, and a character that proposes something regarding the food. Furthermore, based on the date and time information, a question unit that asks a question regarding the date and time is provided, and when answer information is obtained from the user or the portable terminal, a proposal unit proposes a recipe using the food based on the answer information and the expiration date information.

[0035] The expiration date management device for food according to the present invention is formed, for example, with an outer dimension of 80 mm × 80 mm × 10 mm and can be detachably attached to the door of a refrigerator or the like by a magnet. Further, the expiration date management device for food according to the present invention includes a photographing unit, a recognition unit, a display unit, a question unit, and a proposal unit. The photographing unit photographs the expiration date of the food stored in the refrigerator. The recognition unit recognizes the expiration date information of the food in ascending order of the expiration date based on the expiration date. The display unit displays the expiration date information and a character that makes a proposal regarding the food. The question unit asks a question regarding the date and time based on the date and time information. The proposal unit proposes a recipe using the food by the character based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information.

[0036] The expiration date management device for food in the present embodiment is specifically composed of the computer shown in the figure. In the present embodiment, the computer includes a photographing unit, a recognition unit, a display unit, a question unit, and a proposal unit. The photographing unit is a photographing unit provided in a portable terminal held by the user. The recognition unit is a recognition unit that recognizes the expiration date information of the food in ascending order of the expiration date based on the expiration date. The display unit is a display unit that displays the expiration date information and a character that makes a proposal regarding the food. The question unit is a question unit that asks a question regarding the date and time based on the date and time information. The proposal unit is a proposal unit that proposes a recipe using the food by the character based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information.

[0037] In addition, the terminal device according to the present invention includes: a expiration date information acquisition means for acquiring, from a server device, the expiration date information of a product corresponding to a tag received from a detection device; an expiration date information storage means for storing the acquired expiration date information in association with the tag and the product ID; a display means for displaying, for a tag whose expiration date information stored in the expiration date information storage means satisfies a predetermined condition, information on the product specified by the product corresponding to the tag; a recipe storage means for storing recipe information which is information recording the cooking item, the type and amount of the product to be used; and a recipe proposal means for referring to the tag information and acquiring and proposing, from the recipe storage means, recipe information corresponding to the product corresponding to the tag whose expiration date has not reached a predetermined expiration date. Embodiments of the present invention are as described above.

[0038] The recipe proposal system according to the present invention includes: a photographing unit provided in a portable terminal held by a user; a recognition unit for recognizing, based on the expiration date, the expiration date information of foods in ascending order of the short expiration date; a display unit for displaying the expiration date information and a character for making a proposal regarding the food; a question unit for asking a question regarding the date and time based on the date and time information; and a proposal unit for proposing, based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information, a recipe using the food, which is proposed by the character. The user photographs the expiration date of the food stored in the refrigerator, the recognition unit proposes a recipe from the expiration date information recognized and the date and time information, and an optimal recipe is proposed from the content answered by the user.

[0039] Provided is a portable terminal application for photographing the expiration date of food stored in a refrigerator. This application recognizes the expiration date from the photographed image and displays the expiration date information of foods in ascending order of the short expiration date. In addition, based on the date and time information, a question unit for asking a question regarding the date and time, and based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information, a recipe using the food is proposed. This application may be inventory information input by the user or inventory information determined from an image of the inside of the refrigerator, or may be inventory information determined by a change in the weight of food stored in a predetermined location.

[0040] A recipe proposal system in which a control unit of a refrigerator performs voice recognition processing and transmits the analysis result of voice data including an inquiry to a server system. Although it is assumed that the expiration date information of food is recognized by a mobile terminal held by the user, instead, the expiration date information of food may be recognized by a camera provided in the refrigerator. A recipe proposal system that recognizes the expiration date information of food by a camera provided in the refrigerator.

[0041] The recipe proposal system according to the present invention includes a photographing unit provided in a mobile terminal held by a user, a recognition unit that recognizes the expiration date information of foods in ascending order of the expiration date based on the expiration date, a display unit that displays the expiration date information and a character that makes a proposal regarding the food, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food based on the response information from the user or the mobile terminal corresponding to the proposal and the expiration date information, and the character proposes it. The user photographs the expiration date of the food stored in the refrigerator, and the recognition unit recognizes the information and displays it on the display unit. Also, the user can select and actually make the recipe proposed by the character.

[0042] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram of a recipe proposal system according to an embodiment of the present invention. In this embodiment, a photographing unit provided in a mobile terminal held by a user photographs the expiration date of the food stored in the refrigerator, and transmits this photographed data to an information acquisition unit. The information acquisition unit recognizes the expiration date information of the foods in ascending order of the expiration date based on this photographed data, and transmits this information to the display unit. The display unit displays this expiration date information and a character that makes a proposal regarding the food. Also, a question unit that asks a question regarding the date and time based on the date and time information is provided, and a proposal unit that proposes a recipe using the food based on the response information from the user or the mobile terminal and the expiration date information is provided. A photographing unit provided in a mobile terminal owned by a user that photographs the expiration date of food stored in a refrigerator; a recognition unit that recognizes the expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes a proposal regarding the food; a question unit that asks a question regarding the date and time based on the date and time information; based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information, a recipe using the food is proposed, and when the recipe is selected, if the stock quantity of the ingredients to be used as the ingredients of the recipe is insufficient, it is displayed that the stock quantity of the insufficient ingredients is insufficient, it is notified that the insufficient ingredients are additionally required, and when the insufficient ingredients are additionally required, information such as the name of the store where the insufficient ingredients can be obtained and the method of obtaining them is presented. In the present invention, "surplus ingredients in the refrigerator" and "vegetables and fruits not used at home" are also targeted.

[0043] The recipe proposal system according to the present invention includes a photographing unit provided in a mobile terminal owned by a user; a recognition unit that recognizes the expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit that displays the expiration date information and a character that makes a proposal regarding the food; a question unit that asks a question regarding the date and time based on the date and time information; and a proposal unit that proposes a recipe using the food based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information, and is proposed by the character. Then, in the user terminal, the recipe proposal means realized by executing the application refers to the tag information in the recipe storage means, and extracts product information for all tags whose end status is not "ended" and the deadline indicated by the deadline information has not been reached. Then, a recipe is proposed and displayed from the extracted product information.

[0044] The recipe proposal system according to the present invention includes a photographing unit provided in a mobile terminal owned by a user, a recognition unit that recognizes expiration date information of foods in ascending order of the expiration date based on the expiration date, a display unit that displays the expiration date information and a character that proposes a recipe for the food, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food by the character based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information. The user photographs the expiration date of the food stored in the refrigerator, the recognition unit recognizes the expiration date information from the photographed image, and the display unit displays a character to make a proposal. The user can answer a question based on the date and time information and cook with reference to the proposed recipe.

[0045] And the user can, for example, display a shopping list stored in the storage unit of the server on the display unit of the terminal device to check the missing ingredients and the amount of the shortage, and can prevent forgetting to buy the ingredients that need to be purchased.

[0046] When specifying the above-mentioned recipe information, the present recipe proposal unit preferentially selects, in terms of food ingredient management, among the additional ingredients, those that are currently held as surplus ingredients even after cooking and whose expiration date is approaching, and specifies a recipe using the ingredient from the recipes. Further, the present invention is characterized in that the above-described components interact with each other.

[0047] Also, the cooking information from the menu search unit is displayed on the display unit, and one or more pieces of cooking information selected by the user are transmitted to the menu storage unit via the communication control unit.

[0048] Also, the recommended store search unit may search for the name and location of a recommended store based on the purchased food information from the purchased food search unit and the food inventory information of a plurality of food inventory information storage devices and notify the mobile terminal device. Furthermore, the present invention has the above-described technical features and is a recipe proposal system capable of managing food ingredients in a refrigerator and proposing recipes.

[0049] As shown in FIG. 1, the recipe proposal system according to the present invention includes a mobile terminal owned by a user, a refrigerator, and a server. The mobile terminal includes a photographing unit, a recognition unit, a display unit, a question unit, and a proposal unit. The refrigerator includes a storage unit for storing food and an output unit for outputting expiration date information of the food. The server includes a storage unit for storing recipe information and a generation unit for generating proposal information based on the recipe information. The photographing unit photographs the expiration date information of the food, the recognition unit recognizes the expiration date information of the food based on the expiration date information, and the display unit displays the recipe using the food based on the answer information by the user or the mobile terminal corresponding to the proposal and the expiration date information. The object of photographing by the photographing unit is a part including information related to the expiration date of the food, and it may be a receipt when purchasing the food. The question unit asks questions related to the date and time based on the date and time information. The proposal unit proposes a recipe based on the proposal information.

[0050] The mobile terminal device transmits an inventory information acquisition request for requesting acquisition of inventory information to the inventory information management device of the food inventory cooking information management device of the refrigerator via a cellular network, the Internet, a home network, and a communication control device.

[0051] The mobile terminal includes a communication unit, a display unit, an input unit, and a control unit. The control unit extracts expiration date information from the image data acquired from the photographing unit, transmits it to the recognition unit, receives the recognition result, and transmits it to the display unit to control the display of the character. The control unit also determines the question content from the date and time information, transmits it to the display unit, and controls the display of the character. Further, the control unit determines a recipe candidate from the answer content, transmits it to the proposal unit, and controls the display of the character.

[0052] When the mobile terminal acquires information about ingredients, a recipe is proposed based on that information. For example, when the ingredient "egg" is input, a recipe such as "egg dish" is proposed.

[0053] The mobile terminal device is equipped with a storage device that stores the information notified by the expiration date search unit. Since such a recipe proposal system is a mobile terminal owned by the user, it can be used even when the user is away from home. Also, it facilitates the management of the food in the refrigerator, leading to a reduction in food loss.

[0054] In addition, the mobile terminal device can transmit and receive information to and from the food inventory cooking information management device of the refrigerator via a cellular network, the Internet, and a home network.

[0055] The recipe proposal system according to the present invention is equipped with a storage device that stores the information notified by the food inventory cooking information management device. In this storage device, the expiration date information of food, the inventory status of food, recipe information, etc. are stored. Also, the recipe proposal system according to the present invention includes a photographing unit provided in a mobile terminal owned by the user, a recognition unit that recognizes the expiration date information of food in ascending order of the short expiration date based on the expiration date, a display unit that displays the expiration date information and a character that makes a proposal regarding the food, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food by the character based on the response information corresponding to the proposal by the user or the mobile terminal and the expiration date information.

[0056] In addition, the mobile terminal device is equipped with a storage device that stores the information notified by the recommended food for purchase search unit.

[0057] The photographing unit outputs the data of the moving image as the photographing result to the camera module control unit. The camera module control unit generates the data of the still image from the data of the moving image, extracts the expiration date information from the data of the still image, and outputs it to the recognition unit. Also, the recognition unit extracts the food name and the expiration date from the extracted expiration date information, outputs it to the display unit, and causes it to be displayed. Further, the proposal unit generates a recipe candidate from the response information and the expiration date information, and causes the character to propose and display it.

[0058] The imaging unit outputs the imaging results of each of the refrigerator compartment camera and the drawer camera to the camera module control unit. The camera module control unit extracts the food name and expiration date information from the imaging results, transmits them to the recognition unit, and performs recognition processing. Also, from the results after the recognition processing, the display unit displays the character and the recipe proposal screen, and the proposal unit proposes a recipe on the recipe proposal screen based on the content answered by the user.

[0059] The refrigerator is equipped with a refrigerator compartment camera and a drawer camera. The user uses a mobile terminal to photograph the expiration date of the food in the refrigerator. The recognition unit recognizes the expiration date information and displays the expiration date information of the food in ascending order of the expiration date. The display unit displays the expiration date information and a character that proposes a proposal regarding the food. The question unit asks a question regarding the date and time based on the date and time information. The proposal unit proposes a recipe using the food based on the answer information from the user or the mobile terminal corresponding to the proposal and the expiration date information. Here, an example of photographing the expiration date of the food in the refrigerator is given, but the object of photographing by the imaging unit is not limited to the things in the refrigerator, and it may be a receipt at the time of purchasing the food, which is a part including information regarding the expiration date of the food.

[0060] The present invention relates to an inventory management and recipe providing method and apparatus in a refrigerator, etc. By performing an inventory management and recipe providing method and apparatus in a refrigerator under specific conditions, it is intended to provide a technical means that enables a user to easily, efficiently, and enjoyably perform an inventory management and recipe providing method and apparatus in a refrigerator, etc.

[0061] Based on the purchase date of the corresponding food and the time information given from the clock, the expiration date of the corresponding food is calculated, and the name of the corresponding food and the expiration date of the corresponding food are displayed. Also, a recipe is proposed for the corresponding food selected by the user.

[0062] The food information recording means is configured to be able to record at least information on the name of Food A (an example of a food product), information on the stock quantity of the food, information on the expiration date of the food, information on the shelf life of the food, information on the price (list price) of the food, information on the purchaser of the food, information on the quantity of the food held by the purchaser who purchased the food (in other words, the number of foods that the purchaser has purchased but has not yet consumed), and information on the producer of the food. Further, the recipe proposal system includes a photographing unit that photographs the expiration date of the food stored in the refrigerator, a recognition unit that recognizes the expiration date information of the food in ascending order of the expiration date based on the expiration date, a display unit that displays a character that makes a proposal regarding the expiration date information and the food, a questioning unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food by the character based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information. And since the present invention has the above-described configuration, various data such as "the expiration date of the product" and "the price of the product" can be automatically grasped from "the products stored in the refrigerator" and "the expiration date of the product", and it is possible to provide a recipe proposal service optimized under the situation of "the products stored in the refrigerator".

[0063] The food expiration date management device is composed of a menu generation unit for food expiration date management and processing, a menu display control means, an input means, a food item storage unit, a food item display control means, a food item selection control means, a selected food item storage unit, a calendar generation unit, a calendar display control means, an expiration date setting control means, a set date storage unit, an expiration date warning date and time item generation means, an expiration date warning date and time setting and display control means, a warning date and time storage unit, a food storage status calculation means, a food inventory list display control means, a data deletion means, a timing unit, and a liquid crystal display unit. This device can set and manage the expiration date of food. Also, it can issue a warning when the expiration date is approaching. Furthermore, it can check the inventory status of the stored food. Thereby, the management of food within the household becomes easier. A recipe proposal system comprising a photographing unit that photographs the expiration date of food stored in a refrigerator, a classification unit that classifies the food in the refrigerator based on the expiration date, a proposal unit that proposes a recipe for the classified food, a listing unit that lists up the ingredients required for the proposed recipe, a notification unit that notifies the user of the ingredients that are lacking from the listed ingredients, and an ordering unit that automatically orders the selected ingredients when the user selects the notified ingredients. The present invention relates to a method for managing food ingredients in a refrigerator and a recipe proposal system based on that method, which enables automatic classification of the food ingredients in the refrigerator under specific conditions and performs functions such as recipe proposal and automatic ordering of lacking ingredients. The present invention relates to a method for managing food ingredients in a refrigerator and a recipe proposal system based on that method, which enables automatic classification of the food ingredients in the refrigerator under specific conditions and performs functions such as recipe proposal and automatic ordering of lacking ingredients.

[0064] As shown in FIG. 1, the recipe proposal system according to the present invention includes a photographing unit provided in a mobile terminal held by a user that photographs the expiration date of food stored in a refrigerator, a recognition unit that recognizes the expiration date information of foods in ascending order of the expiration date based on the expiration date, a display unit that displays the expiration date information and a character that makes a proposal regarding the food, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information, and the character proposes. Further, the recipe proposal system according to the present invention may have a configuration as shown in FIG. 1, and can automatically determine foods with an expiration date approaching from the types and amounts of foods in the refrigerator, and propose a recipe using the foods. Furthermore, the recipe proposal system according to the present invention may have a configuration as shown in FIG. 1, and can automatically determine foods with an expiration date approaching from the types and amounts of foods in the refrigerator held by the user, propose a recipe using the foods, automatically check for materials lacking from the ingredient list of the recipe selected by the user, create a list of lacking materials, display only the lacking materials from the list, and save it as a "shopping list". The present invention has the following configuration and the following effects. (1) It is possible to automatically propose a recipe from the expiration date information of foods in the refrigerator, and the user can easily and quickly select a recipe without trouble. (2) After selecting a recipe, only the necessary materials can be confirmed from the list of lacking materials and saved as a "shopping list", so the trouble and time required for creating the "shopping list" can be saved. (3) The present invention can manage not only the food management in the refrigerator but also the creation of the "shopping list" in a batch, and is effective in various aspects such as "housework efficiency improvement" and "energy saving".

[0065] This recipe proposal system can perform operations such as the following. After the user takes a picture of the expiration date of the food in the refrigerator and receives the information on the mobile terminal, a question such as "What do you want to eat today?" is displayed. A recipe is proposed based on the content answered by the user and the information on the food taken in the picture. In addition to questions such as "What do you want to eat today?", questions such as "How many people are you cooking for today?" may also be displayed. In claim 1, a recipe proposal system characterized in that the proposal of the recipe is performed by a character, and the character is an animation character. Specifically, the "animation character" is an "animation character" known as an "animation character".

[0066] When the CPU shifts to the expiration date input operation mode, the coordinate data on the touch panel corresponding to "expiration date input" is taken into the CPU from the operation control unit as a command, whereby the food item display control means operates, and food item data such as milk, eggs, fish, pork, beef, vegetables, fruits, bread, etc. is read from the food item storage unit. A food selection screen including various food items and icon marks for scrolling the food item screen is displayed on the liquid crystal display unit in a predetermined pattern. By combining these elements, the recipe proposal system of the present invention is realized. A refrigerator in-stock management device that can detect the in-stock status of products in the refrigerator, automatically create a shopping list from the in-stock status, and notify the user terminal. The refrigerator in-stock management device can detect the in-stock status of products in the refrigerator, automatically create a shopping list, and notify the user terminal. The specific configuration is as follows. The refrigerator in-stock management device detects the in-stock status of products with a sensor installed inside the refrigerator and transmits the information stored in a database to the server. The server automatically generates a shopping list from the product name, quantity, etc. and notifies the user terminal. An application that allows the user to confirm the shopping list and product information etc. is also provided from the user terminal.

[0067] This system operates on a mobile terminal owned by the user. The user takes a picture of the expiration date of the food in the refrigerator and transmits the information to the system. The system recognizes information such as the food name and expiration date from the captured image and proposes recipes based on that information. Also, when the user is questioned about the date and time, the system answers and that answer is also reflected in the recipe proposal.

[0068] And the recipe proposal system of the present invention can propose recipes from the expiration date information of the food in the refrigerator and contribute to reducing food loss. [2. When the purchaser of a food product requests information on the expiration date or shelf life][2-1. In the case of the expiration date] When the purchaser of a food product requests information on the expiration date of the purchased product, the food information confirmation means confirms the expiration date information recorded in the food information recording means 4. And the present invention can propose recipes from the expiration date information of the food in the refrigerator and contribute to reducing food loss. A household appliance that can automatically grasp, manage, and display the remaining amount, expiration date, etc. of ingredients and cooked dishes stored in the refrigerator, characterized by comprising an image processing device that can automatically determine and display the remaining amount, expiration date, etc. from the image data captured by a camera provided inside the refrigerator, and a communication device that can communicate wirelessly from inside or outside the refrigerator. The present invention is a household appliance comprising an image processing device that can automatically determine and display the remaining amount, expiration date, etc. from the image data captured by a camera provided inside the refrigerator, and a communication device that can communicate wirelessly from inside or outside the refrigerator.

[0069] Also, the recipe proposal method according to the present invention is a method of taking a picture of the expiration date of the food stored in the refrigerator and proposing a recipe from that information. A recipe proposal device comprising: means for photographing the expiration date of food stored in a refrigerator; means for recognizing the expiration date information of food in ascending order of the expiration date based on the expiration date; means for displaying a character that makes a proposal regarding the food based on the expiration date information; means for asking a question regarding the date and time based on the date and time information; and means for proposing a recipe using the food based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information. The present invention relates to a method and an apparatus for managing food ingredients in a refrigerator. The specific method is as follows. First, a photograph of the food ingredients in the refrigerator and data such as their expiration dates (hereinafter referred to as "data") are taken with a smartphone or the like. Next, the data is analyzed to determine "items with approaching expiration dates", "items with expired expiration dates", etc. Then, when "items with approaching expiration dates" etc. are determined, "dishes that can be made with this food ingredient" etc. are displayed on a smartphone or the like. These consist of "dish name", "ingredients", "cooking method", etc.

[0070] When the recognition unit determines that the recognized food is a food stored in the refrigerator, for foods that have been stored in the past, in the specified record, the information associated with the food name information of the recognized food is updated.

[0071] When this system determines that the food is a food taken out of the refrigerator, in the specified record, the information associated with the food name information of the recognized food is updated, and it can operate on a portable terminal held by the user.

[0072] Further, when the food recognition unit determines that the recognized food is a food stored in the refrigerator, for foods that have not been stored in the past, various new information is associated with the user of the specified record.

[0073] Furthermore, based on the incoming / outgoing determination result information included in the received food management request information, the food recognition unit determines whether the recognized food is a food stored in the refrigerator or a food taken out of the refrigerator, and displays the result on the display unit.

[0074] Further, the food recognition unit performs a process of obtaining an expiration date based on the date indicated by the date and time information included in the food management requirement information by a predetermined method such as calculation or inquiry, and changing it to the expiration date obtained on the expiration date indicated by the expiration date information.

[0075] When it is determined that the food management requirement information has been received, the food recognition unit performs food recognition processing based on the cut-out food image indicated by the image data included in the food management requirement information.

[0076] The recipe proposal system according to the present invention transmits a food image included in food management requirement information to a device that stores a learned model, and obtains a specific result of the food from this device. Then, the expiration date information is recognized by the recognition unit, and the expiration date information of the food in ascending order of the short expiration date is obtained. Further, based on the date and time information, a question unit that asks a question about the date and time, and based on the response information from the user or the mobile terminal corresponding to the proposal and the expiration date information, a recipe using the food is proposed by the character in the proposal unit.

[0077] A recipe proposal system comprising: a photographing unit provided in a mobile terminal held by a user for photographing the expiration date of food stored in a refrigerator; a recognition unit for recognizing the expiration date information of the food in ascending order of the short expiration date based on the expiration date; a display unit for displaying the expiration date information and a character for making a proposal regarding the food; a question unit for asking a question about the date and time based on the date and time information; and a proposal unit for proposing a recipe using the food by the character based on the response information from the user or the mobile terminal corresponding to the proposal and the expiration date information. And the food recognition unit recognizes the food indicated by the obtained specific result as the food indicated by the cut-out food image.

[0078] This system operates, for example, in the following procedure. First, the user takes a picture of the expiration date of the food in the refrigerator and transmits the image data to the mobile terminal. Next, the recognition unit acquires the food name information from the image data and acquires the expiration date information of the corresponding food from the food name information. Then, the display unit displays the expiration date information of the corresponding food together with the character, and further, the proposal unit proposes a recipe using the corresponding food. If the user likes the recipe, the user can respond, and if not, the user can request another recipe. A recipe proposal system according to claim 1, wherein the proposal of the recipe is made by a character, a voice recognition function is provided on the screen on which the character is displayed, and response information is generated based on the content input by the user by voice. It is provided with a voice recognition function, and response information is generated from the content input by the user by voice.

[0079] This system is equipped with a voice recognition function, and response information is generated from the content input by the user in voice. For example, for a question such as "Are there any simpler dishes?", recipes for simple dishes such as "omelette rice" and "oyakodon" are proposed. Also, requests such as "I want to eat some meat dishes today" can be accepted. In this case as well, the voice recognition function is utilized, and recipes for meat dishes are proposed from the keyword "meat dishes". In the recipe, essential ingredients and ingredients that are not essential but are used for preference or additional flavor changes are distinguished and displayed. In a recipe, there may be cases where multiple menus are combined, such as a set meal with a main dish, a side dish, rice, a side dish, and miso soup. In that case, a combination of recipes for multiple menus is proposed. The recipe proposal system according to the present invention can photograph foods not only in the refrigerator but also in the pantry and freezer, and inventory management including them is possible. With this function, the user can receive recipe proposals that combine foods stored in the pantry and freezer in addition to the essential ingredients in the refrigerator. For example, while the chicken stored in the refrigerator is proposed as an essential ingredient for the main dish in the recipe, canned tomatoes in the pantry and frozen vegetables stored in the freezer may be displayed as optional ingredients. Thereby, the user can utilize the ingredients in the pantry and freezer centering on the essential ingredients in the refrigerator to add or change flavors and enrich the side dishes. Furthermore, the proposed recipe has a function of visually distinguishing essential ingredients from non-essential ingredients. Specifically, essential ingredients are highlighted as indispensable for the main cooking process of the recipe, while optional ingredients are displayed in a different color or with a different mark as elements that can be added or changed according to the user's preference and the inventory status of the ingredients. For example, in the recipe for chicken curry as the main dish, chicken, onion, and spices are displayed as essential ingredients, while tomatoes and eggplants are listed as "add according to preference" as optional ingredients. By such a distinction, it becomes easy for the user to customize the recipe according to their preference, and it is also possible to prevent waste of ingredients. With this system, users can make the most of the food in the refrigerator, pantry, and freezer and receive more flexible and diverse meal suggestions. In this system, when a user receives recipe suggestions based on specific foods, not only single-dish recipes but also suggestions combining multiple menus are possible. For example, combinations such as a main dish and a side dish, rice, side dishes, and miso soup, or curry and lassi are considered. With such recipe suggestions combining multiple menus, a balanced meal can be easily planned. Furthermore, even when a user requests a combination of dishes based on a specific theme or preference, the system flexibly responds to that request and proposes relevant recipes.

[0080] And when the food management server receives food management request information, it transmits it to the food recognition unit through the server communication unit, obtains the necessary recipe data from the recipe database based on the food information and user information received from the food recognition unit, extracts the necessary ingredients and procedures from the obtained recipe data, generates and displays a recipe screen from the extracted ingredients and procedures, and generates and displays a detailed screen of the selected recipe when the user makes a selection. A refrigerator food management device that can photograph and obtain the expiration date information of the food in the refrigerator and propose the types and amounts of food that must be consumed earliest based on that information and the date and time information, comprising a recognition unit that automatically recognizes information such as the types and quantities of food in the refrigerator from the image data photographed by a camera installed in the refrigerator, a display unit that displays the proposed content such as the types and amounts of food that must be consumed earliest based on the recognized information such as the types and quantities of food in the refrigerator and the date and time information, and generates and displays a detailed screen of the selected recipe when the user makes a selection. Referring to the flowchart, the server control unit determines whether the request content received by the server communication unit was sent from the refrigerator food management device.

[0081] A recipe proposal system comprising: a photographing unit provided in a mobile terminal owned by a user for photographing the expiration date of food stored in a refrigerator; a recognition unit for recognizing the expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit for displaying the expiration date information and a character for making a proposal regarding the food; a question unit for asking a question regarding the date and time based on the date and time information; and a proposal unit for proposing a recipe using the food by an animation character based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information. This system determines the expiration date of food from the expiration date information of the food photographed by the mobile terminal owned by the user, displays the food in order starting from the food with the closest expiration date, and proposes a recipe using the food. Also, the system proposes a recipe taking into account the content answered by the user and the date and time information. Further, this system also acquires and manages data such as the purchase date and time and the purchase location from the food data photographed by the mobile terminal owned by the user. The recipe proposal system according to claim 1, wherein the proposal of the recipe is made by a character, and the character is an animation character. A recipe proposal system comprising: a photographing unit provided in a mobile terminal owned by a user for photographing the expiration date of food stored in a refrigerator; a recognition unit for recognizing the expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit for displaying the expiration date information and an animation character for making a proposal regarding the food; a question unit for asking a question regarding the date and time based on the date and time information; and a proposal unit for proposing a recipe using the food by the animation character based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information. This system is the same as that described in claim 1, but an "animation character" is added. This animation character moves around cutely while making proposals for recipes and checking the date and time information. As a result, the user can enjoy cooking.

[0082] Then, when the user selects one candidate recipe from the candidate recipes displayed on the display unit of the terminal device via the operation unit and the operation unit accepts this operation, the recipe proposal unit transmits the selected candidate recipe from the communication unit to the terminal device, thereby proposing detailed information such as the name of the dish, the cooking method of the dish, and the ingredients used in the dish to the user for the candidate recipe selected by the user.

[0083] After the proposal unit extracts the candidate recipes, the recipe proposal system executes processing to select the optimal recipe from the candidate recipes, displays the optimal recipe, and provides it to the user. It extracts candidate recipes from combinations of in-stock ingredients and creates a candidate recipe list. It creates a list of the top N candidate recipes with high evaluation scores from the candidate recipe list, and creates and displays an optimal recipe list from the list of the top N candidate recipes with high evaluation scores. It creates and displays a list of required ingredients and a list of required cooking utensils from the optimal recipe list selected by the user. It creates and displays a shopping list from the list of required ingredients and the list of required cooking utensils from the optimal recipe list selected by the user. It calculates and displays evaluation scores such as the budget and nutritional balance when the user has gathered the required ingredients and cooking utensils from the shopping list, which can be useful for the user to make decisions on shopping and cooking plans. The recipe proposal system according to claim 1, wherein the photographing unit is capable of photographing not only the food in the refrigerator but also the food in the pantry, the freezer, etc. The recipe proposal system according to the present invention is capable of photographing not only inside the refrigerator but also inside the pantry, the freezer, etc.

[0084] Based on the information on surplus ingredients held for the user in ingredient management (i.e., the ingredients currently held) and the information on the ingredients used in each record held by the recipe, the recipe proposal unit identifies and proposes recipe information from the recipes in ascending order of the amount of ingredients that the user needs to newly purchase.

[0085] The recipe proposal system proposes a recommended cooking recipe for the user based on the weight ratio of the in-stock ingredients acquired by the weight ratio acquisition unit and the recipe information stored in the recipe database.

[0086] When identifying the above-mentioned recipe information, the recipe proposal unit preferentially selects, in the user, among the additional ingredients, the ingredients that are currently held as surplus ingredients even after cooking and whose expiration date is approaching, in food ingredient management, and identifies from the recipes the recipes that use the ingredients. Further, the present invention is characterized in that the above-described components interact with each other to form a system.

[0087] Further, based on the information of the surplus ingredients held in the above-mentioned food ingredient management, the recipe proposal unit identifies from among the respective recipe information in the recipes the recipes that can be cooked with the surplus ingredients, and outputs the recipe information to the user terminal.

[0088] The recipe proposal system distributes and displays the recipe information identified in the above-described process to the user terminal by the recipe proposal unit, and ends the process.

[0089] By photographing the expiration date of the food stored in the refrigerator by the user and having it recognized by the mobile terminal, the expiration date information of the stored food can be obtained, and the recipes proposed by the character can be displayed. Further, the user can input information such as the planned date and time of use and the purpose of use of the food in the refrigerator, and have recipes proposed based on those information. In claim 1, the recipe proposal is performed by a character, and the character is an animation character, a recipe proposal system characterized by this. Since this recipe proposal system displays an animation character, the user can obtain cooking ideas while enjoying. Further, since the animation character also teaches cooking procedures and tips, even beginners can easily cook. The recipe proposal system according to the present invention includes a photographing unit provided in a mobile terminal owned by a user, a recognition unit that recognizes expiration date information of foods in ascending order of the expiration date based on the expiration date, a display unit that displays the expiration date information and a character that is an animation character, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food by an animation character based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information.

[0090] In this recipe proposal system, by registering (pairing) the combination of the device adapter provided in the refrigerator and the terminal device with the server, the refrigerator, the server, and the terminal device can communicate with each other.

[0091] The recipe proposal system can generate recipe candidates from the selected stocked ingredients, randomly select one recipe from the recipe candidates generated from the stocked ingredients selected by the user, and display it on the display unit. Also, it can randomly select one recipe from the recipe candidates generated from the stocked ingredients selected by the user and display it on the display unit.

[0092] A recipe proposal system including a photographing unit provided in a mobile terminal owned by a user that photographs the expiration date of foods stored in the refrigerator, a recognition unit that recognizes expiration date information of foods in ascending order of the expiration date based on the expiration date, a display unit that displays the expiration date information and a character that makes a proposal regarding the food, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food by the character based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information. The display unit can display a plurality of candidate recipes and display only the recipe names.

[0093] This system is assumed to be incorporated into home appliances such as refrigerators and freezers used, for example, within a household. Also, this system can be operated on a mobile terminal such as a smartphone or a tablet terminal. In claim 1, the recipe proposal is made by a character, and the recipe proposed by the character is proposed in consideration of information such as cooking utensils and seasonings owned by the user. A recipe proposal system characterized by this. The recipe proposal system of the present invention can propose a recipe in consideration of information such as cooking utensils and seasonings owned by the user. For example, in response to a user's request of "I want to eat a hot pot dish today", "A hot pot dish that can be made with the hot pot you have" can be proposed. Also, in response to a user's request of "I want to eat a dish centered on vegetables today", "A dish that can be made with the vegetables you have" can be proposed. Thus, since the recipe proposal system of the present invention can propose a recipe in consideration of information such as cooking utensils and seasonings owned by the user, it can present a recipe that is easy for the user to actually make and finish deliciously. In claim 1, the recipe proposal is made by a character, and the character is an animation character. A recipe proposal system characterized by this. In the recipe proposal system of the present invention, an animation character is displayed. The animation character has a cute design and is felt to be easy to approach. Also, the animation character moves.

[0094] Also, the recipe proposal method according to the present invention photographs the expiration date of food stored in the refrigerator, recognizes the expiration date information of the food in ascending order of the expiration date from the image photographed by the mobile terminal owned by the user, displays the expiration date information and a character that makes a proposal regarding the food, asks a question regarding the date and time based on the date and time information, and proposes a recipe based on the answer information and the expiration date information.

[0095] A recipe proposal system comprising: a photographing unit provided in a user-owned mobile terminal for photographing the expiration date of food stored in a refrigerator; a recognition unit for recognizing expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit for displaying the expiration date information and a character for making a proposal regarding the food; a question unit for asking a question regarding the date and time based on the date and time information; and a proposal unit for displaying only a cooking recipe capable of creating a desired amount of a dish for the user using the stocked ingredients on the display unit based on the response information by the user or the mobile terminal corresponding to the proposal and the expiration date information, and for proposing a recipe using the food by the character.

[0096] At this time, the current date, day of the week, and time measured by the timekeeping unit and the outside air temperature detected by the temperature sensor are displayed on the screen of the liquid crystal display unit.

[0097] The image displayed on the display unit of this recipe proposal system is the same as the image (Figure 1) displayed on the display unit of the mobile terminal.

[0098] The display unit has a recipe button displayed for each food, and it is determined whether or not the recipe button has been pressed. Also, the recipe proposal method according to the present invention is implemented using the above-described recipe proposal system.

[0099] Also, the recipe proposal method according to the present invention is implemented using the above-described recipe proposal system.

[0100] A recipe proposal system comprising: a photographing unit provided in a user's portable terminal for photographing the expiration date of food stored in a refrigerator; a recognition unit for recognizing the expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit for displaying the expiration date information and an animation character for making a proposal regarding the food; a questioning unit for asking a question regarding the date and time based on the date and time information; and a proposal unit for proposing a recipe using the food by the animation character based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information. This recipe proposal system operates, for example, in the following procedure. First, the user photographs the expiration date of the food in the refrigerator and causes it to be recognized on the portable terminal. Next, on the screen where the animation character is displayed, a question such as "What would be good for dinner today?" is displayed. The user selects and answers from among them. After that, the recipe proposed by the animation character is displayed. In this way, the user can easily select a recipe from the ingredients in the refrigerator.

[0101] The recipe proposal system according to claim 1, wherein the character is an animation character. By displaying the animation character, the user can enjoyably select a recipe. A recipe proposal system comprising: a photographing unit provided in a user's portable terminal for photographing the expiration date of food stored in a refrigerator; a recognition unit for recognizing the expiration date information of food in ascending order of the expiration date based on the expiration date; a display unit for displaying the expiration date information and an animation character for making a proposal regarding the food; a questioning unit for asking a question regarding the date and time based on the date and time information; and a proposal unit for proposing a recipe using the food by the animation character based on the response information by the user or the portable terminal corresponding to the proposal and the expiration date information.

[0102] This system may be an input device with a physical keyboard or the like as the input unit, or may be configured as a touch panel integrated with the display unit.

[0103] The recipe proposal system according to the present invention may be implemented in the following form, for example. (1) Photograph the expiration date of the food in the refrigerator and have it recognized by the mobile terminal owned by the user. (2) Create and display a list of foods in ascending order of expiration date from the recognized expiration date information. (3) When a user's question "I want to cook something" comes in for the food selected from the displayed list, when an answer "I want to cook a specific dish" comes from the user, when an answer "Please tell me if there are any ingredients or seasonings" comes from the user, etc., propose an appropriate recipe. (4) When an answer "I like this recipe" comes from the user for the proposed recipe, when an answer "I don't like this recipe" comes from the user, etc., change the content of future proposals accordingly.

[0104] As shown in FIG. 1, the recipe proposal system according to the present invention includes a photographing unit provided in a mobile terminal owned by a user that photographs the expiration date of food stored in a refrigerator, a recognition unit that recognizes the expiration date information of foods in ascending order of expiration date based on the expiration date, a display unit that displays the expiration date information and a character that makes a proposal regarding the food, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food by the character based on the answer information by the user or the mobile terminal corresponding to the proposal and the expiration date information. Further, the above-mentioned expiration date warning date and time setting display control means displays each warning date and time of "one week ago", "three days ago", "two days ago", and "the day before yesterday" generated by the expiration date warning date and time item generation means on the liquid crystal display unit, and sets the warning date and time of the expiration date of the food by touching with the input means corresponding to one of the warning dates and times to be saved for the food.

[0105] In this case, at the display position of each food item, as shown by enlarging the display position of pork, a pig illustration mark, characters, and the warning date and time of the expiration date are displayed, and the warning date and time are displayed by color-coding or blinking at the position of the warning date and time according to the setting.

[0106] Also in this case, the information such as the date and time obtained from the internal clock of the server device may be used, or the information such as the date and time transmitted by the user terminal together with the tag may be used.

[0107] The recipe proposal method according to the present invention photographs the expiration date of food stored in a refrigerator, recognizes the expiration date information of foods in ascending order of the expiration date from an image photographed by a mobile terminal owned by a user, displays the expiration date information and a character that proposes a proposal regarding the food, asks a question regarding the date and time based on the date and time information, and proposes a recipe based on the answer information and the expiration date information.

[0108] A recipe proposal system including: a photographing unit provided in a mobile terminal owned by a user that photographs the expiration date of food stored in a refrigerator; a recognition unit that recognizes the expiration date information of foods in ascending order of the expiration date based on the expiration date; a display unit that displays the expiration date information and a character that proposes a proposal regarding the food; a question unit that asks a question regarding the date and time based on the date and time information; and a proposal unit that proposes a recipe using the food based on the answer information from the user or the mobile terminal corresponding to the proposal and the expiration date information, and the character proposes. The above expiration date warning date and time item generation means generates in a form in which a predetermined period before the food expires is divided into a plurality of stages as the warning date and time, for example, generated as "one week before", "three days before", "two days before", "the day before".

[0109] The recipe proposal system according to the present invention may be implemented in the following form, for example. (1) Photograph the expiration date of food in the refrigerator and have it recognized by a mobile terminal owned by the user. (2) Create and display a list of foods in ascending order of the expiration date from the recognized expiration date information. (3) When a question from the user such as "want to cook something" comes for the food selected from the displayed list, continue to propose an appropriate recipe while asking additional questions such as "Do you have any conditions?". (4) If it is determined that there are insufficient ingredients for the recipe selected by the user, a "shopping list" or the like is created to support the user in making shopping easier. (5) It is also possible to obtain user profile information such as "Is today a special day?" and propose recipes, shopping lists, etc. while considering the date, time zone, etc.

[0110] The recipe proposal method according to the present invention is implemented using the above-described recipe proposal system.

[0111] A portable terminal is provided that photographs the expiration dates of foods stored in a refrigerator. This portable terminal includes a recognition unit that recognizes expiration date information, a display unit that displays a character that makes a proposal regarding the food, a question unit that asks questions based on date and time information, and a proposal unit that proposes a recipe based on the response information and the expiration date information. The recipe display system includes a food recognition unit that obtains the types of foods stored in the refrigerator from a photographed image of the refrigerator taken from the upper front of the refrigerator, and a terminal device that displays recipe information corresponding to the foods recognized by the food recognition unit on a touch panel (display unit).

[0112] When the user selects one candidate recipe from the candidate recipes displayed on the display unit of the terminal device via the operation unit and the operation unit accepts this operation, the recipe proposal unit transmits the selected candidate recipe from the communication unit to the terminal device, thereby proposing detailed information such as the name of the dish, the cooking method of the dish, and the ingredients used in the dish to the user for the candidate recipe selected by the user.

[0113] As shown in FIG. 1, the recipe proposal system according to the present invention includes a mobile terminal owned by a user, a photographing unit that photographs the expiration date of food stored in a refrigerator, a recognition unit that recognizes the expiration date information of food in ascending order of the expiration date based on the expiration date, a display unit that displays the expiration date information and a character that proposes a proposal regarding the food, a question unit that asks a question regarding the date and time based on the date and time information, and a proposal unit that proposes a recipe using the food based on the answer information by the user or the mobile terminal corresponding to the proposal and the expiration date information, and is proposed by the character. Further, the recipe proposal method according to the present invention is implemented using the above-described recipe proposal system.

[0114] After the proposal unit extracts candidate recipes, the recipe proposal system executes the processes described later to select the optimal recipe from the candidate recipes, and displays the optimal recipe to provide it to the user. Candidate recipes are extracted from combinations of stocked ingredients to create a candidate recipe list. The top N candidate recipe lists with high evaluation scores are created from the candidate recipe list, and the optimal recipe list is created and displayed from the top N candidate recipe lists with high evaluation scores.

[0115] A list of required ingredients and a list of required cooking utensils are created and displayed from the optimal recipe list selected by the user. A shopping list is created and displayed from the list of required ingredients and the list of required cooking utensils in the optimal recipe list selected by the user. Information such as the shopping budget and nutritional balance is calculated and displayed from information such as the budget and nutritional balance when the user has prepared the required ingredients and cooking utensils from the shopping list. Further, the recipe proposal method according to the present invention is a method of photographing the expiration date of food stored in a refrigerator and proposing a recipe from the information.

[0116] The recipe proposal system according to claim 1, comprising: a narrowing-down unit that narrows down the recipes proposed by the character based on search conditions including the types of ingredients, cooking utensils, and cooking time selected by the user on the screen where the character is displayed; and a display unit that displays one or more recipes randomly selected from the narrowed-down recipes. For example, searching based on conditions such as "chicken", "frying pan", and "within 30 minutes", recipes such as "fried chicken" and "stir-fried chicken with vegetables" are displayed.

[0117] The recipe proposal system according to the present invention photographs the expiration date of the food stored in the refrigerator, proposes a recipe from the information, and includes a narrowing-down unit that performs a narrowing-down process based on search conditions including the types of ingredients, cooking utensils, and cooking time selected by the user on the screen displayed at that time, and a display unit that displays one or more recipes randomly selected from the narrowed-down recipes. Further, the recipe proposal method according to the present invention is a method of specifying conditions for searching from ingredients, cooking utensils, etc. in the refrigerator and randomly selecting and displaying one or more from the results.

Claims

[Claim 1] a photographing unit provided in a mobile terminal held by a user, the photographing unit photographing the expiration date of food stored in the refrigerator; A recognition unit that recognizes expiration date information of foods in ascending order of expiration dates based on the expiration dates; a display unit that displays the expiration date information and a character making a suggestion regarding the food; a questioning unit that asks a question about a date and time based on the date and time information; a suggestion unit that allows the character to suggest a recipe using the food based on answer information by the user or the mobile device corresponding to the suggestion and the expiration date information, The display unit displays an ingredient list and a cooking utensil list from a recipe selected by a user, A recipe suggestion system characterized in that the suggestion unit calculates budget and nutritional balance information for the ingredients and cooking utensils and outputs it to the display unit.

Images