Food management device, food management system, food management method, and program
The food management system uses image analysis and machine learning to track expiration dates of foods in refrigerators, addressing the challenge of managing untagged foods and reducing waste.
Patent Information
- Application Number
- JP2023072391
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-26
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing food management systems struggle to accurately determine the expiration dates of foods without RFID tags, barcodes, or non-contact IC tags, leading to increased food waste and loss.
A food management system that uses a camera and machine learning to identify and track the expiration dates of foods within a refrigerator by analyzing images of the stored items, generating food management information including expiration date identification and warning lists.
Effectively tracks and manages expiration dates of foods without tags, reducing food waste by providing clear expiration date indicators and enabling timely consumption or disposal.
Smart Images

Figure 2024157805000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a food management device, a food management system, a food management method, and a program. [Background technology]
[0002] Food stored in refrigerators may have passed its expiration date or best-before date. Most expired food is discarded. Therefore, an increase in expired food leads to food waste. If food retailers such as supermarkets and vegetable producers could collect information on the food stored in refrigerators in each household, it is expected that food waste will be reduced.
[0003] Patent Document 1 discloses a food purchase management support system that supports food management to purchase. The system of Patent Document 2 includes a refrigerator, a server, a mobile storage device, and a cart. The server has a database that manages recipe information for each food item, and a database that manages food information related to foods sold at the store. The mobile storage device stores information about foods to be purchased. The cart is used to transport foods at the store. The cart has an IC (Integrated Circuit) chip reader / writer that reads information about foods stored in the mobile storage device. The refrigerator reads RFID (Radio Frequency Identification) tags attached to the foods using an RFID tag reader / writer. The refrigerator determines a recipe based on the foods stored in the refrigerator. The refrigerator writes data about foods to be purchased at the store to the mobile storage device using the IC chip reader / writer. The refrigerator obtains recipe information from the server based on the information about the foods. The refrigerator determines foods to be purchased at the store based on the recipe information.
[0004] Patent Document 2 discloses a refrigerator that manages the expiration dates of food stored therein. The refrigerator in Patent Document 2 reads the first type information of food stored therein using a barcode attached to the food. The refrigerator calculates the period until the expiration date of the food based on the current date and time and the first type information. For food whose expiration date is shorter than a predetermined period, the refrigerator displays the name of the food determined from the first type information.
[0005] Patent Document 3 discloses an automatic refrigeration system that aims to refrigerate food stored in a refrigerator under optimal conditions. In the method of Patent Document 3, the refrigerator reads information about the contents and storage conditions of the food recorded in a non-contact IC tag attached to the food. The refrigerator issues a warning when the expiration date or best-before date is approaching.
[0006] Patent Document 4 discloses a refrigerator designed to visually check the interior of the refrigerator. An imaging device that captures an image of a storage compartment is disposed inside the refrigerator in Patent Document 4. A control device of the refrigerator in Patent Document 4 extracts food images of food stored inside the storage compartment from an image of the interior of the storage compartment. The control device generates information related to deterioration of food corresponding to the food image from a change over time in the food image. The control device transmits the food image and the information related to deterioration to a mobile terminal via a communication device. [Prior art documents] [Patent documents]
[0007] [Patent Document 1] JP 2008-242911 A [Patent Document 2] JP 2002-147931 A [Patent Document 3] JP 2001-317862 A [Patent Document 4] JP 2017-242911 A Summary of the Invention [Problem to be solved by the invention]
[0008] The method of Patent Document 1 manages food stored in a refrigerator by reading RFID tags attached to the food. The method of Patent Document 2 calculates the time until the expiration date using barcodes attached to the food. The method of Patent Document 3 manages the expiration date of food stored in a refrigerator using non-contact IC tags attached to the food packages or containers. The methods of Patent Documents 1 to 3 can manage the expiration date or best-before date for food that has an RFID tag, barcode, or non-contact IC tag attached. However, the methods of Patent Documents 1 to 3 cannot manage the expiration date or best-before date for food that does not have an RFID tag, barcode, or non-contact IC tag attached.
[0009] The method of Patent Document 4 generates information on the deterioration of food stored in a storage room based on food images captured by an imaging device that captures the storage room. The method of Patent Document 4 makes it possible to check the inventory status and deterioration status of food stored in a refrigerator without using RFID, barcodes, or non-contact IC tags. The method of Patent Document 1 makes it possible to manage food stored in refrigerators installed in each home, but does not provide food management that leads to a reduction in food waste.
[0010] An object of the present disclosure is to provide a food management device, a food management system, a food management method, and a program that can generate information indicating the expiration dates of food stored in a managed refrigerator. [Means for solving the problem]
[0011] A food management device of one embodiment of the present disclosure includes a food information collection unit that collects food information including an expiration date for each food item stored in the managed refrigerator, a food management information generation unit that uses the collected food information to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food item stored in the managed refrigerator, and a food management information output unit that outputs the generated food management information.
[0012] In a food management method of one embodiment of the present disclosure, food information including expiration dates for each food stored in a managed refrigerator is collected, the collected food information is used to generate food management information corresponding to the expiration dates for each food stored in the managed refrigerator, and the generated food management information is output.
[0013] A program of one embodiment of the present disclosure causes a computer to execute a process of collecting food information including the expiration date of each food item stored in a managed refrigerator, a process of using the collected food information to generate food management information corresponding to the expiration date of each food item stored in the managed refrigerator, and a process of outputting the generated food management information. Effect of the Invention
[0014] According to the present disclosure, it is possible to provide a food management device, a food management system, a food management method, and a program that can generate information indicating the expiration dates of food stored in a managed refrigerator. [Brief description of the drawings]
[0015] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a food management system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of the appearance of a managed refrigerator that is a managed object of a food management system according to a first embodiment. FIG. [Diagram 3] FIG. 2 is a conceptual diagram showing an example of a state in which the door of a managed refrigerator that is a managed object of the food management system according to the first embodiment is open. [Figure 4] 1 is a block diagram showing an example of the configuration of a managed refrigerator that is a managed object of a food management system according to a first embodiment [Diagram 5] FIG. 2 is a block diagram showing an example of the configuration of a food information generating device included in the food management system according to the first embodiment [Figure 6]FIG. 2 is a conceptual diagram for explaining a learning example of a food estimation model used by a food information generation device included in the food management system according to the first embodiment. [Figure 7] FIG. 1 is a conceptual diagram showing an example in which a food information generating device included in the food management system in the first embodiment uses a food estimation model to estimate foods stored in a managed refrigerator. [Figure 8] FIG. 2 is a conceptual diagram for explaining the specification of expiration dates by a food information generation unit of a food information generation device provided in the food management system according to the first embodiment. [Figure 9] 1 is an example of a table summarizing food information on a plurality of foods stored in a managed refrigerator that is a managed item of the food management system according to the first embodiment. [Figure 10] 1 is a block diagram showing an example of the configuration of a food management device included in a food management system according to a first embodiment. FIG. [Figure 11] 1 is an example of a table summarizing food management information regarding a plurality of foods stored in a managed refrigerator that is a managed item of the food management system according to the first embodiment. [Figure 12] FIG. 2 is a conceptual diagram showing an example of display information including a food list generated as food management information by the food management system according to the first embodiment. [Figure 13] FIG. 2 is a conceptual diagram showing an example of display information including an expiration date list generated as food management information by the food management system according to the first embodiment. [Figure 14] 3 is a conceptual diagram showing an example of display information including a warning list generated as food management information by the food management system according to the first embodiment. FIG. [Figure 15] 1 is a conceptual diagram showing an example of display of display information including expiration date identification information generated as food management information by the food management system according to the first embodiment. FIG. [Figure 16] 1 is a conceptual diagram showing an example of display of display information including expiration date identification information generated as food management information by the food management system according to the first embodiment. FIG. [Figure 17]1 is a conceptual diagram showing an example of display information according to a food list generated as food management information by the food management system according to the first embodiment. FIG. [Figure 18] 4 is a flowchart for explaining the operation of a managed refrigerator that is a managed object of the food management system according to the first embodiment. [Figure 19] 4 is a flowchart illustrating the operation of a food information generating device included in the food management system according to the first embodiment. [Figure 20] 4 is a flowchart for explaining the operation of the food management device included in the food management system according to the first embodiment. [Figure 21] FIG. 11 is a block diagram showing an example of the configuration of a food management system according to a second embodiment. [Figure 22] FIG. 11 is a block diagram showing an example of the configuration of a food management device included in a food management system according to a second embodiment. [Figure 23] FIG. 11 is a conceptual diagram showing how a food management device included in a food management system according to a second embodiment collects food information about foods stored in managed refrigerators installed in multiple homes. [Figure 24] 13 is an example of a table summarizing food information on multiple foods stored in managed refrigerators installed in multiple homes that are managed by a food management device provided in a food management system in accordance with the second embodiment. [Diagram 25] This is an example of a table summarizing food management information regarding multiple foods stored in managed refrigerators installed in multiple homes that are managed by a food management device provided in a food management system in accordance with the second embodiment. [Figure 26] A conceptual diagram showing an example of food management information generated based on food information regarding multiple foods stored in managed refrigerators installed in multiple homes that are managed by a food management device provided in a food management system in accordance with the second embodiment. [Figure 27]FIG. 11 is a conceptual diagram for explaining an example of learning of an order quantity estimation model used to estimate food order quantities by a food management device provided in a food management system according to a second embodiment. [Figure 28] FIG. 11 is a conceptual diagram showing an example in which a food management device included in a food management system according to a second embodiment estimates an order quantity of food using an order quantity estimation model. [Figure 29] FIG. 11 is a conceptual diagram for explaining an example of an order based on food management information generated by a food management device included in a food management system according to a second embodiment. [Diagram 30] A conceptual diagram showing how ordering information based on food management information generated by a food management device provided in a food management system in accordance with the second embodiment is transmitted from a terminal device installed in a store to a terminal device installed at the headquarters. [Diagram 31] FIG. 13 is a conceptual diagram showing how order information is transmitted from a plurality of stores to a headquarters. [Diagram 32] FIG. 13 is a conceptual diagram showing how order information is transmitted from a terminal device installed at a store or headquarters to a terminal device installed at a contract manufacturer. [Diagram 33] FIG. 13 is a conceptual diagram showing how order information is sent from a plurality of stores to a contract manufacturer. [Diagram 34] 13 is a flowchart for explaining the operation of a food management device included in a food management system according to a second embodiment. [Diagram 35] FIG. 11 is a block diagram showing an example of the configuration of a food management device according to a third embodiment. [Diagram 36] 13 is a flowchart illustrating the operation of the food management apparatus according to the third embodiment. [Figure 37] FIG. 2 is a block diagram showing an example of a hardware configuration for executing processes according to each embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0016] Hereinafter, the embodiments for carrying out the present invention will be described with reference to the drawings. However, the embodiments described below are limited in a way that is technically preferable for carrying out the present invention, but the scope of the invention is not limited to the following. In addition, in all the drawings used to explain the following embodiments, the same reference numerals are used for similar parts unless otherwise specified. In addition, in the following embodiments, repeated explanations of similar configurations and operations may be omitted. In addition, the direction of the arrows in the drawings is an example, and does not limit the direction of signals between blocks.
[0017] (First embodiment) First, a food management system according to a first embodiment will be described with reference to the drawings. The food management system of this embodiment generates food management information for managing food stored in a managed refrigerator. The food management information includes expiration date information indicating expiration dates such as best-before dates and use-by dates of the food stored in the managed refrigerator. An example of managing food stored in a refrigerator will be described below. The method of this embodiment can be applied to managing not only food stored inside a refrigerator, but also food placed outside the refrigerator. The method of this embodiment can also be applied to managing preserved food prepared in preparation for disasters.
[0018] (composition) Fig. 1 is a block diagram showing an example of the configuration of a food management system 10 according to this embodiment. The food management system 10 of this embodiment includes a food information generating device 11 and a food management device 15. In this embodiment, an example of managing food stored in a single managed refrigerator 100 will be described. Below, the managed refrigerator 100 will be described first, followed by the food information generating device 11 and the food management device 15.
[0019] [Managed refrigerators] FIG. 2 is a conceptual diagram showing an example of the external appearance of managed refrigerator 100. In the example of FIG. 2, managed refrigerator 100 has three doors. Managed refrigerator 100 has at least one refrigerator compartment. Managed refrigerator 100 also has at least one freezer compartment. In this embodiment, management of food stored in one of the refrigerator compartments of managed refrigerator 100 will be described. The method of this embodiment can be applied to both the refrigerator compartment and the freezer compartment of managed refrigerator 100.
[0020] Fig. 3 is a conceptual diagram showing an example of a state where one of the doors of the managed refrigerator 100 is open. Inside the managed refrigerator 100, interior lights 101 and cameras 102 are installed. There is no particular limit to the number of interior lights 101 and cameras 102. To accurately manage a plurality of foods stored inside the managed refrigerator 100, it is preferable that a plurality of cameras 102 are installed.
[0021] 4 is a block diagram showing an example of the configuration of managed refrigerator 100. Managed refrigerator 100 has interior light 101, camera 102, photography control unit 103, memory unit 104, image output unit 105, cooling mechanism 107, cooling control unit 108 and thermometer 109.
[0022] Interior light 101 is a lighting fixture that illuminates the inside of managed refrigerator 100. In the example of FIG. 3, interior light 101 is arranged on the ceiling of the refrigerator compartment. Interior light 101 may also be arranged on a wall, shelf, door, etc. of the refrigerator compartment. The light emitter of interior light 101 is not particularly limited. For example, the light emitter of interior light 101 is realized by an LED (Light Emitting Diode) lamp. For example, the light emitter of interior light 101 is realized by an incandescent light bulb, a fluorescent lamp, an inorganic EL (Electroluminescence) lamp, an organic EL lamp, etc.
[0023] The wavelength band of light emitted by the light emitter of interior light 101 is set to a wavelength band that can be captured by camera 102. If camera 102 is sensitive to the visible wavelength band, a light emitter that emits light in the visible wavelength band is used. If camera 102 is sensitive to the infrared wavelength band, a light emitter that emits light in the infrared wavelength band is used. If camera 102 is sensitive to the ultraviolet wavelength band, a light emitter that emits light in the ultraviolet wavelength band is used.
[0024] Interior light 101 is turned on when the door of managed refrigerator 100 is opened. Interior light 101 is turned off when the door of managed refrigerator 100 is closed. For example, interior light 101 is turned on when a switch is pressed to open the door. For example, interior light 101 may be turned on / off according to control by photography control unit 103.
[0025] Furthermore, interior light 101 turns on at the timing when the interior of managed refrigerator 100 is photographed (photographing timing). Interior light 101 turns on in accordance with the control of photography control unit 103. For example, interior light 101 may be set to turn on automatically at a preset photography timing. For example, interior light 101 may be set to turn on in response to an operation by a user using managed refrigerator 100. Furthermore, interior light 101 turns off in accordance with the control of photography control unit 103 when photography by camera 102 is completed. As long as interior light 101 is on when photography is taken by camera 102, there is no particular limitation on the timing at which interior light 101 turns off (light-off timing).
[0026] Camera 102 is an imaging device which captures images of the inside of managed refrigerator 100. At least one camera 102 is installed inside managed refrigerator 100. Camera 102 has a still image capture function. Camera 102 may have a video capture function. In the example of Figure 3, multiple cameras 102 are arranged inside managed refrigerator 100. Camera 102 is realized by a digital camera with communication function. Camera 102 is arranged with its lens facing the inside of managed refrigerator 100. It is preferable that the surface of the lens is treated with an anti-fogging coating so that it is less likely to fog up when managed refrigerator 100 is opened and closed.
[0027] Camera 102 photographs the inside of managed refrigerator 100 at the photographing timing. Camera 102 photographs the inside of managed refrigerator 100 in accordance with the control of photographing control section 103. Camera 102 stores the photographed image data in memory section 104. For example, camera 102 may be set to automatically photograph the inside of managed refrigerator 100 at a preset photographing timing. For example, camera 102 may be set to photograph in response to the interior light 101 being turned on. Camera 102 may also be set to photograph in response to the operation of a user using managed refrigerator 100.
[0028] The photography control unit 103 controls the interior light 101 and the camera 102. For example, the photography control unit 103 is realized by a microcomputer or a microcontroller. For example, the photography control unit 103 has a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, etc.
[0029] The photographing control unit 103 turns on the interior light 101 at the photographing timing. Usually, the arrangement of food stored inside the managed refrigerator 100 changes during the period from when the door of the managed refrigerator 100 is opened to when the door is closed. Therefore, the photographing timing is set to when the door of the managed refrigerator 100 is closed. The photographing timing is not limited to when the door of the managed refrigerator 100 is closed. For example, the photographing timing may be a time set in advance. For example, the photographing timing may be when the door of the managed refrigerator 100 is opened. Also, there are cases where a user wants to check the food stored inside the managed refrigerator 100 in real time. Therefore, the photographing timing may be when a photographing operation by a user using the managed refrigerator 100 is accepted. For example, if a photographing switch is installed on the outside of the managed refrigerator 100, a photographing operation by a user can be accepted without opening the door of the managed refrigerator 100.
[0030] When the interior light 101 is turned on, the photography control unit 103 causes the camera 102 to photograph the inside of the managed refrigerator 100. For example, the photography control unit 103 causes all of the cameras 102 arranged inside the managed refrigerator 100 to photograph. The photography control unit 103 may cause only some of the cameras 102 arranged inside the managed refrigerator 100 to photograph. The photography control unit 103 stores image data including an image photographed by the camera 102 in the memory unit 104. The image data is linked to the time the image was photographed and an identifier indicating the camera 102 that photographed the image. The photography control unit 103 may output the image data photographed by the camera 102 to the image output unit 105.
[0031] The storage unit 104 stores image data including images captured by the camera 102. The image data stored in the storage unit 104 is output to the food information generation device 11 via the image output unit 105.
[0032] The image output unit 105 outputs the image data stored in the storage unit 104 to the food information generation device 11. The timing at which the image data is output is set arbitrarily. For example, the image output unit 105 outputs the image data in response to a request from the food information generation device 11. For example, the image output unit 105 outputs the image data to the food information generation device 11 at each shooting timing. For example, the image output unit 105 outputs the image data to the food information generation device 11 at each predetermined number of shootings. For example, the image output unit 105 outputs the image data to the food information generation device 11 at a predetermined time. In addition, there are cases where a user wants to check the food stored inside the managed refrigerator 100 in real time. Therefore, the shooting timing may be the timing at which a shooting operation by a user using the managed refrigerator 100 is accepted. For example, if a shooting switch is installed on the outside of the managed refrigerator 100, the image data can be output to the food information generation device 11 in accordance with the timing at which a shooting operation by a user is accepted.
[0033] For example, the image output unit 105 transmits image data to the food information generation device 11 via wireless communication. For example, the image output unit 105 transmits image data to the food information generation device 11 via a wireless communication function (not shown) that complies with standards such as Bluetooth (registered trademark) or WiFi (registered trademark). The communication function of the image output unit 105 may be compliant with standards other than Bluetooth (registered trademark) or WiFi (registered trademark). For example, the image output unit 105 may transmit image data to the food information generation device 11 via the Internet. The image output unit 105 may transmit image data to the food information generation device 11 via a wired connection such as a cable.
[0034] Cooling mechanism 107 is a mechanism that cools the inside of managed refrigerator 100. Cooling mechanism 107 cools the inside of managed refrigerator 100 by using a cooling principle or cooling method used in general refrigerators. There is no particular limitation on the cooling principle for cooling the inside of managed refrigerator 100. For example, the cooling principle may be an evaporation compression type, an evaporation absorption type, a Peltier effect type, or the like. Furthermore, there is no particular limitation on the cooling method for cooling the inside of managed refrigerator 100. For example, the cooling method may be a direct cooling type, a fan type, a Peltier type, or the like.
[0035] Cooling control unit 108 controls cooling mechanism 107. For example, photography control unit 103 is realized by a microcomputer or a microcontroller. For example, photography control unit 103 has a CPU, RAM, ROM, flash memory, etc. Cooling control unit 108 controls cooling mechanism 107 according to the internal temperature of managed refrigerator 100 measured by thermometer 109. If the internal temperature of managed refrigerator 100 is higher than a set temperature, cooling control unit 108 increases the cooling by cooling mechanism 107 to reduce the internal temperature of managed refrigerator 100. If the internal temperature of managed refrigerator 100 is equal to or lower than the set temperature, cooling control unit 108 decreases the cooling by cooling mechanism 107. If thermometers 109 are installed at multiple locations, cooling control unit 108 controls cooling mechanism 107 according to the respective installation locations.
[0036] Thermometer 109 is disposed inside managed refrigerator 100. Thermometer 109 is a digital thermometer that measures the temperature inside managed refrigerator 100. For example, thermometer 109 includes a thermistor, a resistance thermometer, or a thermocouple. Thermometer 109 may be a non-contact thermometer such as a thermograph or a radiation thermometer. Thermometer 109 outputs the measured temperature to cooling control unit 108.
[0037] [Food information generation device] 5 is a block diagram showing an example of the configuration of the food information generation device 11. The food information generation device 11 has an image data acquisition unit 111, a storage unit 113, a food information generation unit 115, and a food information output unit 117.
[0038] Image data acquisition unit 111 acquires image data of the interior of managed refrigerator 100 from managed refrigerator 100. For example, image data acquisition unit 111 may be configured to acquire image data of the interior of managed refrigerator 100 taken using a camera function of a terminal device such as a smartphone or tablet. Image data acquisition unit 111 stores the acquired image data in memory unit 113. Image data acquisition unit 111 may output the acquired image data to food information generation unit 115.
[0039] The memory unit 113 stores image data of the interior of the managed refrigerator 100. The image data stored in the memory unit 113 is used by the food information generation unit 115 to generate food information.
[0040] Food information generation unit 115 acquires image data of the interior of managed refrigerator 100 stored in memory unit 113. Food information generation unit 115 may be configured to acquire image data from image data acquisition unit 111. Food information generation unit 115 generates food information using the acquired image data. The food information includes the type (food name) of each food stored in managed refrigerator 100 and the expiration date of each food. The expiration date of a food is the best-before date or use-by date of the food.
[0041] The food information generation unit 115 identifies the food stored in the managed refrigerator 100 from the image data of the interior of the managed refrigerator 100. For example, the food information generation unit 115 identifies the food stored in the managed refrigerator 100 from the image data of the interior of the managed refrigerator 100 by using a machine learning technique.
[0042] FIG. 6 is a conceptual diagram for explaining an example of learning of food estimation model 160 used to estimate food stored in managed refrigerator 100. Image data (explanatory variables) of the interior of managed refrigerator 100 and food names (response variables) of food stored in managed refrigerator 100 are used for learning of food estimation model 160. The image data (explanatory variables) of the interior of managed refrigerator 100 is image data captured in the past. The food names (response variables) are food names of food stored in managed refrigerator 100 at the time the image data was captured.
[0043] For example, the food estimation model 160 is constructed by learning using a linear regression algorithm. For example, the food estimation model 160 is constructed by learning using a support vector machine (SVM) algorithm. For example, the food estimation model 160 is constructed by learning using a Gaussian process regression (GPR) algorithm. For example, the food estimation model 160 is constructed by learning using a random forest (RF) algorithm. The food estimation model 160 may be constructed by unsupervised learning that classifies foods identified from input image data according to the image data. There is no particular limitation on the algorithm used to learn the food estimation model 160.
[0044] FIG. 7 is a conceptual diagram showing an example of estimating food stored in the interior of the managed refrigerator 100 using the food estimation model 160. The food estimation model 160 is a model constructed in advance by the learning shown in FIG. 6. In the example of FIG. 7, image data of the interior of the managed refrigerator 100 is used as input data. In response to input of image data, the food estimation model 160 outputs food names of food identified from the image data. In the example of FIG. 7, food names such as food F, food G, . . . , food N are output as estimation results from the food estimation model 160.
[0045] The food information generating unit 115 identifies the expiration date for each food extracted from the image data. For example, the food information generating unit 115 identifies an expiration date such as a best-before date or a use-by date indicated on a label of the food. For example, the food information generating unit 115 identifies the expiration date of the food based on a barcode or two-dimensional code indicated on the food.
[0046] FIG. 8 is a conceptual diagram for explaining the specification of a deadline by the food information generation unit 115. In the example of FIG. 8, a symbol string (number string) indicating the expiration date of the food is written in the expiration date display area of the label of the food. In the example of FIG. 8, a number string indicating that the expiration date of the food is March 31, 2024, such as "24.03.31", is written in the expiration date display area. In addition, in the example of FIG. 8, an identification code used to identify the food is written. For example, the food information generation unit 115 specifies the expiration date of the food from the symbol string or identification code indicating the expiration date or use-by date using a machine learning technique.
[0047] For example, the food information generating unit 115 may identify the expiration date of the food according to the color or shape of the food. Fresh foods such as vegetables and fresh fish do not have a symbol string or identification code indicating a best-before date or expiration date. The deterioration state of fresh foods such as vegetables and fresh fish can be determined according to the color of their surfaces. For example, the food information generating unit 115 determines the deterioration state of the food according to the hue, saturation, and brightness of the food identified from the image data. For example, the food information generating unit 115 may determine the deterioration state of the food using the method disclosed in Patent Document 4 (JP Patent Publication 2017-242911 A). For example, the food information generating unit 115 may determine the deterioration state of the food identified from the image data using a machine learning method.
[0048] The food information generating unit 115 generates food information including a food name of the food estimated using the image data and expiration date information indicating an expiration date specified for each food. The expiration date information includes an expiration date for each food.
[0049] Fig. 9 is an example of a table summarizing food information on a plurality of foods stored in the managed refrigerator 100. Today's date (2023 / 3 / 20) is shown in the upper right corner of the table in Fig. 9. The table in Fig. 9 summarises food information on foods stored in the managed refrigerator 100 in household A. The food information includes food name, quantity, and expiration date information for each food.
[0050] For foods such as vegetables such as cabbage, apples, and paprika, and foods prepared at home, the expiration date cannot be determined using a symbol string or identification code indicating a best-before date or expiry date. For example, the expiration date of such foods is determined according to the color of the food. For example, the expiration date of such foods may be determined according to the date and time that has elapsed since the food was identified in the managed refrigerator 100.
[0051] Beverages such as wine may be labeled with a manufacturing date but not with a best-by date. Such foods may be excluded from foods to be managed at the stage where they are identified using image data. For beverages such as wine that have a best-by date, the degree of maturity may be determined according to the time elapsed since the manufacturing date, and the best-by date may be notified as expiration date information.
[0052] The food information output unit 117 outputs the food information generated by the food information generation unit 115 to the food management device 15. The food information output to the food management device 15 is used to generate food management information related to the food stored in the managed refrigerator 100.
[0053] [Food control equipment] 10 is a block diagram showing an example of the configuration of food management apparatus 15. Food management apparatus 15 has a food information collection unit 151, a storage unit 153, a food management information generation unit 155, and a food management information output unit 157.
[0054] The food information collecting unit 151 collects food information on foods stored in the managed refrigerator 100 from the food information generating device 11. The food information collecting unit 151 stores the collected food information in the storage unit 153. The food information collecting unit 151 may output the acquired food information to the food management information generating unit 155.
[0055] Memory unit 153 stores food information about food stored in managed refrigerator 100. The food information stored in memory unit 153 is used by food management information generation unit 155 to generate food management information.
[0056] The food management information generating unit 155 acquires food information related to food stored in the managed refrigerator 100, which is stored in the memory unit 153. The food management information generating unit 155 may be configured to acquire food information from the food information collecting unit 151. The food management information generating unit 155 generates food management information using the acquired food information. The food management information is information according to the expiration date of food stored in the managed refrigerator 100. The food management information includes food name, quantity, expiration date information, expiration date score, and expiration date identification information for each food. The food management information generating unit 155 calculates the expiration date score according to the expiration date information. The expiration date score is a numerical value obtained by subtracting the current date from the expiration date. The food management information generating unit 155 generates expiration date identification information according to the expiration date score. The expiration date identification information is information according to the date and time until the expiration date.
[0057] For example, the food management information generating unit 155 assigns expiration identification information to each food item, such as "there is still plenty of time left until the expiration date" (○), "the expiration date is approaching" (△), or "the expiration date has passed" (×), depending on the date and time until the expiration date. In addition, the photographing date and time of the image data from which the expiration date information is generated may be included in the food management information. The expiration date identification information may be expressed by a symbol other than "○, △, or ×" as long as the symbol is set according to the date and time until the expiration date.
[0058] FIG. 11 is an example of a table summarizing food management information for a plurality of foods stored in the managed refrigerator 100. The upper right corner of the table in FIG. 11 shows today's date (2023 / 3 / 20). The table in FIG. 11 is a table summarizing food management information for foods stored in the managed refrigerator 100 of household A. For example, for eggs, the deadline is "2023 / 3 / 23" and the deadline score is 3. That is, for eggs, there are 3 days until the deadline. The eggs are given deadline identification information (△) indicating that the deadline is approaching. For example, for milk, the deadline is "2023 / 3 / 29" and the deadline score is 9. That is, for milk, there are 9 days until the deadline and deadline identification information (○) indicating that there is plenty of time until the deadline is given. For example, for bottled 2, the deadline is "2022 / 12 / 31" and the deadline score is -79. That is, for bottling 2, 79 days have passed since the expiration date, and expiration date identification information (x) indicating that the expiration date has passed is given.
[0059] For example, food management information generation unit 155 generates display information including a list of foods stored in managed refrigerator 100 (food list) as the food management information. For example, food management information generation unit 155 generates display information including a list of expiration dates of foods stored in managed refrigerator 100 (expiration date list) as the food management information. For example, food management information generation unit 155 generates display information including a list (warning list) indicating warnings based on expiration date identification information according to the date and time until the expiration date of foods stored in managed refrigerator 100 as the food management information.
[0060] Food management information output unit 157 outputs the food management information generated by food management information generation unit 155. For example, food management information output unit 157 displays food management information related to food stored in managed refrigerator 100 on a screen of a mobile terminal of a user who uses managed refrigerator 100. For example, food management information output unit 157 may output the food management information to an external system or the like that uses the food management information. There are no particular limitations on the use of the food management information output from food management information output unit 157.
[0061] For example, the food management system 10 is connected to an external system built on a cloud or a server through a mobile terminal (not shown) carried by a user. The mobile terminal (not shown) is a portable communication device. For example, the mobile terminal is a mobile communication device having a communication function such as a smartphone, a smart watch, or a mobile phone. For example, the food management system 10 is connected to the mobile terminal through wireless communication. For example, the food management system 10 is connected to the mobile terminal through a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). The communication function of the food management system 10 may be conforming to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark). For example, the food management system 10 may be connected to the mobile terminal through a wire such as a cable. Food management information regarding food stored in the refrigerator to be managed 100 may be used by an application installed on the mobile terminal. In this case, the mobile terminal executes a process using the food management information by application software or the like installed on the mobile terminal.
[0062] 12 to 17 are conceptual diagrams showing examples of food management information displayed on the screen of mobile terminal 170 of a user who uses managed refrigerator 100. The food management information displayed on the screen of mobile terminal 170 relates to food stored in managed refrigerator 100.
[0063] Fig. 12 is a conceptual diagram showing an example of display information including a food list generated as food management information. Fig. 12 shows an example of a food list including the quantity of each food item stored in the managed refrigerator 100 displayed on the screen of mobile terminal 170 as food management information. At the bottom of the screen, hatching is used to show that the quantity display has been selected from among buttons for accepting display selections of quantity, expiry date, and warning. The user can check the food list displayed on the screen of mobile terminal 170 and consider which foods to purchase depending on the quantity of each food item.
[0064] Fig. 13 is a conceptual diagram showing an example of display information including an expiration date list generated as food management information. Fig. 13 shows an example of an expiration date list including expiration dates for each food item stored in managed refrigerator 100 displayed on the screen of mobile terminal 170 as food management information. At the bottom of the screen, hatching is used to show that, of buttons for accepting selection of displaying quantity, expiration date, and warning, display of expiration date has been selected. The user can check the expiration date list displayed on the screen of mobile terminal 170 and consider which food items to consume according to the expiration date for each food item.
[0065] FIG. 14 is a conceptual diagram showing a display example of display information including a warning list generated as food management information. FIG. 14 shows an example of a warning list including warnings according to expiration date identification information for each food stored in the managed refrigerator 100, displayed on the screen of the mobile terminal 170 as food management information. Foods with expiration date identification information of △ and × are displayed on the screen of the mobile terminal 170. Also, warning information corresponding to warnings such as "The expiration date of eggs and paprika is approaching" and "Please discard bottle 2" are displayed on the screen of the mobile terminal 170. At the bottom of the screen, the state in which the warning display is selected is shown by hatching among buttons that accept the display selection of quantity, expiration date, and warning. The user can consider foods to dispose of according to the warning list and warning information displayed on the screen of the mobile terminal 170.
[0066] FIG. 15 is a conceptual diagram showing a display example of display information including expiration date identification information generated as food management information. FIG. 15 is an example in which expiration date identification information for each food stored in the managed refrigerator 100 is displayed on the screen of the mobile terminal 170 as food management information. In the example of FIG. 15, the food management device 15 acquires an image of the interior of the managed refrigerator 100 from the food information generating device 11. The expiration date identification information (○, △, ×) is displayed in association with the food on the screen of the mobile terminal 170. In the example of FIG. 15, the expiration date identification information (○, △, ×) is displayed superimposed on the food on the screen of the mobile terminal 170. At the bottom of the screen, hatching is used to show that the display of the refrigerator compartment X has been selected from among buttons that accept the display selection of the refrigerator compartment X, the refrigerator compartment Y, the inside of the door, and the freezer compartment. The user can visually check the expiration date of the food according to the expiration date identification information (○, △, ×) displayed on the screen of the mobile terminal 170. For example, the deadline identification information may be indicated by a symbol other than "○, △, ×". For example, the deadline identification information may be a deadline score corresponding to the date and time until the deadline, instead of symbols such as "○, △, ×".
[0067] FIG. 16 is a conceptual diagram showing a display example of display information including expiration date identification information generated as food management information. FIG. 16 is an example in which expiration date identification information for each food stored in the managed refrigerator 100 is displayed on the screen of the mobile terminal 170 as food management information. Recipes using foods that are close to their expiration date are displayed on the screen of the mobile terminal 170. In addition, recipe information corresponding to a warning such as "The expiration date of eggs and paprika is close" or "We recommend a recipe using eggs and paprika" is displayed on the screen of the mobile terminal 170. Furthermore, an image of a dish corresponding to the recipe information and a link to a site where the recipe of the dish can be referred to are displayed on the screen of the mobile terminal 170. At the bottom of the screen, a button for accepting the selection of recipe information and a button for accepting the display of another recipe are displayed. The user can select a recipe using a food that is close to its expiration date according to the recipe information displayed on the screen of the mobile terminal 170.
[0068] FIG. 17 is a conceptual diagram showing an example of display information according to a food list generated as food management information. FIG. 17 shows an example in which food management system 10 is connected to an online supermarket system for ordering food via the Internet. In the example of FIG. 17, a list of foods stored in managed refrigerator 100 whose quantity has become zero is displayed on the screen of mobile terminal 170 as food management information. Also, on the screen of mobile terminal 170, information according to the quantity of the food, such as "Eggs and paprika are out" and "Would you like to order eggs and paprika?", is displayed for the food whose quantity has become zero. At the bottom of the screen, buttons used for ordering foods whose quantity has become zero, such as eggs and paprika, and buttons for accepting orders for other foods are displayed. At the bottom of the screen, buttons for automatically ordering foods whose quantity has become zero and buttons for selecting not to order are displayed. The user can check the information displayed on the screen of mobile terminal 170 and select foods to order.
[0069] (operation) Next, an example of the operation of the food management system 10 according to this embodiment will be described with reference to the drawings. The operation of the managed refrigerator 100 will also be described below. The operation of the food information generation device 11 and the food management device 15 included in the food management system 10 will also be described separately below.
[0070] [Managed refrigerators] Fig. 18 is a flow chart for illustrating an example of an operation of managed refrigerator 100. In describing the process following the flow chart of Fig. 18, components of managed refrigerator 100 will be described as the subject of operation. The subject of operation of the process following the flow chart of Fig. 18 may be managed refrigerator 100. Description of the processing of the cooling mechanism will be omitted.
[0071] In Fig. 18, first, when it is time to photograph (Yes in step S101), photographing control unit 103 controls to turn on interior light 101 of managed refrigerator 100 (step S102). When interior light 101 is turned on, the interior of managed refrigerator 100 becomes bright. If it is not time to photograph (No in step S101), photographing control unit 103 waits until it is time to photograph.
[0072] Next, photography control section 103 controls camera 102 to take a photograph (step S103). By controlling camera 102 to take a photograph while interior light 101 is on, image data of the interior of managed refrigerator 100 is generated.
[0073] Next, the photographing control unit 103 stores the image data photographed by the camera 102 in the storage unit 104 (step S104).
[0074] Next, the photography control unit 103 controls to turn off the interior light 101 of the managed refrigerator 100 (step S105). By turning off the interior light 101, the interior of the managed refrigerator 100 becomes dark.
[0075] If it is time to output the image (Yes in step S106), the image output unit 105 outputs the image data stored in the storage unit 104 to the food information generation device 11 (step S107). If it is not time to output the image (No in step S106), the process returns to step S101.
[0076] After step S107, if management of the managed refrigerator 100 is to be continued (Yes in step S108), the process returns to step S101. If it is time to end management of the managed refrigerator 100 (No in step S108), the process according to the flowchart in Fig. 18 ends. Management of the managed refrigerator 100 may be continued or ended based on a continuation condition or an end condition set in advance.
[0077] [Food information generation device] Fig. 19 is a flowchart for explaining an example of the operation of the food information generation device 11. In explaining the process according to the flowchart of Fig. 19, the components of the food information generation device 11 will be described as the subject of the operations. The subject of the operations according to the flowchart of Fig. 19 may be the food information generation device 11.
[0078] In FIG. 19, first, image data acquisition unit 111 acquires image data of the interior of managed refrigerator 100 (step S111).
[0079] Next, the food information generating unit 115 extracts food from the acquired image data of the interior of the managed refrigerator 100 (Step S112).
[0080] If the extracted food is a food to be managed (Yes in step S113), the food information generation unit 115 extracts the expiration date of the food to be managed (step S114). If the extracted food is not a food to be managed (No in step S113), the process proceeds to step S117.
[0081] After step S114, when identification of all foods is completed (Yes in step S115), the food information generation unit 115 generates expiration date information of the foods to be managed (step S116). If identification of all foods is not completed (No in step S115), the process returns to step S113.
[0082] If the answer is No in step S113, or after step S116, the food information output unit 117 outputs the food information including the expiration date information to the food management device 15.
[0083] [Food control equipment] Fig. 20 is a flowchart for explaining an example of the operation of food management apparatus 15. In explaining the process according to the flowchart of Fig. 20, the components of food management apparatus 15 will be described as the subject of operations. The subject of operations of the process according to the flowchart of Fig. 20 may be food management apparatus 15.
[0084] In FIG. 20, first, the food information collection unit 151 collects food information on the food stored in the managed refrigerator 100 (step S151).
[0085] Next, the food management information generation unit 155 generates food management information using the collected food information (Step S152).
[0086] Next, the food management information output unit 157 outputs the generated food management information (Step S153).
[0087] As described above, the food management system of this embodiment includes a food information generating device and a food management device. The food information generating device generates food information including the expiration date for each food stored in the managed refrigerator. The food management device includes a food information collecting unit, a memory unit, a food management information generating unit, and a food management information output unit. The food information collecting unit collects food information including the expiration date for each food stored in the managed refrigerator. The memory unit stores food information for the food stored in the managed refrigerator. The food management information generating unit uses the collected food information to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food stored in the managed refrigerator. The food management information output unit outputs the generated food management information.
[0088] The food management system of this embodiment uses food information on food stored in a managed refrigerator to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food. The expiration date identification information is information indicating the expiration date of the food. In other words, according to this embodiment, information indicating the expiration date of the food stored in the managed refrigerator can be generated.
[0089] In one aspect of this embodiment, the food management information generation unit generates food management information including a warning list that summarizes warnings based on expiration date identification information. The food management information output unit outputs the food management information including the warning list. According to this aspect, it is possible to generate food management information including a warning list that summarizes warnings based on expiration date identification information.
[0090] In one aspect of this embodiment, the food information collection unit acquires an image of the interior of the managed refrigerator. The food management information generation unit generates food management information including an image in which expiration date identification information is associated with each food stored in the managed refrigerator. The food management information output unit outputs food management information including an image in which expiration date identification information is associated with each food stored in the managed refrigerator. According to this aspect, food management information can be generated including an image in which expiration date identification information is associated with each food stored in the managed refrigerator.
[0091] In one aspect of the present embodiment, the food management information generation unit generates food management information including recipes using foods stored in the managed refrigerator that are close to their expiration date based on the expiration date identification information. The food management information output unit outputs the food management information including the recipes. According to this aspect, it is possible to generate food management information including recipes using foods that are close to their expiration date.
[0092] In one aspect of this embodiment, the food information generation device has an image data acquisition unit, a food information generation unit, and a food information output unit. The image data acquisition unit acquires image data of the interior of the managed refrigerator. The food information generation unit identifies food stored in the managed refrigerator from the image data. The food information generation unit identifies expiration dates based on images of the identified food. The food information generation unit generates food information including the identified expiration dates. The food information output unit outputs the generated food information to the food management device. According to this aspect, food information including expiration dates of food stored in the managed refrigerator can be generated based on image data of the interior of the managed refrigerator.
[0093] Second embodiment Next, a food management system according to a second embodiment will be described with reference to the drawings. The food management system of this embodiment generates food management information for managing food stored in a plurality of managed refrigerators. The food management information includes expiration date information on expiration dates such as best-before dates and use-by dates of food stored in a plurality of managed refrigerators. The food management system of this embodiment includes a plurality of managed refrigerators. Each of the plurality of food information generating devices is provided in each of the plurality of managed refrigerators. The plurality of managed refrigerators are placed at any location within the range of the managed district. The location of the plurality of managed refrigerators is not particularly limited. For example, the plurality of managed refrigerators are placed in managed homes within the range of the managed district. For example, the plurality of managed refrigerators are placed in managed stores within the range of the managed district. In the following description, the same configurations and functions as those of the food management system according to the first embodiment may be omitted.
[0094] (composition) FIG. 21 is a block diagram showing an example of a configuration of a food management system 20 according to this embodiment. The food management system 20 of this embodiment comprises a food information generation device 21 and a food management device 25. The food management system 20 comprises a plurality of food information generation devices 21. In this embodiment, an example of comprehensively managing food stored in a plurality of managed refrigerators 200 will be described. The managed refrigerator 200 has a similar configuration to the managed refrigerator 100 of the first embodiment. Moreover, the food information generation device 21 has a similar configuration to the food information generation device 11 of the first embodiment. In the following, descriptions of the managed refrigerator 200 and the food information generation device 21 will be omitted, and the food management device 25 will be mainly described.
[0095] [Food control equipment] 22 is a block diagram showing an example of the configuration of the food management apparatus 25. The food management apparatus 25 has a food information collection unit 251, a storage unit 253, a food management information generation unit 255, and a food management information output unit 257.
[0096] The food information collecting unit 251 collects food information on foods stored in the multiple managed refrigerators 200 from the food information generating device 21. The food information collecting unit 251 stores the collected food information in the memory unit 253. The food information collecting unit 251 may output the acquired food information to the food management information generating unit 255.
[0097] The memory unit 253 stores food information related to food stored in the managed refrigerator 200. The food information stored in the memory unit 253 is used by the food management information generation unit 255 to generate food management information.
[0098] Food management information generation unit 255 acquires food information related to foods stored in the multiple managed refrigerators 200, which is stored in memory unit 253. Food management information generation unit 255 may be configured to acquire food information from food information collection unit 251. Food management information generation unit 255 generates food management information using the acquired food information. The food management information is information related to foods stored in the multiple managed refrigerators 200.
[0099] For example, food management information generation unit 255 generates display information including a list (food list) of foods stored in the multiple managed refrigerators 200 as the food management information. For example, food management information generation unit 255 generates display information including a list (expiration list) of expiration dates of foods stored in the multiple managed refrigerators 200 as the food management information. For example, food management information generation unit 255 generates display information including a list (warning list) showing warnings according to expiration date identification information of foods stored in the managed refrigerators 200 as the food management information.
[0100] Food management information output unit 257 outputs the food management information generated by food management information generation unit 255. For example, food management information output unit 257 displays food management information related to food stored in each of the managed refrigerators 200 on a screen of a mobile terminal of a user who uses each of the managed refrigerators 200. For example, food management information output unit 257 may output the food management information to an external system or the like that uses the food management information. There are no particular limitations on the use of the food management information output from food management information output unit 257.
[0101] For example, food management system 20 is connected to an external system built on a cloud or a server through a mobile terminal (not shown) carried by a user. The mobile terminal (not shown) is a portable communication device. For example, the mobile terminal is a mobile communication device having a communication function such as a smartphone, a smart watch, or a mobile phone. For example, food management system 20 is connected to the mobile terminal through wireless communication. For example, food management system 20 is connected to the mobile terminal through a wireless communication function (not shown) conforming to a standard such as Bluetooth (registered trademark) or WiFi (registered trademark). The communication function of food management system 20 may be conforming to a standard other than Bluetooth (registered trademark) or WiFi (registered trademark). For example, food management system 20 may be connected to the mobile terminal through a wire such as a cable. Food management information regarding food stored in managed refrigerator 200 may be used by an application installed on the mobile terminal. In this case, the mobile terminal executes a process using the food management information by application software or the like installed on the mobile terminal.
[0102] 23 is a conceptual diagram showing how food management device 25 installed in a store collects food information related to foods stored in managed refrigerators 200 installed in multiple homes. Food management device 25 collects food information related to foods stored in managed refrigerators 200 installed in multiple homes from food information generation device 21 which generates food information for those managed refrigerators 200.
[0103] The store is a store that sells food. There are no particular limitations on the business form of the store. For example, the store may be a retail store such as a supermarket that sells food. For example, the store may be an online supermarket where food can be ordered via the Internet. For example, the food management device 25 may be managed by an organization that handles administrative affairs, such as a city hall or a town hall.
[0104] The multiple households are residences of customers who use the store. For example, the multiple households entrust the management of managed refrigerators 200 installed in the households to the store by contract. For example, the multiple households are households located in a management area managed by the store. In addition, managed refrigerator 200 may be installed in a store that handles food such as a restaurant, instead of a residence.
[0105] FIG. 24 is an example of a table summarizing food information on multiple foods stored in managed refrigerators 200 installed in multiple homes. Today's date (2023 / 3 / 20) is shown in the upper right corner of the table in FIG. 24. The table in FIG. 24 includes food information on foods stored in managed refrigerators 200 installed in multiple homes (home A, home B, home C, ...). The food information includes food name, quantity, and expiration date information for each food item.
[0106] FIG. 25 is an example of a table summarizing food management information for multiple foods stored in managed refrigerators 200 installed in multiple homes. Today's date (2023 / 3 / 20) is shown in the upper right corner of the table in FIG. 25. The table in FIG. 25 includes food management information for foods stored in managed refrigerators 200 installed in multiple homes (home A, home B, home C, ...). The food management information includes, for each food, food name, quantity, expiration date information, expiration date score, and expiration date identification information.
[0107] FIG. 26 is a conceptual diagram showing an example of food management information generated based on food information on a plurality of foods stored in managed refrigerators 200 installed in a plurality of homes. The table in FIG. 26 includes statistical information indicating the sum of expiration date identification information for each food. The table in FIG. 26 indicates the sum of each of "○", "△", and "×" for each food. The table in FIG. 26 also includes the order quantity for each food. The statistical information and the order quantity are included in the food management information. The order quantity for each food is calculated based on the statistical information for each food. For example, the order quantity for each food is calculated using a machine learning technique.
[0108] FIG. 27 is a conceptual diagram for explaining an example of learning of an estimation model (order quantity estimation model 260) used to estimate the order quantity of food to be ordered by a store. The order quantity estimation model 260 outputs the order quantity of food in response to input of statistical information. The order quantity estimation model 260 uses food name and statistical information (explanatory variables) for each food, and the order quantity (response variable) for each food. The statistical information (explanatory variables) is actual statistical information when food was ordered in the past. The statistical information is the total value of the expiration date identification information for each food. The total value of the expiration date identification information is the total value of each of "○", "△", and "×" for each food. The order quantity (response variable) is the order quantity for each food that was actually ordered when the statistical information was obtained. The order quantity may be a numerical value estimated by the order quantity estimation model 260, or may be a numerical value set by a store clerk based on experience.
[0109] For example, the order quantity estimation model 260 is constructed by learning using a linear regression algorithm. For example, the order quantity estimation model 260 is constructed by learning using a support vector machine (SVM) algorithm. For example, the order quantity estimation model 260 is constructed by learning using a Gaussian process regression (GPR) algorithm. For example, the order quantity estimation model 260 is constructed by learning using a random forest (RF) algorithm. The order quantity estimation model 260 may be constructed by unsupervised learning that classifies foods identified from input image data according to the image data. There are no particular limitations on the algorithm used to train the order quantity estimation model 260.
[0110] FIG. 28 is a conceptual diagram showing an example of estimating the order quantity of food items to be ordered by a store using order quantity estimation model 260. Order quantity estimation model 260 outputs the order quantity of food items in response to input of statistical information. Order quantity estimation model 260 is a model constructed in advance by the learning shown in FIG. 27. In the example of FIG. 28, statistical information of food items stored in managed refrigerators 200 installed in multiple homes is used as input data. Order quantity estimation model 260 outputs the order quantity for each food item in response to input of statistical information for each food item.
[0111] Fig. 29 is a conceptual diagram showing how to place an order based on food management information generated according to food information collected from multiple households. In the example of Fig. 29, food information collected from multiple households is collected in food management device 25 installed in a store. Food management device 25 may be installed in terminal device 281 installed in the store. Food management device 25 may also be built in a server or cloud connected to terminal device 281 via the Internet.
[0112] In the example of Fig. 29, food management information generated using food information collected by food management device 25 displays statistical information for each food item and order quantities corresponding to the statistical information in a table format on the screen of terminal device 281. The person in charge of ordering at the store checks the food management information displayed on the screen of terminal device 281 and changes the order quantity as necessary. When the order button displayed on the screen of terminal device 281 is pressed, order information for the order quantity displayed on the screen of terminal device 281 is sent to the supplier for each food item.
[0113] 30 is a conceptual diagram showing how order information is transmitted from terminal device 281 installed in a store to terminal device 282 installed in a headquarters that manages the store. The headquarters, which receives an order from the store, arranges for the order to be placed with a food vendor in accordance with the order information from the store. Note that orders may also be placed directly from terminal device 281 installed in the store to a food vendor without going through the headquarters.
[0114] FIG. 31 is a conceptual diagram showing how order information is transmitted from multiple stores managed by the headquarters to the headquarters. The headquarters, which receives orders from multiple stores, arranges orders with food vendors for each food item according to the order information for each store. For example, food information collected for each store may be transmitted from the multiple stores to the headquarters, and food management information may be generated at the headquarters. In this case, food management device 25 is installed at the headquarters.
[0115] FIG. 32 is a conceptual diagram showing how order information is sent from terminal device 281 installed in a store or terminal device 282 installed in the headquarters to terminal device 285 installed in a contract producer. For example, a contract producer is a producer that handles food products such as vegetables and fruits. A contract producer may be a producer that handles any food product, not just foods such as vegetables and fruits. In response to order information from a store, the contract producer prepares to order the food products it handles. The contract producer arranges for the food products that are ready to be ordered to be delivered to the store or headquarters that placed the order.
[0116] Figure 33 is a conceptual diagram showing how order information is sent to a contract producer from multiple stores that have contracts with that contract producer. The contract producer, who receives orders from multiple stores, arranges orders for each food item according to the order information for each store. The contract producer can create production plans for vegetables, fruits, etc. according to the order status from multiple stores. If a scheme for creating production plans in this way can be established, it will be possible to reduce food waste not only at the supply stage, but also at the production stage.
[0117] (operation) Next, an example of the operation of the food management system 20 according to this embodiment will be described with reference to the drawings. The operations of the managed refrigerator 200 and the food information generating device 21 are similar to the corresponding configurations in the first embodiment, so the description will be omitted. Below, the operation of the food management device 25 will be described.
[0118] [Food control equipment] Fig. 34 is a flowchart for explaining an example of the operation of food management apparatus 25. In explaining the process according to the flowchart of Fig. 34, the components of food management apparatus 25 will be described as the subject of the operations. The subject of the operations according to the flowchart of Fig. 34 may be food management apparatus 25.
[0119] In FIG. 34, first, the food information collection unit 251 collects food information on foods stored in the managed refrigerator 200 installed in the managed home (step S251).
[0120] Next, the food management information generation unit 255 generates food management information using the collected food information (Step S252).
[0121] Next, the food management information output unit 257 outputs the generated food management information (Step S253).
[0122] As described above, the food management system of this embodiment includes a food information generating device and a food management device. The food information generating device generates food information including the expiration date for each food stored in the managed refrigerator. The food management device includes a food information collecting unit, a memory unit, a food management information generating unit, and a food management information output unit. The food information collecting unit collects food information including the expiration date for each food stored in the managed refrigerators. The memory unit stores food information for the food stored in the managed refrigerators. The food management information generating unit uses the food information collected for the managed refrigerators to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food stored in the managed refrigerators. The food management information output unit outputs the generated food management information.
[0123] The food management system of this embodiment uses food information on foods stored in multiple managed refrigerators to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food. The expiration date identification information is information indicating the expiration date of the food. In other words, according to this embodiment, information indicating the expiration date of foods stored in multiple managed refrigerators can be generated.
[0124] In one aspect of this embodiment, the food management information generation unit generates statistical information indicating a sum of expiration date identification information for each food item based on expiration date identification information corresponding to the date and time until the expiration date of the food items stored in the multiple managed refrigerators. The food management information generation unit calculates an order quantity for each food item to be ordered based on the statistical information. The food management information generation unit generates order information including the calculated order quantity for each food item. The food management information output unit transmits the generated order information to a supplier for each food item. According to this aspect, food items can be ordered based on the order information generated according to the expiration dates of the foods stored in the multiple managed refrigerators.
[0125] (Third embodiment) Next, a food management device 35 according to a third embodiment will be described with reference to the drawings. The food management device 35 of this embodiment has a simplified configuration of the food management device according to the first and second embodiments.
[0126] (composition) 35 is a block diagram showing an example of the configuration of a food management apparatus 35 according to this embodiment. The food management apparatus 35 includes a food information collection unit 351, a food management information generation unit 355, and a food management information output unit 357.
[0127] The food information collection unit 351 collects food information including the expiration date of each food stored in the managed refrigerator. The food management information generation unit 355 uses the collected food information to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food stored in the managed refrigerator. The food management information output unit outputs the generated food management information.
[0128] (operation) Fig. 36 is a flowchart for explaining an example of the operation of food management apparatus 35. In explaining the process according to the flowchart of Fig. 36, the components of food management apparatus 35 will be described as the subject of the operations. The subject of the operations according to the flowchart of Fig. 36 may be food management apparatus 35.
[0129] In FIG. 36, first, the food information collection unit 351 collects food information including the expiration date of each food item stored in the managed refrigerator (step S351).
[0130] Next, food management information generation unit 355 uses the collected food information to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food stored in the managed refrigerator (Step S352).
[0131] Next, the food management information output unit outputs the generated food management information (Step S353).
[0132] The food management device of this embodiment uses food information on food stored in a managed refrigerator to generate food management information including expiration date identification information corresponding to the date and time until the expiration date of the food. The expiration date identification information is information indicating the expiration date of the food. In other words, according to this embodiment, information indicating the expiration date of the food stored in a managed refrigerator can be generated.
[0133] (Hardware) Next, a hardware configuration for executing control and processing according to each embodiment of the present disclosure will be described with reference to the drawings. Here, an information processing device 90 (computer) in FIG. 37 is given as an example of such a hardware configuration. The information processing device 90 in FIG. 37 is an example of a configuration for executing control and processing according to each embodiment, and does not limit the scope of the present disclosure.
[0134] As shown in Fig. 37, an information processing device 90 includes a processor 91, a main storage device 92, an auxiliary storage device 93, an input / output interface 95, and a communication interface 96. In Fig. 37, the interface is abbreviated as I / F (Interface). The processor 91, the main storage device 92, the auxiliary storage device 93, the input / output interface 95, and the communication interface 96 are connected to each other via a bus 98 so as to be able to communicate data with each other. In addition, the processor 91, the main storage device 92, the auxiliary storage device 93, and the input / output interface 95 are connected to a network such as the Internet or an intranet via the communication interface 96.
[0135] The processor 91 loads a program (instructions) stored in the auxiliary storage device 93 or the like into the main storage device 92. For example, the program is a software program for executing the control and processing of each embodiment. The processor 91 executes the program loaded into the main storage device 92. The processor 91 executes the program to execute the control and processing of each embodiment.
[0136] The main memory device 92 has an area in which a program is loaded. The processor 91 loads a program stored in the auxiliary memory device 93 or the like in the main memory device 92. The main memory device 92 is realized by a volatile memory such as a dynamic random access memory (DRAM). Furthermore, a non-volatile memory such as a magneto resistive random access memory (MRAM) may be configured / added to the main memory device 92.
[0137] The auxiliary storage device 93 stores various data such as programs. The auxiliary storage device 93 is realized by a local disk such as a hard disk or a flash memory. Note that it is also possible to configure the main storage device 92 to store various data and omit the auxiliary storage device 93.
[0138] The input / output interface 95 is an interface for connecting the information processing device 90 to peripheral devices based on standards and specifications. The communication interface 96 is an interface for connecting to an external system or device through a network such as the Internet or an intranet based on standards and specifications. The input / output interface 95 and the communication interface 96 may be a common interface for connecting to an external device.
[0139] Input devices such as a keyboard, a mouse, and a touch panel may be connected to the information processing device 90 as necessary. These input devices are used to input information and settings. When a touch panel is used as the input device, a screen having the function of the touch panel becomes the interface. The processor 91 and the input devices are connected via an input / output interface 95.
[0140] The information processing device 90 may be equipped with a display device for displaying information. When the display device is equipped, the information processing device 90 is equipped with a display control device (not shown) for controlling the display of the display device. The information processing device 90 and the display device are connected via an input / output interface 95.
[0141] The information processing device 90 may be provided with a drive device. The drive device mediates between the processor 91 and a recording medium (program recording medium) in reading data and programs stored in the recording medium and writing the processing results of the information processing device 90 to the recording medium. The information processing device 90 and the drive device are connected via an input / output interface 95.
[0142] The above is an example of a hardware configuration for enabling the control and processing according to each embodiment of the present disclosure. The hardware configuration in FIG. 37 is an example of a hardware configuration for executing the control and processing according to each embodiment, and does not limit the scope of the present disclosure. A program for causing a computer to execute the control and processing according to each embodiment is also included in the scope of the present disclosure.
[0143] A program recording medium on which the program according to each embodiment is recorded is also included in the scope of the present disclosure. The recording medium can be realized by an optical recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc). The recording medium may be realized by a semiconductor recording medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card. The recording medium may also be realized by a magnetic recording medium such as a flexible disk or other recording medium. When the program executed by the processor is recorded on a recording medium, the recording medium corresponds to a program recording medium.
[0144] The components of each embodiment may be combined in any manner. The components of each embodiment may be realized by software. The components of each embodiment may be realized by a circuit.
[0145] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]
[0146] 10, 20 Food Control System 11, 21 Food information generation device 15, 25, 35 Food control equipment 100, 200 Managed refrigerators 101 Interior light 102 Camera 103 Shooting control section 104 Storage section 105 Image output unit 107 Cooling mechanism 108 Cooling control unit 109 Thermometer 111 Image data acquisition unit 113 Storage section 115 Food Information Generation Department 117 Food Information Output Unit 151, 251, 351 Food Information Collection Division 153, 253 storage section 155, 255, 355 Food Management Information Generation Department 157, 257, 357 Food management information output section 170 Mobile Devices 281, 282, 285 Terminal equipment 90 Information processing equipment 91 Processor 92 Main memory 93 Auxiliary storage device 95 Input / Output Interface 96 Communication Interface
Claims
1. a food information collecting unit that collects food information including expiration dates for each food item stored in the managed refrigerator; a food management information generating unit that generates food management information including expiration date identification information corresponding to a date and time until the expiration date of the food stored in the managed refrigerator by using the collected food information; A food management device comprising: a food management information output unit that outputs the generated food management information.
2. The food management information generation unit includes: generating the food control information including a warning list in which warnings based on the expiration date identification information are compiled; The food management information output unit includes: The food management device according to claim 1 , wherein the food management information includes the warning list.
3. The food information collection unit includes: Acquire an image of the interior of the managed refrigerator, The food management information generation unit includes: generating the food management information including an image in which the expiration date identification information is associated with each food item stored in the managed refrigerator; The food management information output unit includes: The food management device according to claim 1 , wherein the food management information includes an image in which the expiration date identification information is associated with each food item stored in the managed refrigerator.
4. The food management information generation unit includes: generating the food management information including recipes using food items stored in the managed refrigerator that are close to their expiration dates based on the expiration date identification information; The food management information output unit includes: The food management device of claim 1 , wherein the food management information includes the recipe.
5. The food information collection unit includes: Collecting the food information including the expiration date of each food item stored in the plurality of managed refrigerators; The food management information generation unit includes: generating the food management information according to the expiration date of each food stored in the plurality of managed refrigerators using the food information collected for the plurality of managed refrigerators; The food management information output unit includes: The food management device according to claim 1 , wherein the generated food management information is output.
6. The food management information generation unit includes: generating statistical information indicating a total of expiration date identification information for each food item based on expiration date identification information corresponding to a date and time until the expiration date of the food items stored in the plurality of managed refrigerators; Calculate the order quantity for each food item to be ordered based on the statistical information; generating order information including the calculated order quantity for each food item; The food management information output unit includes: The food management device according to claim 5 , wherein the generated ordering information is transmitted to a supplier for each food item.
7. A food management device according to any one of claims 1 to 6, A food management system comprising: a food information generating device that generates food information including an expiration date for each food item stored in a managed refrigerator.
8. The food information generating device includes: An image data acquisition unit that acquires image data of an interior of the managed refrigerator; a food information generating unit that identifies food stored in the managed refrigerator from the image data, identifies a deadline based on an image of the identified food, and generates the food information including the identified deadline; The food management system according to claim 7 , further comprising: a food information output unit that outputs the generated food information to the food management device.
9. The computer Collect food information including the expiration date of each food item stored in the managed refrigerator, Using the collected food information, generate food management information according to the expiration date of each food stored in the managed refrigerator; A food management method that outputs the generated food management information.
10. A process of collecting food information including expiration dates for each food item stored in the managed refrigerator; A process of generating food management information according to the expiration date of each food stored in the managed refrigerator using the collected food information; and outputting the generated food management information.
Citation Information
Patent Citations
Automatic refrigerating system for food and refrigerator
JP2001317862A
Refrigerator and food control method
JP2002147931A
Food purchase management support system
JP2008242911A
JP2017-242911A
