Food management method, program, food management system, and food management tag

Abstract

In a food management method, a captured image is acquired (S101), the captured image being an image obtained by capturing an image of a food to which a food management label is attached or a storage container for storing the food to which the food management label is attached, the food management label being a label having an identifier associated with the food. Furthermore, in the food management method, the food or the storage container to which the food management label is attached is identified on the basis of the acquired captured image (S102, S103). Furthermore, in the food management method, the remaining amount of the food is estimated on the basis of the size of the identified food management tag (S104). Furthermore, in the food management method, remaining amount information relating to the estimated remaining amount of the food is output (S105).

Description

Technical Field

[0001] The present disclosure relates to a food management method, a program, a food management system, and a food management label for managing food. Background Art

[0002] Patent Document 1 discloses a determination device that determines whether items (food) are stored or taken out of a storage device (refrigerator).

[0003] (Prior art literature)

[0004] (Patent Document)

[0005] Patent Document 1: Japanese Patent Application Publication No. 2016-023584 Summary of the invention

[0006] Problems to be solved by the invention

[0007] The present disclosure provides a food management method and the like that can easily manage food.

[0008] Methods for solving problems

[0009] In a food management method involved in one embodiment of the present invention, a camera image is obtained, which is an image obtained by photographing a food with a food management label or a storage container with the food management label for storing the food by a camera element, and the food management label is a label with an identifier associated with the food. Based on the obtained camera image, the food with the food management label or the storage container is identified, and based on the size of the identified food management label, the remaining amount of the food is estimated, and remaining amount information related to the estimated remaining amount of the food is output.

[0010] A program according to one aspect of the present disclosure is for causing one or more processors to execute the food management method.

[0011] A food management system according to one embodiment of the present disclosure includes an acquisition unit, an identification unit, an estimation unit, and an output unit. The acquisition unit acquires a camera image, wherein the camera image is an image obtained by photographing a food with a food management label or a storage container with the food management label for storing the food by an imaging element, wherein the food management label is a label having an identifier associated with the food. The identification unit identifies the food with the food management label or the storage container based on the camera image acquired by the acquisition unit. The estimation unit estimates the remaining amount of the food based on the size of the food management label identified by the identification unit. The output unit outputs remaining amount information related to the remaining amount of the food estimated by the estimation unit.

[0012] A food management label according to one embodiment of the present disclosure is a food management label that is installed on food or a storage container for storing the food and has an identifier associated with the food. The food management label has a variable portion that can change in size according to the remaining amount of the food.

[0013] Effects of the Invention

[0014] The food management method and the like according to the present disclosure have the advantage of facilitating food management. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a block diagram showing the overall structure of the food management system in Implementation Example 1.

[0016] Figure 2 This is a diagram showing a first usage example of the food management label in the first embodiment.

[0017] Figure 3 This is a diagram showing a second usage example of the food management label in the first embodiment.

[0018] Figure 4 This is a flowchart showing an operation example of the food management system in Implementation Example 1.

[0019] Figure 5 This is a block diagram showing the overall structure of a food management system in a modified example including the first embodiment.

[0020] Figure 6 This is a block diagram showing the overall structure of the food management system in Implementation Example 2.

[0021] Figure 7 This is a diagram showing an example of use of the food management label in the second embodiment.

[0022] Figure 8 This is a diagram showing an example of a food management label in the second embodiment.

[0023] Fig. 9 This is a flowchart showing an example of the operation of the food management system in Implementation Example 2. DETAILED DESCRIPTION

[0024] (Insights that form the basis of this disclosure)

[0025] First, the focus of the inventors of the present invention will be described below.

[0026] As in the technology disclosed in the previous patent document 1, the food is identified by taking an image including the food and performing appropriate image recognition processing on the image obtained by taking the food. Using this technology, for example, when managing the food stored in the refrigerator, the following problems may occur.

[0027] That is, the shape and size of food may change depending on the usage of a portion of the food, such as when the food is used for cooking. Therefore, the above-mentioned technology has a problem that it is difficult to determine whether the unused food and the food of which a portion has been used for cooking are the same food. In addition, the above-mentioned technology also has a problem that it is difficult to determine whether the food before and after use is the same, so it is of course difficult to grasp the remaining amount of food.

[0028] Here, for example, it is considered that by attaching RF (Radio Frequency) tags to food, the food can be identified using RFID (Radio Frequency Identifier) ​​technology. However, there is a problem that RF tags are not yet widely used, and it is not realistic to attach RF tags to all food. In addition, even if RF tags are attached to food, the remaining amount of food cannot be grasped.

[0029] In view of the above circumstances, the inventors of the present invention have studied the present disclosure.

[0030] The following describes the embodiments in detail with reference to the accompanying drawings as appropriate. However, unnecessary detailed descriptions may be omitted. For example, detailed descriptions of well-known matters or repeated descriptions of substantially the same structures may be omitted. This is to avoid the following description from becoming too lengthy and to facilitate understanding by those skilled in the art.

[0031] In addition, the drawings and the following descriptions are provided by the inventors of the present invention to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the contents described in the scope of the technical claims.

[0032] (Implementation Method 1)

[0033] [1-1. Overall composition]

[0034] First, use Figure 1 The overall structure of the food management system 100 in the first embodiment will be described. Figure 11 is a block diagram showing the overall structure of the food management system 100 in the first embodiment. In the first embodiment, the food management system 100 is used to manage the remaining amount of the food 3 stored in the refrigerator 2, in other words, to manage the usage amount of the food 3. The food 3 may be directly stored in the refrigerator 2 or stored in a storage container 5 (see Figure 3 ) is stored in the refrigerator 2. In the first embodiment, the food 3 or the storage container 5 is stored in the refrigerator 2 in a state where the food management label 4 is attached.

[0035] The food management label 4 is mounted on the food 3 or the storage container 5 for storing the food 3, and has an identifier associated with the food 3. The identifier here can be a specific string such as a so-called ID (Identifier) ​​number, or can be a simple inherent feature that can be distinguished from other food management labels. More specifically, the identifier can be in a form in which the result of image recognition processing performed on the image of the food management label in the recognition unit 112 of the food management system 100 described later can be distinguished from the result of image recognition processing performed on the image of other food management labels.

[0036] In the first embodiment, the food management label 4 is, for example, a note in the shape of a cat, on which a character string indicating the name of the food 3 is written (see Figure 2 In other words, in the first embodiment, the identifier of the food management label 4 is a combination of the shape of the food management label 4 and the character string written on the food management label 4.

[0037] In addition, the identifier of the food management label 4 may be, for example, a character string written on the food management label 4, a shape of the food management label 4, or a pattern applied to the food management label 4. Furthermore, the identifier of the food management label 4 may be a combination of two or more of a shape, a pattern, and a character string. Furthermore, in Embodiment 1, the food management label 4 may also be an RF tag.

[0038] The food management label 4 is attached to the food 3 or the storage container 5 by, for example, sticking the surface coated with an adhesive to any part of the food 3 or the storage container 5. In addition, the food management label 4 may be directly attached to the food 3 or the storage container 5 as described above, or may be attached to the food 3 or the storage container 5 using, for example, a rubber band or a paper clip. Furthermore, the food management label 4 may be attached to the packaging of the food 3, for example.

[0039] The refrigerator 2 includes an image sensor 21, a communication interface (hereinafter referred to as communication I / F (Interface)) 22, a processor 23, and a storage device 24. In addition, the refrigerator 2 may also include a cold storage room with a door, a drawer constituting a vegetable room, etc. In addition, in the first embodiment, the image sensor 21, the communication I / F 22, the processor 23, and the storage device 24 are provided in the refrigerator 2, but they may also be provided around the refrigerator 2 as another device independent of the refrigerator 2.

[0040] The imaging element 21 is an image sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor, and is used to capture the food 3 or the storage container 5 with the food management label 4. The imaging element 21 is provided, for example, in the storage of the refrigerator 2, on the door of the refrigerator 2, or on a member protruding from the top plate of the refrigerator 2 toward the door side.

[0041] In addition, the imaging element 21 may be arranged inside or outside the refrigerator 2. Furthermore, the imaging element 21 may not be provided in advance by the refrigerator 2, but may be installed later. Furthermore, the imaging element 21 may be in a form that can be carried by the user. In this case, the user may also hold the imaging element 21 to photograph the food management label 4. Furthermore, the imaging element 21 may be in an independent form, for example, it may be assembled in any device such as a portable terminal having an imaging function.

[0042] Furthermore, there may be a plurality of imaging elements 21, such as an imaging element 21 for a refrigerator, an imaging element 21 for a freezer, or an imaging element 21 for a drawer. When there are a plurality of imaging elements 21, the processor 23 may determine which imaging element 21 is used to obtain an image.

[0043] Furthermore, the imaging element 21 is not only used to photograph the food 3 or storage container 5 with the food management label 4 as described later, but can also be used to photograph other objects such as users. When the imaging element is used to photograph the user, the user can be identified based on the image obtained by photographing the user.

[0044] The imaging element 21, for example, uses the opening and closing of the door of the refrigerator 2 by the user as a trigger to photograph the food 3 or the storage container 5 with the food management label 4. Specifically, the imaging element 21 may photograph the food 3 or the storage container 5 with the food management label 4 while the door of the refrigerator 2 is open. Alternatively, the imaging element 21 may photograph the food 3 or the storage container 5 with the food management label 4 outside the refrigerator 2 during a certain period of time after the door of the refrigerator 2 is detected to be closed. Alternatively, the imaging element 21 may photograph the food 3 or the storage container 5 with the food management label 4 inside the refrigerator 2 after a certain period of time has passed since the door of the refrigerator 2 is detected to be closed. In addition, the imaging element 21 may always photograph regardless of the opening and closing of the refrigerator 2. Therefore, the photographing by the imaging element 21 is performed at the timing when the user puts the food 3 or the storage container 5 into the refrigerator 2 (storage space) or when the user takes the food 3 or the storage container 5 out of the refrigerator 2.

[0045] The communication I / F22 communicates with the communication I / F10 of the food management system 100 via an external network NT1 such as the Internet. The communication between the communication I / F22 and the communication I / F10 of the food management system 100 may be wired communication in addition to wireless communication. Furthermore, there is no particular limitation on the standard of communication between the communication I / F22 and the food management system 100. When the imaging element 21 photographs the food 3 or the storage container 5 with the food management label 4, the communication I / F22 sends the data of the photographed image to the communication I / F10 of the food management system 100.

[0046] The processor 23 controls the operation of the refrigerator 2. The processor 23 is configured to include a memory, and various functions are achieved by the processor 23 executing a program stored in the memory.

[0047] The storage device 24 is a storage device that stores information (computer programs, etc.) required for the processor 11 to execute various functions. Although the storage device 24 is implemented by, for example, a semiconductor memory, there is no particular limitation, and a known electronic information storage method can be used. The storage device 24 stores, for example, data of an image captured by the imaging element 21.

[0048] [1-2. Food management system]

[0049] Next, the food management system 100 is described in detail. In the first embodiment, the food management system 100 is composed of, for example, a server 1 installed at a distance from a facility (here, a residence where a user lives) where a refrigerator 2 is installed. In addition, the food management system 100 may include not only the server 1 but also the refrigerator 2.

[0050] In the first embodiment, the description is made on the case where there is one refrigerator 2 as the target of the food management system 100 . However, there may be a plurality of refrigerators 2 as the target of the food management system 100 .

[0051] like Figure 1 As shown, the food management system 100 (server 1 ) includes a communication I / F 10 , a processor 11 , and a storage device 12 .

[0052] As described above, the communication I / F 10 has a function of communicating with the communication I / F 22 of the refrigerator 2 via the external network NT1. The communication I / F 10 receives the data of the camera image captured by the imaging element 21 and sent from the communication I / F 22 of the refrigerator 2. The camera image data received by the communication I / F 10 is provided to the processor 11.

[0053] The processor 11 is configured to include a memory, and various functions are performed by executing a program stored in the memory by the processor 11. Specifically, the processor 11 functions as an acquisition unit 111, an identification unit 112, an estimation unit 113, and an output unit 114 by executing a program stored in the memory by the processor 11. In other words, the food management system 100 includes the acquisition unit 111, the identification unit 112, the estimation unit 113, and the output unit 114.

[0054] The acquisition unit 111 acquires a camera image obtained by photographing the food 3 with the food management label 4 or the storage container 5 with the food management label 4 by the imaging element 21. In the first embodiment, the acquisition unit 111 acquires the camera image data transmitted from the communication I / F 22 of the refrigerator 2 via the communication I / F 10.

[0055] The recognition unit 112 recognizes the food 3 or the storage container 5 to which the food management label 4 is attached, based on the camera image obtained by the acquisition unit 111. Specifically, the recognition unit 112 recognizes the image of the portion of the food management label 4 included in the camera image by performing appropriate image recognition processing on the camera image obtained by the acquisition unit 111, and recognizes the image of the portion of the food 3 or the storage container 5 included in the camera image. Here, the recognition unit 112 recognizes the portion of the food management label 4 included in the camera image as an identifier of the food management label 4. The identifier of the food management label 4 recognized by the recognition unit 112 and the image of the portion of the food 3 or the storage container 5 recognized by the recognition unit 112 are stored in the storage device 12 as data associated with each other. Accordingly, the identifier of the food management label 4 is associated with the image of the portion of the food 3 or the storage container 5 to which the food management label 4 is attached and stored in the storage device 12.

[0056] In the first embodiment, the recognition unit 112 further recognizes the type of the food 3 based on the camera image obtained by the acquisition unit 111. Therefore, in the first embodiment, the identifier of the food management label 4 is associated not only with the image of the food 3 or the storage container 5 to which the food management label 4 is attached, but also with the type of the food 3 and stored in the storage device 12. The recognition unit 112 can recognize the type of the food 3 by using, for example, a learned model that performs machine learning in a manner that uses the image of the food 3 as an input and the type of the food 3 as an output.

[0057] The estimation unit 113 estimates the remaining amount of the food 3 based on the size of the food management label 4 identified by the recognition unit 112. In the first embodiment, the estimation unit 113 estimates the remaining amount of the food 3 by comparing the size of the food management label 4 identified by the recognition unit 112 with the size of the food 3 or the size of the storage container 5 identified by the recognition unit 112. Specifically, the estimation unit 113 estimates the remaining amount of the food 3 based on the ratio of the area of ​​the food 3 or the storage container 5 in the identified camera image to the area of ​​the food management label 4 in the identified camera image. In addition, the area of ​​the area here may also be the number of pixels contained in the area. The remaining amount of the food 3 estimated by the estimation unit 113 is associated with the identifier of the food management label 4 and the food 3 with the food management label 4 and stored in the storage device 12.

[0058] Alternatively, the remaining amount of food 3 may be associated with the registration date of the food 3 or the update date after the remaining amount of the food 3 is updated and stored in the storage device 12. Alternatively, the remaining amount of food 3 may be associated with the storage location in the refrigerator 2 and stored in the storage device 12. Furthermore, when there are multiple imaging elements 21, the remaining amount of food 3 may be associated with information for determining which imaging element 21 is used to capture the image and stored in the storage device 12.

[0059] Furthermore, in the case where the user can be identified based on the image captured by the imaging element 21, the remaining amount of the food 3 can also be associated with the user's identifier and stored in the storage device 12. In this case, even if the user is away from the refrigerator 2, such as when going out, the user can browse the above-mentioned various information including the remaining amount of the food 3 associated with the user's identifier using an information terminal such as a smart phone owned by the user. Furthermore, when browsing the above-mentioned various information, the user can also modify at least a part of the above-mentioned various information by operating the information terminal.

[0060] And for example, in a case where the remaining amount of food 3 is associated only with the user's identifier and stored in the storage device 12, the above-mentioned registration date, update date, storage location in the refrigerator 2, or information for determining which camera element 21 took the photo, etc. may be further added and associated with these.

[0061] That is, in the first embodiment, the size of the food management label 4 does not change, and of course, the size does not change before and after the food 3 is used. However, the size of the food 3 changes every time it is used, for example, for cooking. Moreover, when the food 3 is used, for example, for cooking, the size of the storage container 5 changes by transferring the food 3 to a smaller storage container. Therefore, by finding the size of the food 3 or the ratio of the size of the storage container 5 to the size of the food management label 4, the remaining amount of the food 3 can be estimated.

[0062] Figure 2 This is a diagram showing a first usage example of the food management label 4 in the first embodiment. Figure 2 FIG. 2 shows an image obtained by photographing a food 3 with a food management label 4 attached thereto by the image pickup element 21. Figure 2 In the example shown, the food 3 is cabbage, and the food management label 4 is a note imitating the shape of a cat with "cabbage" written on it. Figure 2 In the camera image shown in (a), food 3 is unused. Figure 2 In the captured image shown in (b), the food 3 is half used due to cooking or the like.

[0063] For example, it is assumed that when the user first stores the food 3 in the refrigerator 2, the acquisition unit 111 acquires Figure 2 In this case, the estimation unit 113 estimates that the remaining amount of the food 3 is 100%. Figure 2 The camera image shown in (a) is used to calculate the ratio of the area of ​​the identified food 3 to the area of ​​the identified food management label 4 as "α". This ratio is associated with the identifier of the food management label 4 and the food 3 with the food management label 4 and stored in the storage device 12.

[0064] Thus, for example, when a user takes out a food 3 with the same food management label 4 as described above from the refrigerator 2 and then puts the food 3 back into the refrigerator 2, the obtaining unit 111 obtains Figure 2 In this case, the estimation unit 113 Figure 2The camera image shown in (b) of FIG. 1 is used to calculate the ratio of the area of ​​the recognized food 3 to the area of ​​the recognized food management label 4 as "α / 2". Thus, the estimation unit 113 estimates that the remaining amount of the food 3 is 50% by comparing it with the ratio "α" when the remaining amount of the food 3 read from the storage device 12 is 100%.

[0065] Here, at the timing when the imaging element 21 takes a picture, the distance of the food 3 with the food management label 4 attached thereto relative to the imaging element 21 may change. Therefore, the area of ​​the region of the food management label 4 in the recognized camera image and the area of ​​the region of the food 3 may change each time the imaging element 21 takes a picture. In the first embodiment, the estimation unit 113 estimates the remaining amount of the food 3 based on the ratio of the area of ​​the food 3 in the recognized camera image to the area of ​​the recognized food management label 4, using the fact that the size of the food management label 4 does not change. Therefore, at the timing when the imaging element 21 takes a picture, even if the area of ​​the region of the food management label 4 in the recognized camera image and the area of ​​the region of the food 3 change, the remaining amount of the food 3 can be estimated.

[0066] Figure 3 This is a diagram showing a second usage example of the food management label 4 in the first embodiment. Figure 3 FIG. 2 shows an image obtained by photographing a storage container 5 with a food management label 4 attached thereto by the image pickup element 21. Figure 3 In the example shown, the food 3 is green beans, which are stored in a storage container 5. Figure 3 In the example shown, the food management label 4 is a note that imitates the shape of a cat and has "green beans" written on it. Figure 3 In the captured image shown in (a) of FIG. 1 , the food 3 is unused and stored in the first storage container 51. Figure 3 In the captured image shown in (b) of FIG. 1 , half of the food 3 is used due to cooking or the like, and is stored in the second storage container 52 which is half the size of the first storage container.

[0067] For example, it is assumed that when the user stores the storage container 5 (first storage container 51) in the refrigerator 2 for the first time, the acquisition unit 111 acquires Figure 3 In this case, the estimation unit 113 estimates that the remaining amount of food 3 is 0%. Figure 3The camera image shown in (a) is used to calculate the ratio of the area of ​​the recognized storage container 5 (first storage container 51) to the area of ​​the recognized food management label 4 as "β". This ratio is associated with the identifier of the food management label 4 and the food 3 with the food management label 4 and stored in the storage device 12.

[0068] Thus, for example, when the user takes out the storage container 5 (first storage container 51) with the same food management label 4 as described above from the refrigerator 2, and then moves the food 3 to another storage container 5 (second storage container 52) and stores the storage container 5 in the refrigerator 2, the acquisition unit 111 acquires Figure 3 In this case, the estimation unit 113 Figure 3 Based on the captured image shown in (b), the ratio of the area of ​​the recognized storage container 5 (second storage container 52) to the area of ​​the recognized food management label 4 is calculated as "β / 2". In this way, the estimation unit 113 estimates that the remaining amount of the food 3 is 50% by comparing it with the ratio "β" when the remaining amount of the food 3 read from the storage device 12 is 0%.

[0069] Here, the distance of the storage container 5 with the food management label 4 attached to the imaging element 21 may change every time the imaging element 21 takes an image. Therefore, the area of ​​the region of the food management label 4 in the recognized imaging image and the area of ​​the region of the storage container 5 may change every time the imaging element 21 takes an image. In the first embodiment, the estimation unit 113 estimates the remaining amount of the food 3 based on the ratio of the area of ​​the region of the storage container 5 in the recognized imaging image to the area of ​​the region of the recognized food management label 4, using the fact that the size of the food management label 4 does not change. Therefore, even if the area of ​​the region of the food management label 4 in the recognized imaging image and the area of ​​the region of the storage container 5 change every time the imaging element 21 takes an image, the remaining amount of the food 3 can be estimated.

[0070] In addition, it is considered that the storage container 5 is a transparent container and the food 3 stored in the storage container 5 can be photographed by the imaging element 21. In this case, the estimation unit 113 can estimate the remaining amount of the food 3 by comparing the size of the food management label 4 recognized by the recognition unit 112 and the size of the food 3 stored in the storage container 5 recognized by the recognition unit 112.

[0071] Furthermore, it is considered that the imaging element 21 cannot capture the food 3 stored in the storage container 5 when the lid of the storage container 5 is closed. In this case, if the imaging element 21 captures the interior of the storage container 5 when the lid of the storage container 5 is opened by the user, the estimation unit 113 can estimate the remaining amount of the food 3 by comparing the size of the food management label 4 recognized by the recognition unit 112 with the size of the food 3 stored in the storage container 5 recognized by the recognition unit 112.

[0072] The output unit 114 outputs the remaining amount information related to the remaining amount of the food 3 estimated by the estimation unit 113. The remaining amount information may include, for example, information showing the remaining amount of the food 3 when the food 3 is 100% when not used, and may also include information showing the used amount of the food 3. In addition to the information directly showing the remaining amount of the food 3, the remaining amount information may also include information that needs to be notified to the user, for example, whether the food 3 is still sufficiently remaining or only slightly remaining.

[0073] In Embodiment 1, the output unit 114 periodically outputs (sends) the remaining quantity information to the information terminal owned by the user, for example, via the communication I / F10. Furthermore, the output unit 114 outputs (sends) the remaining quantity information to the information terminal, for example, via the communication I / F10, when the communication I / F10 receives a request signal from the information terminal. The request signal is sent from the information terminal to the communication I / F10, for example, by the user performing a prescribed operation using the information terminal. In addition, in the case where the refrigerator 2 has a display, the output unit 114 may also output (send) the remaining quantity information to the refrigerator 2 via the communication I / F10. In this case, the processor 23 of the refrigerator 2 may also cause the display to display the remaining quantity information obtained via the communication I / F22.

[0074] The storage device 12 is a storage device that stores information (computer programs, etc.) required for various parts of the processor 11 to execute various functions. Although the storage device 12 is implemented by, for example, a semiconductor memory, there is no special limitation, and a known electronic information storage method can be used. The storage device 12 stores data such as the identifier of the food management label 4 and the association between the food 3 with the food management label 4.

[0075] [2. Work]

[0076] The following use Figure 4 The operation (food management method) of the food management system 100 configured as above will be described. Figure 41 is a flowchart showing an operation example of the food management system 100 in Embodiment 1. First, the acquisition unit 111 acquires a camera image obtained by photographing the food 3 with the food management label 4 or the storage container 5 with the food management label 4 by the imaging element 21 (S101). Here, the acquisition unit 111 acquires the camera image data transmitted from the communication I / F 22 of the refrigerator 2 via the communication I / F 10.

[0077] Next, the recognition unit 112 recognizes the food management label 4 included in the camera image based on the camera image obtained by the acquisition unit 111 (S102). Furthermore, the recognition unit 112 recognizes the food 3 or storage container 5 included in the camera image based on the obtained camera image (S103). The processes S102 and S103 are not limited to this order, and can be executed in the reverse order or in parallel.

[0078] Next, the estimation unit 113 estimates the remaining amount of the food 3 by comparing the size of the food management label 4 identified by the identification unit 112 with the size of the food 3 identified by the identification unit 112 or the size of the storage container 5 (S104). Furthermore, the output unit 114 outputs the remaining amount information related to the remaining amount of the food 3 estimated by the estimation unit 113 (S105).

[0079] [3. Advantages, etc.]

[0080] Hereinafter, advantages of the food management system 100 (food management method) in the first embodiment will be described.

[0081] As described above, the problem with the technology of identifying food by capturing an image including food and performing appropriate image recognition processing on the captured image is that it is difficult to determine whether the food before and after use is the same food, and it is difficult to estimate the remaining amount of food. There is also a problem that even if a method of attaching an RF tag to food and using RFID technology to identify food is used, the remaining amount of food cannot be grasped.

[0082] In contrast, in the food management system 100 (food management method) of the first embodiment, by identifying the food 3 or the storage container 5 with the food management label 4, it is possible to determine whether the food 3 before and after use is the same food. Furthermore, in the food management system 100 (food management method) of the first embodiment, the remaining amount of the food 3 can be estimated based on the size of the identified food management label 4 (particularly in the first embodiment, by comparing the size of the identified food management label 4 with the size of the identified food 3 or the size of the storage container 5).

[0083] Therefore, the advantage of the food management system 100 (food management method) of implementation mode 1 is that since the remaining amount of food 3 can be grasped by referring to the estimated remaining amount of food 3, it is easy to manage food 3 compared to the case where food 3 is only identified.

[0084] [4. Modification of Implementation Example 1]

[0085] Modifications of the first embodiment are described below by way of example.

[0086] Figure 5 1 is a block diagram showing the overall structure of a food management system 100A in a modified example of the first embodiment. The food management system 100A is composed of a server 1A. The structure of the server 1A is the same as that of the server 1. Figure 5 As shown, in this modification, the refrigerator 2 further includes a microphone 25. Furthermore, in the food management system 100A of this modification, the acquisition unit 111 further acquires the sound information related to the remaining amount of the food 3 collected by the microphone 25, which is different from the food management system 100 of the first embodiment. Thus, in the food management system 100A of this modification, the output unit 114 includes the information related to the remaining amount of the food 3 based on the sound information acquired by the acquisition unit 111 in the remaining amount information for output, which is different from the food management system 100 of the first embodiment.

[0087] The microphone 25 collects the sound made by the user, for example, when the user puts the food 3 or the storage container 5 into the refrigerator 2 (storage space) or when the user takes the food 3 or the storage container 5 out of the refrigerator 2. For example, when the user stores a cabbage as the food 3 into the refrigerator 2, the user makes a sound such as "1 cabbage". Accordingly, the acquisition unit 111 not only obtains the camera image of the cabbage (food 3) with the food management label 4, 4A, but also obtains the sound information such as "1 cabbage". In this way, in the storage device 12, the identifier of the food management label 4, 4A recognized by the recognition unit 112 and the identification data of the cabbage (food 3) recognized by the recognition unit 112 and the sound information such as "1 cabbage" are associated and stored.

[0088] In this way, the output unit 114 can output the remaining quantity information including the sound information. That is, the output unit 114 can output the remaining quantity information including the identification data of the cabbage (food 3) identified by the identification unit 112, the remaining quantity of the cabbage (food 3) estimated by the estimation unit 113, and the sound information of "1 cabbage".

[0089] In the remaining amount information, the sound information such as "1 cabbage" is data indicating the actual remaining amount of cabbage (food 3). Therefore, for example, when the estimation unit 113 estimates the remaining amount of food 3 using a learned model that has been machine-learned in a manner that uses a camera image of food 3 or a storage container 5 with a food management label 4, 4A as input and the remaining amount of food 3 as output, the remaining amount information can be used as learning data for re-learning the learned model. That is, this is because the sound information in the remaining amount information shows the actual remaining amount of food 3 and can therefore become correct answer data. Therefore, in the food management system 100A of this variant, there is an advantage in that it is easy to collect a data set for re-learning that uses the actual remaining amount of food 3 as correct answer data.

[0090] In addition, the learned model can also be machine-learned in a manner that further outputs the type of food 3. Even in this case, the remaining amount information can be used as learning data when relearning the learned model. That is, since the sound information in the remaining amount information also includes the type of food 3, it can become the correct answer data.

[0091] (Implementation Method 2)

[0092] [1. Composition]

[0093] Below, using Figure 6 The food management system 100B in the second embodiment will be described. Figure 6 1 is a block diagram showing the overall structure of a food management system 100B in Embodiment 2. The food management system 100B in Embodiment 2 is composed of a server 1B. The structure of the server 1B is the same as that of the server 1. Figure 6 As shown, in the food management system 100B of the second embodiment, it is assumed that the food management tag 4A is attached to the food 3 or the storage container 5. Thus, in the food management system 100B of the second embodiment, the estimation unit 113 estimates the remaining amount of the food 3 based on the size of the variable portion 42 in the food management tag 4A recognized by the recognition unit 112, which is different from the food management system 100 of the first embodiment.

[0094] That is, in the second embodiment, the estimation unit 113 estimates the remaining amount of the food 3 based only on the size of the food management label 4A (here, the size of the variable portion 42 ) without referring to the size of the food 3 or the size of the storage container 5 .

[0095] Figure 7 This is a diagram showing a usage example of the food management label 4A in the second embodiment. Figure 7FIG. 2 shows an image obtained by photographing the food 3 to which the food management label 4A is attached by the imaging element 21. Figure 7 In the example shown, the food 3 is cabbage. Figure 7 In the camera image shown in (a), food 3 is unused. Figure 7 In the camera image shown in (b), only half of the food 3 is used due to cooking or the like.

[0096] like Figure 7 As shown, the food management label 4A in the second embodiment is a rectangular note as a whole, and is composed of a constant part 41 and a variable part 42. The constant part 41 has, for example, a blank column for filling in a character string of the food 3, and its size is a constant part. The variable part 42 has a plurality of (here, 6) removable small pieces, and its size is a variable part. The variable part 42 is removed, for example, by the user tearing off the small piece according to the remaining amount of the food 3, so that its size changes.

[0097] That is, the food management label 4A has a variable portion 42 whose size can be changed according to the remaining amount of the food 3. Furthermore, the variable portion 42 is configured to be removable by the user according to the remaining amount of the food 3.

[0098] As described above, in the second embodiment, the estimation unit 113 can estimate the remaining amount of the food 3 based on the size of the variable portion 42 in the food management label 4A recognized by the recognition unit 112 .

[0099] Here, the processor 11 or the recognition unit 112 distinguishes and recognizes the invariable portion 41 and the variable portion 42 of the food management label 4A as follows.

[0100] As an example, a mark that can identify the invariable part 41 is attached to the invariable part 41, and a mark that can identify the removable small pieces is attached to each of the plurality of removable small pieces in the variable part 42. In this case, the processor 11 or the recognition unit 112 can recognize the invariable part 41 and the plurality of removable small pieces in the variable part 42 by performing appropriate image recognition processing on the image captured by the imaging element 21.

[0101] Furthermore, the processor 11 or the recognition unit 112 only needs to identify the number of small pieces of the variable part 42 that are not removed from the constant part 41 and remain or the area of ​​the small pieces of the remaining small pieces. In the former case, the processor 11 or the recognition unit 112 can also count the number of marks that can be determined as small pieces to identify the number of small pieces of the remaining small pieces that are not removed from the constant part 41. In the latter case, the processor 11 or the recognition unit 112 can also identify the area of ​​the small pieces of the remaining small pieces that are not removed from the constant part 41 based on the change in the area of ​​the entire identified small pieces. Further, it can also be that when each of the multiple small pieces of the variable part 42 that can be removed is given a characteristic of each small piece such as an inherent number, the processor 11 or the recognition unit 112 can identify which small piece of the constant part 41 is not removed and remains.

[0102] In addition, although the small piece of the variable part 42 is removed from the invariable part 41, it may be included in the image captured by the imaging element 21. In this case, the processor 11 or the recognition unit 112 recognizes the small piece based on the distance between the small piece and the invariable part 41, that is, if the small piece is not near the invariable part 41, it can be recognized that the small piece has been removed from the invariable part 41.

[0103] Furthermore, as another example, the same mark is printed on the entire constant portion 41 and the variable portion 42. In this case, the processor 11 or the recognition unit 112 can also recognize the amount of change in the variable portion 42 based on the change in the area of ​​the mark recognized based on the image captured by the imaging element 21. For example, it is assumed that the area occupied by the variable portion 42 is 10% of the total area of ​​the mark on the constant portion 41 and the variable portion 42, and this information is pre-stored in the memory. In this case, when the area of ​​the recognized mark decreases by 1%, the processor 11 or the recognition unit 112 can estimate that the area of ​​the variable portion 42 has decreased by 10%, in other words, 10% of the food 3 has been consumed.

[0104] Hereinafter, estimation of the remaining amount of food 3 by the estimation unit 113 will be described in detail.

[0105] For example, it is assumed that when the user first stores the food 3 in the refrigerator 2, the acquisition unit 111 acquires Figure 7 In this case, the estimation unit 113 estimates that the remaining amount of the food 3 is 100%. Figure 7The area of ​​the variable portion 42 in the region of the recognized food management label 4A is calculated as "γ" based on the camera image shown in (a). This area is associated with the identifier of the food management label 4A and the food 3 with the food management label 4A and stored in the storage device 12.

[0106] Thus, for example, when a user takes out a food 3 with the same food management label 4A as described above from the refrigerator 2 and then puts the food 3 back into the refrigerator 2, the obtaining unit 111 obtains Figure 7 Here, the variable portion 42 of the food management label 4A is manually removed by half according to the remaining amount of the food 3. The estimation unit 113 Figure 7 The area of ​​the variable portion 42 in the region of the recognized food management label 4A is calculated as "γ / 2" based on the camera image shown in (b). Thus, the estimation unit 113 estimates that the remaining amount of the food 3 is 50% by comparing it with the area "γ" when the remaining amount of the food 3 read from the storage device 12 is 100%.

[0107] Furthermore, in the second embodiment, the food 3 may also be installed Figure 8 The food management label 4B shown replaces the food management label 4A. Figure 8 This is a diagram showing an example of a food management label 4B in the second embodiment. Figure 8 FIG. 2 shows an image obtained by photographing the food 3 to which the food management label 4B is attached by the imaging element 21. Figure 8 In the example shown, the food 3 is cabbage, and the food management label 4B is the packaging of the cabbage. Here, the packaging is partially removed or partially folded by the user according to the remaining amount of the food 3. Therefore, the food management label 4B becomes a variable part of the whole. Figure 8 Although not shown in the figure, the food management label 4B may also have multiple character strings showing the name of the food 3 as the identifier, and may have a pattern different from other food management labels 4B. In addition, the food management label 4B may also have a one-dimensional code such as a barcode, a two-dimensional code such as a QR code (registered trademark), or a color code as the identifier. These codes may also be written on a piece of paper with an adhesive portion, and attached to the food management label 4B by pasting the piece of paper to the food management label 4B. Figure 8 In the camera image shown in (a), food 3 is unused. Figure 8 In the camera image shown in (b), only half of the food 3 is used due to cooking or the like.

[0108] For example, when the user first stores food 3 in refrigerator 2, acquisition unit 111 acquires Figure 8In this case, the estimation unit 113 estimates that the remaining amount of the food 3 is 100%. Figure 8 The area of ​​the region (i.e., the variable portion) of the recognized food management label 4B is calculated as "δ". This area is associated with the identifier of the food management label 4B and the food 3 with the food management label 4B and stored in the storage device 12.

[0109] Thus, for example, when a user takes out a food 3 with the same food management label 4B as described above from the refrigerator 2 and then puts the food 3 back into the refrigerator 2, the obtaining unit 111 obtains Figure 8 Here, the food management label 4B is manually removed in half or folded in half according to the remaining amount of the food 3 by the user. The estimation unit 113 Figure 8 The area of ​​the region (i.e., the variable portion) of the recognized food management label 4B is calculated as "δ / 2" from the camera image shown in (b). Thus, the estimation unit 113 estimates that the remaining amount of the food 3 is 50% by comparing it with the area "δ" when the remaining amount of the food 3 read from the storage device 12 is 100%.

[0110] [2. Work]

[0111] The following use Fig. 9 The operation (food management method) of the food management system 100B configured as above will be described. Fig. 9 1 is a flowchart showing an operation example of the food management system 100B in Embodiment 2. First, the acquisition unit 111 acquires a camera image obtained by photographing the food 3 with the food management label 4A, 4B or the storage container 5 with the food management label 4A, 4B by the imaging element 21 (S201). Here, the acquisition unit 111 acquires the camera image data transmitted from the communication I / F 22 of the refrigerator 2 via the communication I / F 10.

[0112] Next, the recognition unit 112 recognizes the food management labels 4A and 4B included in the camera image based on the camera image obtained by the acquisition unit 111 (S202). Furthermore, the recognition unit 112 recognizes the food 3 or storage container 5 included in the camera image based on the obtained camera image (S203). The processes S202 and S203 are not limited to this order and can be executed in the reverse order or in parallel.

[0113] Next, the estimation unit 113 estimates the remaining amount of the food 3 based on the size of the variable portion 42 in the food management label 4A, 4B recognized by the recognition unit 112 (S204). And, the output unit 114 outputs the remaining amount information related to the remaining amount of the food 3 estimated by the estimation unit 113 (S205).

[0114] [3. Advantages, etc.]

[0115] The advantages of the food management system 100B (food management method) in Embodiment 2 are described below. In the food management system 100B (food management method) in Embodiment 2, as described above, the remaining amount of food 3 is estimated based only on the size of the food management label 4A (here, the size of the variable portion 42) without referring to the size of the food 3 or the size of the storage container 5. Therefore, the food management system 100B (food management method) in Embodiment 2 has the advantage that the remaining amount of food 3 can be estimated as long as the size of the variable portion 42 of the food management label 4A, 4B can be identified, without having to identify the size of the food 3 or the size of the storage container 5, and thus the processing load can be reduced.

[0116] [4. Modification of Implementation Example 2]

[0117] Modifications of the second embodiment are described below by way of example.

[0118] In the second embodiment, although the food management label 4B is a package that wraps the food 3, it is not limited to this. For example, the food management label may also be an edible ink. In this case, the food management label can be directly printed on the food 3. Thus, in this case, the food management label is the same as when it is a package. Since the size changes according to the remaining amount of the food 3, the remaining amount of the food 3 can be estimated by referring to the size of the food management label (variable part) by the estimation unit 113.

[0119] Furthermore, the food management label may be made of a more transparent ink. In this case, since the food management label itself has no design characteristics, there is no need to worry about the design changes caused by printing the food management label on the food 3, and the food management label can be printed on the entire food 3. Transparent ink, for example, can be made of a material that reacts to light of a specific wavelength other than visible light such as infrared light. That is, in the case where the food management label is made of transparent ink, even if visible light is incident, it is hardly reflected, so it is difficult for the human eye to recognize the food management label. However, when the food management label is incident with light of a specific wavelength such as infrared light, the light is reflected, so the food management label can be photographed by receiving the reflection of the light through the imaging element 21. In this case, the imaging element 21 is configured to not have a filter that blocks light of a specific wavelength.

[0120] In the second embodiment, the estimation unit 113 may also perform the estimation process performed by the estimation unit 113 in the first embodiment, i.e., the process of estimating the remaining amount of the food 3 by comparing the size of the food management label 4A (here, the invariant portion 41) with the size of the food 3 or the size of the storage container 5, together with the above-mentioned estimation process. In this case, the estimation unit 113 may also adopt the average value of the results of the two estimation processes as the remaining amount of the food 3.

[0121] (Variation Example)

[0122] As described above, the embodiments are described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited thereto, and embodiments with appropriate changes, replacements, additions, omissions, etc. can also be applied. Furthermore, the various components described in the above embodiments can be combined to form new embodiments.

[0123] Therefore, modifications common to each embodiment are described below by way of example.

[0124] In each of the above-mentioned embodiments, the food management labels 4, 4A, 4B may also be formed of a material that reflects light of a specific wavelength. Here, the light of a specific wavelength refers to, for example, illumination light or near-infrared light irradiated into the storage of the refrigerator 2. In this case, for example, when the imaging element 21 disposed in the storage of the refrigerator 2 is used for photographing, since the food management labels 4, 4A, 4B reflect the light of a specific wavelength, the imaging element 21 can easily photograph. That is, in this case, there is an advantage that the imaging element 21 can clearly photograph the food management labels 4, 4A, 4B irradiated with light of a specific wavelength.

[0125] In the above-mentioned embodiments, although the food 3 or the storage container 5 is stored in the refrigerator 2, the present invention is not limited thereto. For example, the food 3 or the storage container 5 may be stored in a storage space other than the refrigerator 2, such as a food storage room or the like. In this case, the imaging element 21 and the communication I / F 22 may also be provided in the storage space.

[0126] In the above-mentioned embodiments, although the imaging element 21 captures the food 3 or the storage container 5 when the food 3 or the storage container 5 is placed in the refrigerator 2 or taken out from the refrigerator 2, the present invention is not limited thereto. For example, the imaging element 21 may capture the interior of the refrigerator 2 periodically without depending on the opening and closing of the door of the refrigerator 2.

[0127] In the above-mentioned embodiments, although the camera image taken by the refrigerator 2 is sent to the food management system 100, 100A, 100B, it is not limited to this. For example, the camera image taken by an information terminal such as a smart phone owned by the user may be sent to the food management system 100, 100A, 100B. In this case, the information terminal may also have an imaging element 21 and a communication I / F 22. And, in this case, a series of processes such as estimating the remaining amount of food 3 by the food management system 100, 100A, 100B are performed, for example, when the user operates the information terminal and sends the camera image to the food management system 100, 100A, 100B at a fixed time.

[0128] The food management labels 4, 4A, 4B in the above-mentioned embodiments can be distributed in the market independently. That is, the food management labels 4, 4A, 4B are not attached to the food 3 or the storage container 5, but can be arbitrarily attached to the food 3 or the storage container 5 by the user.

[0129] And for example, in each of the above embodiments, although the food management systems 100, 100A, and 100B are implemented by servers 1, 1A, and 1B, respectively, this is not limited to this. For example, the food management systems 100, 110A, and 100B can be implemented by information terminals such as smart phones owned by users. In this case, the communication I / F 10, processor 11, and storage device 12 of the food management systems 100, 100A, and 100B are implemented by the communication I / F, processor, and storage device possessed by the information terminal, respectively.

[0130] And for example, in each of the above-mentioned embodiments, although the food management system 100, 100A, 100B is implemented as a single device, it can also be implemented by multiple devices. In the case where the food management system 100, 100A, 100B is implemented by multiple devices, the components possessed by the food management system 100, 100A, 100B can also be arbitrarily distributed to multiple devices. In other words, the present disclosure can be implemented by cloud computing or by edge computing. As an example, the food management system 100, 100A, 100B can also be set in a storage space for storing food 3 or a storage container 5 such as a refrigerator 2.

[0131] And for example, in each of the above-mentioned embodiments, all or part of the constituent elements of the food management system 100, 100A, 100B in the present disclosure may be constituted by dedicated hardware or implemented by executing a software program suitable for each constituent element. Each constituent element may also be implemented by a program execution unit such as a CPU (Central Processing Unit) or a processor reading out and executing a software program recorded on a recording medium such as a (Hard Disk Drive: mechanical hard disk) or a semiconductor memory.

[0132] Furthermore, the components of the food management system 100, 100A, 100B in the present disclosure may be composed of one or more electronic circuits, each of which may be a general-purpose circuit or a dedicated circuit.

[0133] One or more electronic circuits may include, for example, semiconductor devices, ICs (Integrated Circuits) or LSIs (Large Scale Integrations). ICs or LSIs may be integrated into one chip or multiple chips. Although they are referred to as ICs or LSIs here, they are called differently depending on the degree of integration, and may also be called system LSIs, VLSIs (Very Large Scale Integrations), or ULSIs (Ultra Large Scale Integrations). In addition, FPGAs (Field Programmable Gate Arrays) that can be programmed after LSIs are manufactured can also be used for the same purpose.

[0134] Furthermore, the general or specific aspects of the present disclosure may also be implemented by a system, device, method, integrated circuit or computer program. Alternatively, it may also be implemented by a computer-readable non-temporary recording medium such as an optical disc, HDD or semiconductor memory storing the computer program. The present disclosure may also be implemented, for example, by a program that executes the notification method in the above-mentioned embodiment by a computer. Furthermore, the program may be recorded on a computer-readable non-temporary recording medium such as a CD-ROM, or may be distributed through channels such as the Internet.

[0135] As described above, each embodiment as a technical example of the present disclosure has been described, and the drawings and detailed descriptions have been provided for this purpose.

[0136] Therefore, the components recorded in the drawings and detailed descriptions include not only the components necessary for solving the problems, but also components that are not necessary for solving the problems in order to illustrate the above-mentioned technology. Therefore, these non-essential components should not be immediately regarded as essential because they are recorded in the drawings or detailed descriptions.

[0137] Furthermore, since the above-described embodiments are examples for illustrating the technology of the present disclosure, various changes, substitutions, additions, omissions, etc. can be made within the scope of the claims or the scope of the equivalents.

[0138] (Summarize)

[0139] As described above, in the food management method of the embodiment, a camera image is obtained by photographing the food 3 with the food management label 4, 4A, 4B having an identifier associated with the food 3 or the storage container 5 for storing the food 3 with the food management label 4, 4A, 4B by the camera element 21 (S101, S201). And, in the food management method, the food 3 with the food management label 4, 4A, 4B or the storage container 5 is identified based on the obtained camera image (S102, S103, S202, S203). And, in the food management method, the remaining amount of the food 3 is estimated based on the size of the identified food management label 4, 4A, 4B (S104, S204). And, in the food management method, the remaining amount information related to the estimated remaining amount of the food 3 is output (S105, S205).

[0140] The above method has the advantage that the remaining amount of food 3 can be grasped by referring to the estimated remaining amount of food 3, so that the food 3 can be managed more easily than when the food 3 is simply identified.

[0141] For example, in the food management method, the remaining amount of the food 3 is estimated by comparing the size of the recognized food management tag 4, 4A with the size of the recognized food 3 or the size of the storage container 5.

[0142] The above method has the advantage that the estimation accuracy of the remaining amount of food 3 can be easily improved compared with the case where only the size of food 3 is referred to.

[0143] And for example, in the food management method, the type of food 3 is further identified based on the obtained camera image.

[0144] The above method has the advantage that not only the remaining amount of food 3 but also the type of food 3 can be managed, so the food 3 can be managed easily and in more detail.

[0145] For example, the food management tags 4A and 4B have a variable portion 42 whose size can be changed according to the remaining amount of the food 3. In the food management method, the remaining amount of the food 3 is estimated based on the size of the variable portion 42 in the recognized food management tags 4A and 4B.

[0146] The above method has the advantage that there is no need to identify the size of the food 3 or the size of the storage container 5. Instead, the remaining amount of the food 3 can be estimated by simply identifying the size of the variable portion 42 of the food management label 4A, 4B, thereby reducing the processing load.

[0147] For example, in the food management method, the sound information about the remaining amount of food 3 collected by the microphone 25 is further obtained. In the food management method, information about the remaining amount of food 3 based on the obtained sound information is included in the remaining amount information and output.

[0148] The above method has the advantage that, for example, when the estimation model for estimating the remaining amount of food 3 is re-learned through machine learning, it is easy to collect a data set for re-learning that uses the actual remaining amount of food 3 as correct answer data.

[0149] For example, the image pickup device 21 captures the image when the user places the food 3 or the storage container 5 into the storage space (refrigerator 2) or when the user takes the food 3 or the storage container 5 out of the storage space.

[0150] The above-described method has the advantage that the remaining amount of food 3 can be estimated at a timing when there is a high possibility that food 3 will be used, and thus the remaining amount of food 3 can be managed easily and accurately.

[0151] Furthermore, for example, the food management labels 4, 4A, and 4B are formed of a material that reflects light of a specific wavelength.

[0152] The above arrangement has the advantage that the imaging element 21 can clearly capture the food management labels 4 , 4A, and 4B in a state where they are irradiated with light having a specific wavelength.

[0153] Furthermore, the program in the embodiment causes one or more processors to execute the above-mentioned food management method.

[0154] The above method has the advantage that the remaining amount of food 3 can be grasped by referring to the estimated remaining amount of food 3, so that the food 3 can be managed more easily than when the food 3 is simply identified.

[0155] Furthermore, the food management system 100, 100A, 100B in the embodiment includes an acquisition unit 111, a recognition unit 112, an estimation unit 113, and an output unit 114. The acquisition unit 111 acquires a camera image obtained by capturing a food 3 having a food management label 4, 4A, 4B having an identifier associated with the food 3 or a storage container 5 for storing the food 3 having the food management label 4, 4A, 4B attached thereto by the imaging element 21. The recognition unit 112 recognizes the food 3 or the storage container 5 having the food management label 4, 4A, 4B attached thereto based on the camera image acquired by the acquisition unit 111. The estimation unit 113 estimates the remaining amount of the food 3 based on the size of the food management label 4, 4A, 4B recognized by the recognition unit 112. The output unit 114 outputs the remaining amount information related to the remaining amount of the food 3 estimated by the estimation unit 113.

[0156] The above method has the advantage that the remaining amount of food 3 can be grasped by referring to the estimated remaining amount of food 3, so that the food 3 can be managed more easily than when the food 3 is simply identified.

[0157] Furthermore, the food management labels 4A and 4B in the embodiment are food management labels that are mounted on the food 3 or the storage container 5 storing the food 3 and have an identifier associated with the food 3 , and have a variable portion 42 that can change in size according to the remaining amount of the food 3 .

[0158] The above method has the advantage that since the remaining amount of food 3 can be known by attaching food management labels 4A, 4B to food 3 or storage container 5, it is easier to manage food 3 than when a label simply for identifying food 3 is attached to food 3 or storage container 5.

[0159] Furthermore, for example, the variable portion 42 is configured to be removable by the user according to the remaining amount of the food 3 .

[0160] The above method has the advantage that since the user can change the size of the variable portion 42 according to the remaining amount of food 3, it is easy to improve the estimation accuracy of the remaining amount of food 3 based on the size of the variable portion 42.

[0161] Industrial Applicability

[0162] The present disclosure can be applied to a system for managing food, etc.

[0163] Explanation of symbols

[0164] 1. 1A, 1B servers

[0165] 100, 100A, 100B Food Management System

[0166] 10Communication I / F

[0167] 11 Processor

[0168] 111 Acquisition Department

[0169] 112 Identification Department

[0170] 113 Estimation Department

[0171] 114 Output

[0172] 12 Storage Devices

[0173] 2 Refrigerator (storage space)

[0174] 21 Camera Components

[0175] 22 Communication I / F

[0176] 23 Processor

[0177] 24 Storage Devices

[0178] 25 Microphone

[0179] 3 Food

[0180] 4. 4A, 4B Food Management Labels

[0181] 41 unchanged parts

[0182] 42 Variable parts

[0183] 5 Storage Container

[0184] 51 First Storage Container

[0185] 52 Second storage container

[0186] NT1 External Network

Claims

1. A food management method, in which: obtaining a camera image, wherein the camera image is an image obtained by photographing a food with a food management label or a storage container with the food management label for storing the food, wherein the food management label is a label having an identifier associated with the food, Based on the acquired camera image, the food or the storage container with the food management label is identified, estimating the remaining amount of the food according to the size of the identified food management label, Remaining amount information related to the estimated remaining amount of the food is output.

2. The food management method according to claim 1, In the food management method, The remaining amount of the food is estimated by comparing the recognized size of the food management label with the recognized size of the food or the size of the storage container.

3. The food management method according to claim 2, In the food management method, The type of the food is further identified based on the obtained camera image.

4. The food management method according to any one of claims 1 to 3, The food management label has a variable portion, and the size of the variable portion can be changed according to the remaining amount of the food. The remaining amount of the food is estimated based on the size of the variable portion in the identified food management tag.

5. The food management method according to any one of claims 1 to 3, In the food management method, further obtaining sound information related to the remaining amount of the food collected by the microphone, Information on the remaining amount of the food based on the obtained sound information is included in the remaining amount information and outputted.

6. The food management method according to any one of claims 1 to 3, The image pickup device captures the food or the storage container at a timing when a user puts the food or the storage container into a storage space or when a user takes the food or the storage container out of a storage space.

7. The food management method according to any one of claims 1 to 3, The food management label is formed of a material that reflects light of a specific wavelength.

8. A program for causing one or more processors to execute the food management method according to any one of claims 1 to 3.

9. A food management system comprising: an acquisition unit that acquires a camera image, wherein the camera image is an image obtained by photographing a food with a food management label or a storage container with the food management label for storing the food by an imaging element, wherein the food management label is a label having an identifier associated with the food; a recognition unit that recognizes the food or the storage container to which the food management label is attached based on the camera image obtained by the acquisition unit; an estimating unit that estimates the remaining amount of the food based on the size of the food management label identified by the identifying unit; as well as An output unit outputs remaining amount information related to the remaining amount of the food estimated by the estimation unit.

10. A food management label, which is attached to a food or a storage container for storing the food and has an identifier associated with the food. The food management label has a variable portion, and the size of the variable portion can be changed according to the remaining amount of the food.

11. The food management label according to claim 10, The variable portion is configured to be removable by a user according to the remaining amount of the food.

Images