Food management device, food management system, food management method, food management program, and recording medium

Abstract

Manage multiple food items of the same type individually. [Solution] A food management device (MA) comprising: a location identification unit (14) that identifies the location of food items shown in a first image inside the storage room; an identifier assignment unit (21) that assigns an identifier to each food item; a type determination unit (15) that determines the type of food item; and a recording unit (30) that records the identifier, location, and type of each food item as inventory information.

Description

Technical Field

[0001] The present disclosure relates to a food management apparatus, a food management system, a food management method, a food management program, and a recording medium.

Background Art

[0002] The food management system disclosed in Patent Document 1 includes: a refrigerator compartment camera that captures an image of an opening of a refrigerator compartment of a refrigerator; a drawer camera that captures an image of a drawer pulled out from a storage compartment other than the refrigerator compartment; and a food detection unit that detects food entering and exiting the refrigerator based on imaging results of the refrigerator compartment camera and the drawer camera.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] However, merely detecting the entry and exit of food makes it difficult to individually manage a plurality of foods of the same type when a plurality of foods of the same type are stored in a refrigerator.

[0005] In order to improve the accuracy of food management, there is a demand for individually managing each food item in a refrigerator.

Means for Solving the Problem

[0006] To solve the above problems, a food management device according to one aspect of the present disclosure comprises: a location identification unit that identifies the location of at least one food item in the storage compartment of a refrigerator by analyzing a first image of the storage compartment of the refrigerator; an identifier assignment unit that assigns an identifier to each of the at least one food item; a type determination unit that determines the type of food item for each of the at least one food item by analyzing a second image of the storage compartment in which the food item is shown; and a recording unit that records the identifier, location, and type of each of the at least one food item as inventory information.

[0007] To solve the above problems, a food management system according to one aspect of this disclosure has a configuration that includes a food management device according to one aspect of this disclosure and the refrigerator.

[0008] To solve the above problems, a food management method according to one aspect of the present disclosure includes a location identification step of identifying the location of at least one food item in the storage compartment of a refrigerator by analyzing a first image of the storage compartment of the refrigerator; an identifier assignment step of assigning an identifier to each of the at least one food item; a type determination step of determining the type of food item for each of the at least one food item by analyzing a second image of the storage compartment of the refrigerator in which the food item is shown; and a recording step of recording the identifier, location, and type of each of the at least one food item as inventory information.

[0009] To solve the above problems, the food management program according to one aspect of the present disclosure is a food management program for causing a computer to function as a food management device according to one aspect of the present disclosure, and is a program for causing the computer to function as the location identification unit, the identifier assignment unit, the type determination unit, and the recording unit.

[0010] To address the above issues, a recording medium according to one aspect of this disclosure is a computer-readable recording medium on which a food management program according to one aspect of this disclosure is recorded. [Effects of the Invention]

[0011] According to one aspect of this disclosure, multiple food items of the same type can be managed individually within a refrigerator. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic diagram showing an example of the configuration of a refrigerator incorporating a food management device according to one embodiment of the present disclosure. [Figure 2] Figure 1 is a functional block diagram showing an example configuration of the food management device. [Figure 3] Figure 2 is a flowchart illustrating an example of a food management method performed by the food management device shown. [Figure 4] Figure 2 is a flowchart illustrating an example of an expiration date setting method performed by the expiration date setting unit of the food management device. [Figure 5] Figure 2 is a flowchart illustrating an example of a food management method performed by the food management device shown. [Figure 6] Figure 5 is a schematic diagram illustrating an example of visual warning data in the food management method shown. [Figure 7] Figure 2 is a flowchart illustrating another example of a food management method performed by the food management device shown. [Figure 8] Figure 2 is a flowchart illustrating an example of a food management method performed by the food management device shown. [Figure 9] Figure 8 is a schematic diagram showing an example of a screen displayed in the user interface of the food management method. [Figure 10] This is a schematic diagram showing an example of the first data indicating location in the food management method shown in Figure 8. [Figure 11] Figure 2 is a flowchart illustrating another example of a food management method performed by the food management device shown. [Figure 12] Figure 11 is a schematic diagram illustrating an example of a food management method, showing a first data point indicating location and a second data point indicating expiration date and remaining quantity. [Figure 13]It is a schematic diagram showing a configuration example of a food management system including a food management device according to an embodiment of the present disclosure. MODE FOR CARRYING OUT THE INVENTION

[0013] Embodiment 1 (Configuration of Refrigerator) Figure 1 is a schematic diagram showing a configuration example of a refrigerator incorporating a food management device according to an embodiment of the present disclosure. As shown in Figure 1, the refrigerator R1 includes a storage compartment CC for storing food FD, a camera CM for capturing images of the inside of the storage compartment CC, and the food management device MA according to the present embodiment 1. The refrigerator R1 may include one or more storage compartments CC, and may include one or more cameras CM for each storage compartment CC. The refrigerator R1 may optionally include a storage compartment without a camera. For simplification, the description will focus on one storage compartment CC and the camera CM corresponding to the storage compartment CC. Descriptions of cases where the refrigerator R1 is empty and cases where non-food objects are stored in the refrigerator R1 are omitted. Therefore, in the following description, an object captured in an image taken by the camera CM is food FD.

[0014] (Configuration of Food Management Device) Figure 2 is a functional block diagram showing a configuration example of the food management device shown in Figure 1. As shown in Figure 2, the food management device MA is connected to the camera CM, and includes a control device CU and at least one storage device ST1, ST2. The control device CU includes: (i) a position specifying unit 14 that specifies the position of at least one food FD captured in a first image within the storage compartment CC by analyzing the first image of the inside of the storage compartment CC of the refrigerator R1; (ii) an identifier assigning unit 21 that assigns an identifier to each of the at least one food FD, the identifier being assigned to the food FD or the position of the food FD; (iii) a type determining unit 15 that determines the type of the food FD for each of the at least one food FD by analyzing a second image of the food FD captured inside the storage compartment CC; and (iv) a recording unit 30 that associates the identifier, the position, and the type of each of the at least one food FD as inventory information and records the inventory information in the storage device ST1.

[0015] According to the above configuration, for each food item FD, the identifier, position, and type are associated with each other and recorded as inventory information. Since the positions of the food items FD are individually recorded, the data of the food items FD in the inventory information can be individually associated with the actual food items FD. Therefore, even when a plurality of food items FD of the same type are stored in the refrigerator R1, the food items FD in the refrigerator R1 can be individually managed.

[0016] The position specifying unit 14 may specify the position of the food item FD based on a learning result obtained by machine learning. The type determining unit 15 may determine the type of the food item FD based on a learning result obtained by machine learning. Each of these learning results may be stored in the same storage device ST1 as the inventory information, or may be stored in a storage device ST2 different from the inventory information.

[0017] The control device CU may further include an expiration date setting unit 22 that sets an expiration date for the food item FD for each of at least one food item FD. The recording unit 30 may, for each of at least one food item FD, associate the identifier, position, type, and expiration date with each other and record the associated information as inventory information in the storage device ST1. The expiration date of the food item FD may be a best-before date, a use-by date, or any other expiration date. The expiration date setting unit 22 can reduce the labor required for a user to set an expiration date.

[0018] The expiration date setting unit 22 sets the expiration date of the food item FD based on the type of the food item FD and the current time only for a food item FD whose position is not recorded in existing inventory information at the current time. For a food item FD whose position is recorded in existing inventory information, the same expiration date as that recorded in the existing inventory information is set. With the above arrangement, an expiration date can be newly set for a food item FD newly stored in the refrigerator R1, and the existing expiration date can be succeeded to for a food item FD that has been stored in the refrigerator R1 from before. The term "current time" as used herein refers to the time point when the expiration date setting unit 22 sets the expiration date, the time point when the camera CM captures a first image, or the time point corresponding to a series of processes from capturing the first image to recording the inventory information.

[0019] The control device CU may further include an expiration date management unit 41 that manages the expiration date of at least one food FD based on inventory information. The expiration date management unit 41, for example, determines whether there is a food FD among at least one food FD whose expiration date is less than or equal to a threshold value at the current time. Such an expiration date management unit 41 can find food FDs that are nearing their expiration date. The expiration date management unit 41, for example, determines whether there is a food FD among at least one food FD whose expiration date is in the past than the time when a threshold has elapsed from the current time. Such an expiration date management unit 41 can find food FDs that are nearing their expiration date or are old. Here, "current time" refers to the time when the expiration date management unit 41 manages the expiration date. The expiration date management unit 41 may also manage the expiration date based on a specified time instead of the current time, for example, the date on which the user's vacation ends.

[0020] The control device CU may further include a remaining quantity calculation unit 23 that calculates the remaining quantity of at least one food FD in the storage chamber CC for each type of food FD. This allows for both individual management of food FDs and management of food FDs by type. The recording unit 30 may also record the remaining quantity as inventory information for each type of food FD. In addition / or, the remaining quantity may be calculated and recorded for each combination of food FD type and expiration date. Alternatively, the remaining quantity may not be recorded, and the remaining quantity calculation unit 23 may calculate the remaining quantity based on inventory information as needed.

[0021] The control device CU may further include (i) a receiving unit 42 that receives designation information that specifies at least one type of food FD from at least one type of food FD based on inventory information, and (ii) an output generation unit 43 that generates first data DT1 (see Figures 10 and 12) indicating the location of the food FD corresponding to the at least one type specified by the designation information, based on inventory information. The refrigerator R1 may be equipped with a user interface UI, which may be a combination of a touch panel display, a speaker and a microphone, or a combination of physical buttons and a display. The user can input designation information to the food management device MA through the user interface UI, and the food management device MA can provide the first data DT1 to the user through the user interface UI.

[0022] The control device CU may further include a remaining quantity calculation unit 23 that calculates the remaining quantity of at least one food FD in the storage chamber CC for each combination of food FD type and expiration date, and the output generation unit 43 may generate, in addition to the first data, second data DT2 (see Figure 12) that indicates the expiration date and remaining quantity of food FD corresponding to at least one type specified by the specified information. The second data DT2 may also be provided to the user through the interface UI. The remaining quantity indicated by the second data DT2 is the remaining quantity calculated for each combination of food FD type and expiration date.

[0023] The control device CU may include (i) an extraction unit 11 that extracts a valid frame from a moving or still image captured by the camera CM as a first image, (ii) an extraction unit 12 that extracts the range in which each food FD is shown from the first image as a second image, (iii) an object detection unit 13 that detects objects from the first image, and (iv) an image synthesis unit 44 that either combines the first data with the first image or combines the first data and the second data with the first image.

[0024] (Food management method) Figure 3 is a flowchart illustrating an example of a food management method performed by the food management device shown in Figure 2. Refrigerator R1 uses camera CM to photograph the inside of its storage compartment CC and sends the captured data to the food management device MA. This photography may be performed periodically, when the refrigerator R1's opening / closing sensor detects the opening of the storage compartment CC, or when requested by the user through the user interface UI. This photography may capture still images or videos.

[0025] As shown in Figure 3, the food management device MA determines whether or not it has received the captured data (step S10). If it has received the data, the extraction unit 11 of the food management device MA extracts a valid frame from the captured data as the first image (step S20), and the object detection unit 13 of the food management device MA detects an object from the first image (step S30). The detected object is food FD. The extraction unit 11 may extract the first image based on the learning results from machine learning. The object detection unit 13 may detect food FD from the first image based on the learning results from machine learning. These learning results may be stored in the same storage device ST1 as the inventory information, or in a different storage device ST2 than the inventory information.

[0026] Next, the location identification unit 14 of the food management device MA identifies the location of at least one food FD in the storage chamber CC by analyzing the first image (step S40, location identification step). The identifier assignment unit 21 of the food management device MA assigns an identifier to at least one food FD (step S50, identifier assignment step), and the type determination unit 15 of the product management device MA determines the type of food FD for at least one food FD by analyzing the second image of the storage chamber CC in which the food FD is visible (step S70, type determination step). The cropping unit 12 of the food management device MA crops the area in which the food FD is visible from the first image as the second image (step S60), and this second image may be used in step S70. The cropping unit 12 may crop the second image from the first image based on the learning results from machine learning. These learning results may be stored in the same storage device ST1 as the inventory information, or in a different storage device ST2.

[0027] Then, the recording unit 30 of the food management device MA records, for at least one food FD, the identifier assigned in step S50, the location identified in step S40, and the type determined in step S70 as inventory information (step S100, recording step). Steps S40, S50, S70, and S100 are performed for all food FDs detected in step S30.

[0028] Optionally, the expiration date setting unit 22 of the food management device MA may set an expiration date for at least one food FD (step S80, expiration date setting step). In this case, in step S100, the recording unit 30 of the food management device MA may record the expiration date set in step S80 in addition to the identifier, location, and type for at least one food FD as inventory information.

[0029] Figure 4 is a flowchart showing an example of an expiration date setting method executed by the expiration date setting unit of the food management device shown in Figure 2. As shown in Figure 4, the expiration date setting unit 22 determines whether an expiration date has been set for all food FDs detected in step S30 (step S81). If there are food FDs with unset expiration dates remaining (No in step S81), it selects one from the unset food FDs (step S82). Then, it compares the old inventory information read from the storage device ST1 with the selected food FD to determine whether the location of the selected food FD is recorded in the old inventory information (step S83). If it is recorded in the old inventory information (Yes in step S83), it sets the expiration date of the selected food FD so that the selected food FD inherits the expiration date of the selected food FD recorded in the old inventory information (step S84). If it is not recorded in the old inventory information (No in step S83), it sets the expiration date of the selected food FD based on the type of food FD and the current time (step S85). Here, "the present time" refers to the time when the deadline setting unit 22 sets the deadline, the time when the camera CM captures the first image, or the time corresponding to the series of processes from capturing the first image to recording inventory information.

[0030] As shown in Figure 3, optionally, the remaining quantity calculation unit 23 of the food management device MA may calculate the remaining quantity of at least one food FD in the storage chamber CC for each type of food FD (step S90, remaining quantity setting step). In this case, in step S100, the recording unit 30 of the food management device MA may associate the remaining quantity calculated in step S90 for each type of food FD and record it as part of the inventory information. Alternatively, the remaining quantity calculation unit 23 may calculate the remaining quantity for each combination of food FD type and expiration date, and the recording unit 30 may record it.

[0031] Optionally, in step S100, the recording unit 30 of the food management device MA may record the first image as part of the inventory information.

[0032] Figure 5 is a flowchart showing an example of a food management method performed by the food management device shown in Figure 2. As shown in Figure 5, the expiration date management unit 41 of the food management device MA reads inventory information from the storage device ST1 (step S200) and, based on the inventory information, determines whether or not there is at least one food FD that is nearing its expiration date (step S210, first expiration date management step). Specifically, it determines whether or not there is a food FD whose absolute difference between the current date and the expiration date is less than or equal to the threshold, using a default threshold or a threshold specified by the user. This determination may be performed periodically, when the opening / closing sensor of the refrigerator R1 detects the opening of the storage compartment CC, or when requested by the user through the user interface UI.

[0033] In step S210, "the present moment" refers to the time when the expiration date management unit 41 executes step S210, or the time corresponding to a series of processes including step S210. For example, if step S210 is performed when the opening / closing sensor of the refrigerator R1 detects the opening of the storage compartment CC, it may be the time when the opening / closing sensor detects the opening of the storage compartment CC.

[0034] The food management device MA may notify the user of the result of step S210 through the user interface UI, thereby supporting the user in managing the expiration dates of food FDs. For example, if there are food FDs that are nearing their expiration date, the food management device MA generates warning data (step S220) and instructs the user interface UI to execute this warning data (step S220).

[0035] Figure 6 is a schematic diagram showing an example of visual warning data in the food management method shown in Figure 5. Figure 6 shows an example where eggplant FD1, with an expiration date of June 16, 20XX, is a food FD with an approaching expiration date, and cabbage FD2 is a food FD with a later expiration date. In this case, the output generation unit 43 of the food management device MA generates first warning data AL1 indicating the location of eggplant FD1 and second warning data AL2 indicating the expiration date of eggplant FD1. Then, the image synthesis unit 44 of the food management device MA reads the first image IM1 from the storage device ST1 and synthesizes the first warning data AL1 and the second warning data AL2 with the first image IM1 to generate synthesized warning data ALC. The food management device MA commands the user interface UI to display the synthesized warning data ALC. To facilitate the user associating the expiration date of eggplant FD1 with the location of eggplant FD1, it is preferable that the second warning data AL2 is generated and synthesized in accordance with the first warning data AL1.

[0036] Figure 7 is a flowchart showing another example of the food management method performed by the food management device shown in Figure 2. As shown in Figure 7, the expiration date management unit 41 of the food management device MA determines, based on inventory information, whether or not there is at least one food FD that is nearing its expiration date or is old (step S240, second expiration date management step). Specifically, it determines whether or not there is a food FD whose expiration date at the current time is less than or equal to a threshold. Here, the absolute value is not taken.

[0037] In step S240, "the present moment" refers to the time when the deadline management unit 41 executes step S240, or the time corresponding to the series of processes including step S240. For example, if step S240 is performed when requested by a user through the user interface UI, it may be the time when the user enters the request into the user interface UI.

[0038] Figure 8 is a flowchart showing an example of a food management method performed by the food management device shown in Figure 2. As shown in Figure 8, the receiving unit 42 of the food management device MA reads inventory information from the storage device ST1 (step S300) and receives designation information that specifies at least one type from at least one type of food FD based on the inventory information (step S330, receiving process).

[0039] Figure 9 is a schematic diagram showing an example of a screen displayed in the user interface in the food management method shown in Figure 8. For example, if the user interface UI includes a touch panel display, the output generation unit 43 generates an inventory list SL based on inventory information (step S310), and the food management device MA instructs the user interface UI to display the inventory list SL and the first image IM1 as shown in the upper part of Figure 9 (step S320). The user specifies at least one type on the displayed inventory list SL, and the user interface UI sends the specification information based on this to the food management device MA. Then, in step S330, the receiving unit 42 of the food management device MA determines whether or not it has received the specification information from the user interface UI.

[0040] The inventory list (SL) should preferably be user-friendly. For example, if there are food products of the same type but with different expiration dates, only the earliest expiration date among them may be listed in the inventory list (SL). For example, the inventory list (SL) may be sorted by expiration date or by remaining quantity.

[0041] Figure 10 is a schematic diagram showing an example of first data indicating location in the food management method shown in Figure 8. When the reception unit 42 receives the specified information, the output generation unit 43 of the food management device MA updates the inventory list SL so that it reflects the specified information (step S340). For example, it checks the checkbox corresponding to the specified type or changes the background color of the corresponding column. The output generation unit 43 of the food management device MA also generates first data DT1 indicating the location of food FD corresponding to the type specified by the specified information, as shown in Figure 10 (step S350). Then, the image synthesis unit 44 of the food management device MA generates a composite image IMC by synthesizing the first data DT1 with the first image IM1 (step S360). The food management device MA commands the user interface UI to display the inventory list SL and the composite image IMC, as shown in the lower part of Figure 9 (step S370).

[0042] Figure 11 is a flowchart showing another example of a food management method performed by the food management device shown in Figure 2. Figure 12 is a schematic diagram showing an example of a first data indicating location and a second data indicating expiration date and remaining quantity in the food management method shown in Figure 11. As shown in Figures 11 and 12, the output generation unit 43 of the food management device MA may generate second data DT2 indicating the expiration date and remaining quantity of food FDs corresponding to the type specified by the specified information, in addition to the first data DT1 (step S380). This remaining quantity is the remaining quantity calculated for each combination of food FD type and expiration date. Then, the image synthesis unit 44 of the food management device MA generates a composite image IMC by synthesizing the first data DT1 and the second data DT2 with the first image IM1 (step S390). Preferably, the second data DT2 is generated and synthesized according to the first data DT1 in order to facilitate the user in associating the expiration date and remaining quantity of food FDs with the location of the food FDs.

[0043] (Implemented by software) The functions of the food management device MA are food management programs that enable the computer to function as the food management device MA, and can be realized by food management programs that enable the computer to function as a location identification unit 14, an identifier assignment unit 21, a type determination unit 15, and a recording unit 30. The food management program is optional, and the computer may also function as one of the other control blocks (each part included in the control device CU) of the food management device MA.

[0044] In this case, the food management device MA includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the above program. By executing the above program using this control device and storage device, each of the functions described in each of the above embodiments is realized. The storage device for executing the program may be the same as or different from the storage device ST1 for storing inventory information, and may be the same as or different from the storage device ST2 for storing learning results.

[0045] The above food management program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the food management device MA. In the latter case, the program may be supplied to the food management device MA via any wired or wireless transmission medium.

[0046] Furthermore, some or all of the functions of each of the above control blocks can also be implemented by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of this disclosure. In addition, it is also possible to implement the functions of each of the above control blocks by, for example, a quantum computer.

[0047] Furthermore, the processing in each control block of the food management device MA may be performed by AI (Artificial Intelligence). In this case, the AI ​​may operate on the control device described above, or it may operate on another device (for example, an edge computer or a cloud server).

[0048] [Embodiment 2] Figure 13 is a schematic diagram showing an example configuration of a food management system including a food management device according to one embodiment of the present disclosure. As shown in Figure 13, the food management system MS comprises one or more refrigerators R2 and a food management device MA according to the present disclosure. Each refrigerator R2 comprises a storage chamber CC for storing food FD and a camera CM for photographing the inside of the storage chamber CC.

[0049] The food management device MA according to this disclosure may be implemented by equipment other than the refrigerator R2, and a camera CM or a control unit that controls the camera CM may be connected to said equipment. The food management device MA according to this disclosure may be partially implemented by each refrigerator R2, and the remaining part may be implemented by equipment other than the refrigerator R2. The food management device MA may be implemented by a plurality of interconnected devices, and one or more of said devices may be connected to a camera CM or a control unit that controls the camera CM. For example, the food management device MA may use a computer connected to the refrigerator R2 via a local area network (LAN), or a combination of a remote server connected to the refrigerator R2 via an internet line and a storage device attached to the remote server.

[0050] The food management device MA may be connected to one or more user interface UIs. Each user interface UI may be located on the refrigerator R2 or on a device other than the refrigerator R2. For example, an information terminal such as a smartphone, tablet, smart speaker, or laptop computer may be used as a user interface UI.

[0051] In the example shown in Figure 13, the food management system MS comprises a refrigerator R2, a food management device MA implemented by a remote server, and a user interface UI implemented by an information terminal. The refrigerator R2 and the user interface UI are connected to the food management device MA via a communication network CN.

[0052] The relationship between a server and an information terminal can be either a PULL type, where the server sends information to the information terminal in response to a request from the terminal, or a PUSH type, where the server spontaneously sends information to the information terminal. In a food management system MS, the relationship between the food management device MA and the user interface UI can be either a PULL type, a PUSH type, or both.

[0053] 〔summary〕 A food management device (MA) according to Embodiment 1 of the present disclosure comprises: a location identification unit (14) that identifies the location of at least one food item (FD) in the storage compartment (CC) of a refrigerator (R1, R2) by analyzing a first image of the storage compartment (CC) of the refrigerator (R1, R2); an identifier assignment unit (21) that assigns an identifier to each of the at least one food item; a type determination unit (15) that determines the type of food item for each of the at least one food item by analyzing a second image of the storage compartment showing the food item; and a recording unit (30) that records the identifier, location, and type of each of the at least one food item as inventory information.

[0054] The food management device according to aspect 2 of the present disclosure may further include an expiration date setting unit (22) for each of the at least one food items, and the recording unit may record the identifier, location, type, and expiration date for each of the at least one food items as inventory information.

[0055] In the food management device according to aspect 3 of the present disclosure, the expiration date setting unit may be configured to set the expiration date based on the type of food and the current time, but only for food whose location is not currently recorded in the inventory information.

[0056] The food management device according to aspect 4 of the present disclosure may further include an expiration date management unit (41) that determines, based on the inventory information, whether or not there is a food item among the at least one food item for which the difference between the current date and the expiration date is less than or equal to a threshold.

[0057] The food management device according to aspect 5 of the present disclosure may further include an expiration date management unit (41) that determines, based on the inventory information, whether or not there is a food item among the at least one food item whose expiration date is in the past a threshold time from the present time.

[0058] The food management device according to aspect 6 of the present disclosure may further include a remaining quantity calculation unit (23) that calculates the remaining quantity of at least one food item in the storage chamber for each type of food item, in any one aspect of aspects 1 to 5 of the present disclosure.

[0059] The food management device according to aspect 7 of the present disclosure may further include, in any one aspect of aspects 1 to 6 of the present disclosure, a receiving unit (42) that receives designation information that specifies at least one type of food from among the at least one type of food based on the inventory information, and an output generation unit (43) that generates first data (DT1) indicating the location of the food corresponding to the at least one type of food specified by the designation information, based on the inventory information.

[0060] The food management device according to aspect 8 of the present disclosure may further include, in any one aspect of aspects 2 to 5 of the present disclosure, a remaining quantity calculation unit (23) that calculates the remaining quantity of the at least one food in the storage chamber for each combination of the food type and expiration date; a receiving unit (42) that receives designation information that specifies at least one type of food from the at least one food based on the inventory information; and an output generation unit (43) that generates first data (DT1) indicating the location of the food corresponding to the at least one type specified by the designation information, and second data (DT2) indicating the expiration date and remaining quantity of the food corresponding to the at least one type specified by the designation information, based on the inventory information.

[0061] The food management system (MS) according to aspect 9 of this disclosure may have a configuration that includes a food management device (MA) according to any one of aspects 1 to 8 above, and the refrigerators (R1, R2).

[0062] A food management method according to aspect 10 of the present disclosure is a method that includes: a location identification step (step S40) which identifies the location of at least one food item in the storage compartment of a refrigerator by analyzing a first image of the storage compartment of the refrigerator; an identifier assignment step (step S50) which assigns an identifier to each of the at least one food item; a type determination step (step S70) which determines the type of food item by analyzing a second image of the storage compartment of the refrigerator in which the food item is shown; and a recording step (step S100) which records the identifier, location, and type of each of the at least one food item as inventory information.

[0063] Each aspect of the Food Management Device (MA) of this Disclosure may be implemented by a computer, in which case the Food Management Program for the Food Management Device, which enables the implementation of the Food Management Device by computer by operating the computer as each part (software element) of the Food Management Device, and a computer-readable recording medium on which the Program is recorded also fall within the scope of this Disclosure.

[0064] The food management program according to aspect 11 of this disclosure is a food management program for causing a computer to function as a food management device (MA) in any one of aspects 1 to 8 above, and is a program for causing the computer to function as the location identification unit (14), the identifier assignment unit (21), the type determination unit (15), and the recording unit (30).

[0065] The recording medium according to aspect 12 of this disclosure is a computer-readable recording medium on which the food management program described in aspect 11 is recorded.

[0066] This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment. [Explanation of Symbols]

[0067] 14 Location identification part 15 Type determination section 21 Identifier Assignment Unit 22 Deadline Setting Section 23 Remaining Quantity Calculation Unit 41 Deadline Management Department 42 Reception Department 43 Output generation unit CC Confinement Chamber CM Camera DT1 First Data DT2 2nd Data FD Food IM1 First Image MA food management equipment MS Food Management System R1, R2 Refrigerator

Claims

1. A positioning unit that identifies the location of at least one food item shown in the first image within the storage compartment of a refrigerator by analyzing the first image of the storage compartment, An identifier assigning unit that assigns an identifier to each of the at least one food item, A type determination unit determines the type of food by analyzing a second image of the storage chamber in which the food is shown for each of the at least one food items. A food management device comprising a recording unit that records, for each of the at least one food items, the identifier, the location, and the type in association as inventory information.

2. Each of the above-mentioned at least one food item is further provided with an expiration date setting unit for setting the expiration date of the food item. The food management device according to claim 1, wherein the recording unit records the identifier, location, type, and expiration date for each of the at least one food items as inventory information.

3. The food management device according to claim 2, wherein the expiration date setting unit sets the expiration date only for food items whose location is not currently recorded in the inventory information, based on the type of food item and the current time.

4. The food management device according to claim 2, further comprising a shelf-life management unit that determines, based on the inventory information, whether or not there is a food item among the at least one food item for which the difference between the current date and the expiration date is less than or equal to a threshold.

5. The food management device according to claim 2, further comprising an expiration date management unit that determines, based on the inventory information, whether or not there is a food item among the at least one food item whose expiration date is in the past a threshold time from the present time.

6. The food management device according to claim 1, further comprising a remaining quantity calculation unit that calculates the remaining quantity of at least one food item in the storage chamber for each type of food item.

7. A receiving unit that receives designation information to specify at least one type of food from the aforementioned at least one type of food, based on the inventory information, The food management device according to claim 1, further comprising: an output generation unit that generates first data indicating the location of food corresponding to the at least one type of food specified by the designated information, based on the inventory information.

8. A remaining quantity calculation unit calculates the remaining quantity of at least one food item in the storage compartment for each combination of food item type and expiration date, A receiving unit that receives designation information to specify at least one type of food from the aforementioned at least one food item based on the inventory information, The food management device according to claim 2, further comprising: an output generation unit that generates, based on the inventory information, first data indicating the location of food corresponding to the at least one type of food specified by the designated information, and second data indicating the expiration date and remaining quantity of food corresponding to the at least one type of food specified by the designated information.

9. A food management system comprising the food management device described in claim 1 and the refrigerator.

10. A position identification step involves analyzing a first image of the inside of the storage compartment of a refrigerator to identify the position of at least one food item shown in the first image within the storage compartment, An identifier assignment step is performed for each of the aforementioned at least one food item, and A type determination step in which, for each of the at least one food item, the type of food item is determined by analyzing a second image of the storage chamber in which the food item is shown, A food management method comprising a recording step of recording, for each of the at least one food items, the identifier, the location, and the type in association as inventory information.

11. A food management program for causing a computer to function as a food management device according to claim 1, wherein the computer functions as the location identification unit, the identifier assignment unit, the type determination unit, and the recording unit.

12. A computer-readable recording medium that stores the food management program described in claim 11.

Images