Imaging device

Abstract

This disclosure provides a camera that can reduce the number of parts and start taking pictures of the inside of a refrigerator at an appropriate time. [Solution] The imaging device in this disclosure is an imaging device for imaging a refrigerator from the front and above, comprising a camera for imaging the refrigerator, and a first sensor for detecting at least the open state of a rotary opening / closing body provided in the first storage compartment of the refrigerator, and at least the open state of a pull-out opening / closing body provided in the second storage compartment of the refrigerator, wherein the camera starts imaging when the first sensor detects that at least one of the rotary opening / closing body and the pull-out opening / closing body is in an open state.

Description

Technical Field

[0001] The present disclosure relates to a photographing device.

Background Art

[0002] Patent Document 1 discloses a photographing device that photographs a refrigerator from the front upper side of the refrigerator. The photographing device disclosed in Patent Document 1 includes a distance measuring sensor that detects the opening and closing of a rotary door and a distance measuring sensor that detects the opening and closing of a drawer, and starts photographing the inside of the refrigerator when any of the distance measuring sensors detects an open state.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure provides a photographing device that can reduce the number of components and start photographing the inside of a refrigerator at an appropriate timing.

Means for Solving the Problems

[0005] The photographing device in the present disclosure is a photographing device that photographs the refrigerator from the front upper side of the refrigerator, and includes a camera that photographs the refrigerator, and a first sensor that detects at least an open state of a rotary opening / closing body provided in a first storage chamber of the refrigerator and at least an open state of a drawer-type opening / closing body provided in a second storage chamber of the refrigerator. The camera starts photographing when the first sensor detects an open state of at least one of the rotary opening / closing body and the drawer-type opening / closing body.

Effects of the Invention

[0006] The photographing device in the present disclosure can reduce the number of components and start photographing the inside of a refrigerator at an appropriate timing. [Brief explanation of the drawing]

[0007] [Figure 1] Diagram showing the configuration of the management system in Embodiment 1 [Figure 2] Perspective view of the imaging device as seen from below in Embodiment 1 [Figure 3] Plan view showing the internal configuration of the case member in Embodiment 1 [Figure 4] Plan view of the camera in its installed state according to Embodiment 1, viewed from above. [Figure 5] A plan view of the refrigerator and the camera installed on the refrigerator in Embodiment 1, as seen from the front. [Figure 6] A plan view from the right of the refrigerator and the imaging device installed on the refrigerator in Embodiment 1. [Figure 7] A plan view from the front of the refrigerator and the imaging device installed on the refrigerator in Embodiment 1. [Figure 8] A plan view from the right of the refrigerator and the imaging device installed on the refrigerator in Embodiment 1. [Figure 9] Block diagram showing the configuration of the control system of the imaging device in Embodiment 1. [Figure 10] Block diagram showing the configuration of the terminal device and the control system of the management server in Embodiment 1. [Figure 11] Flowchart showing the operation of the imaging device in Embodiment 1 [Figure 12] Flowchart showing the operation of the imaging device in Embodiment 1 [Figure 13] Flowchart showing the operation of the imaging device and management server in Embodiment 1 [Figure 14] Flowchart showing the operation of the terminal device and management server in Embodiment 1 [Figure 15] Plan view from above of the camera installed in the refrigerator in Embodiment 2. [Figure 16] A plan view of the refrigerator and the camera installed on the refrigerator in Embodiment 2, as seen from the front. [Figure 17] Plan view of the refrigerator and the imaging device installed in the refrigerator in Embodiment 2 as seen from the right side [Figure 18] Block diagram showing the configuration related to the control system of the imaging device in Embodiment 2

Embodiments for Carrying Out the Invention

[0008] (Findings etc. on which the present disclosure is based) When the inventors came up with the present disclosure, as in Patent Document 1, there was an imaging device that starts imaging the inside of the refrigerator when a sensor detects the open state of a rotary opening / closing body or a drawer-type opening / closing body of the refrigerator. However, in the conventional imaging device, since a sensor for detecting the open state is provided for each of the rotary opening / closing body and the drawer-type opening / closing body, the inventors discovered the problem that the number of parts of the imaging device is large, and in order to solve that problem, they came to constitute the subject matter of the present disclosure. Therefore, the present disclosure provides an imaging device that can reduce the number of parts and start imaging the inside of the refrigerator at an appropriate timing.

[0009] Hereinafter, embodiments will be described in detail with reference to the drawings. However, there may be cases where more detailed explanations than necessary are omitted. For example, there may be cases where detailed explanations of already well-known matters or duplicate explanations for substantially the same configurations are omitted. Note that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims thereby.

[0010] (Embodiment 1) [1-1. Configuration] [1-1-1. Configuration of the management system] FIG. 1 is a diagram showing the configuration of the management system 1 in Embodiment 1. The management system 1 is a system that manages the items stored in the refrigerator 2 and the imaging device 3 that photographs the refrigerator 2. As an example of item management in this embodiment, the management system 1 manages an image in which the items stored in the refrigerator 2 are captured. Note that the item management performed by the management system 1 is not limited to the image, and other objects such as the expiration date of the item, the entry and exit dates of the item, and the type of the item may also be managed. Note that the items stored in the refrigerator 2 include food.

[0011] The management system 1 includes a refrigerator 2. The refrigerator 2 includes a main body 20 with an open front. In the main body 20, a refrigerating chamber 21, an ice-making chamber 22, a fresh-freezing chamber 23, a freezing chamber 24, and a vegetable chamber 25 are formed. These storage chambers are cooled by cold air. A rotary door 21A is installed at the opening in the front of the refrigerating chamber 21. The door 21A includes a rotary right door 21B and a rotary left door 21C. In each of the ice-making chamber 22, the fresh-freezing chamber 23, the freezing chamber 24, and the vegetable chamber 25, drawers 22A, 23A, 24A, and 25A capable of storing items are installed respectively. Each of the drawers 22A to 25A has an open top in the installed state. The refrigerating chamber 21 is an example of the "first storage chamber". Also, the door 21A is an example of the "rotary opening / closing body". Also, the right door 21B is an example of the "first rotary opening / closing body". Also, the left door 21C is an example of the "second rotary opening / closing body".

[0012] Hereinafter, when the ice-making chamber 22, the fresh-freezing chamber 23, the freezing chamber 24, and the vegetable chamber 25 are not distinguished, they are denoted by the symbol "26" and expressed as the "drawer storage chamber 26". Hereinafter, when the drawers 22A, 23A, 24A, and 25A are not distinguished, they are denoted by the symbol "26A" and expressed as the "drawer 26A". The drawer storage chamber 26 is an example of the "second storage chamber". Also, the drawer 26A is an example of the "drawer-type opening / closing body".

[0013] A first marker MK1 is attached to the refrigerator compartment 21 at a predetermined location. The first marker MK1 is a marker with identification information for the refrigerator compartment 21 on its main surface. The first marker MK1 is attached to the refrigerator compartment 21 with the identification information for the refrigerator compartment 21 facing upwards so that the imaging device 3 can read the identification information for the refrigerator compartment 21.

[0014] A second marker MK2 is attached to a predetermined position inside the drawer storage compartment 26. The second marker MK2 is a marker on which the identification information of the drawer storage compartment 26 is attached to the main surface. The second marker MK2 is attached inside the refrigerator compartment 21 with the identification information of the drawer storage compartment 26 facing upwards so that the camera 3 can read the identification information of the drawer storage compartment 26. In this embodiment, the case in which the second marker MK2 is attached to the vegetable compartment 25 is given as an example, but the second marker MK2 may be attached to all of the drawer storage compartments 26 of the refrigerator 2, or to some of the drawer storage compartments 26 of the refrigerator 2. The identification information for the drawer storage compartment 26 to which the second marker MK2 is attached corresponds to the type of drawer storage compartment 26 to which the second marker MK2 is attached. For example, the second marker MK2 attached to the vegetable compartment 25 contains the identification information for the vegetable compartment 25.

[0015] Hereafter, when the first marker MK1 and the second marker MK2 are not distinguished, they will be referred to as "marker MK" with the designation "MK".

[0016] The management system 1 includes an imaging device 3. The camera 3 is detachably mounted on the top surface 20A of the main casing 20 of the refrigerator 2. The camera 3, while installed in the refrigerator 2 (hereinafter referred to as "installed state"), photographs the refrigerator 2 from the front and above.

[0017] The management system 1 includes a terminal device 4. The terminal device 4 is a portable computer such as a smartphone. The terminal device 4 has an application program installed that is related to the management of items stored in the refrigerator 2, and communicates with the management server 5 through the functions of this application program. In the following explanation, this application program will be referred to as the "management app" and assigned the code "411".

[0018] In Figure 1, users P who are at home are shown with a solid line, and users P who have left home H are shown with a dotted line. When terminal device 4 is used by user P who is at home, it communicates with the management server 5 either via communication device 6 or without communication device 6. Also, when terminal device 4 is used by user P who has left home H and cannot establish a communication connection with communication device 6, it communicates with the management server 5 without communication device 6.

[0019] The communication device 6 connects to a network NW, which is composed of a public telephone network, and communicates with the management server 5 via the network NW. The communication device 6 functions as an interface device for connecting the refrigerator 2, the camera 3, and the terminal device 4 to the network NW. The communication device 6 establishes a local network at home H.

[0020] Management system 1 includes a management server 5. The management server 5 is a server device that manages the items stored in the refrigerator 2 and the camera 3. The management server 5 is connected to the network NW. In each diagram, the management server 5 is represented by a single block, but this does not necessarily mean that the management server 5 is composed of a single device.

[0021] [1-1-2. Configuration of the imaging device] The configuration of the imaging device 3 will be explained with reference to Figures 2-9. Figures 2-8 illustrate the X, Y, and Z axes. The X, Y, and Z axes are orthogonal to each other. The Z axis indicates the vertical direction and corresponds to the height direction of the imaging device 3 in its installed state. The X and Y axes are parallel to the horizontal direction. The X axis indicates the left-right direction. The left-right direction corresponds to the left-right direction of the imaging device 3 in its installed state, in other words, the width direction of the imaging device 3. Also, the left-right direction corresponds to the left-right direction of the refrigerator 2, in other words, the width direction of the refrigerator 2. The Y axis indicates the front-back direction. The positive direction of the X axis is to the right. The positive direction of the Y axis is forward. The positive direction of the Z axis is upward.

[0022] Figure 2 is a perspective view of the imaging device 3 from below. The imaging device 3 comprises a case member 30 and a cover member 31. The case member 30 is a rectangular member in plan view. The case member 30 has a base 32 located at the rear of the case member 30 and installed on the top surface 20A of the refrigerator 2, and an imaging unit 33 extending in front of the base 32 and provided with a predetermined gap from the top surface 20A of the refrigerator 2.

[0023] Multiple legs 35 are provided on the bottom surface 34 of the base 32. The legs 35 are components for installing the case member 30 with a predetermined gap from the top surface 20A of the refrigerator 2, and are made of, for example, rubber.

[0024] A position regulating portion 37 is provided on the front end surface 36 of the base portion 32. The position regulating portion 37 is a piece that extends downward from the bottom surface 34 and contacts the front end edge 20B of the main body 20 of the refrigerator 2 to regulate the position of the refrigerator 2 in the front-rear direction. The position regulating portion 37 extends in the width direction of the base portion 32 (the width direction of the imaging device 3) on the front end surface 36 and has a mark 38 in the center of the width direction of the base portion 32. The mark 38 indicates the center in the width direction of the imaging device 3. The length of the position regulating portion 37 extending from the bottom surface 34 is shorter than the length from the top surface 20A of the main body 20 to the door packing provided on the upper inner surface of the door 21A, which is approximately parallel to the top surface 20A.

[0025] An operating section 39 for receiving operations is provided on the front end surface 36 of the base portion 32. The operating section 39 includes a first button 39A for turning the power of the imaging device 3 on and off, a second button 39B for wirelessly connecting the imaging device 3 and the communication device 6, and a third button 39C for starting imaging on the imaging device 3. The first button 39A, the second button 39B, and the third button 39C are arranged in this order from left to right.

[0026] Figure 3 is a plan view showing the internal structure of the case member 30. As shown in Figure 3, the base 32 includes a power supply board 40. The power supply board 40 is connected to an external power source and supplies power to each part of the imaging device 3. The power supply board 40 is connected to a power harness 41 capable of transmitting power. The power harness 41 is routed inside the case member 30 to the right of the power supply board 40. The power harness 41 extends from the rear end of the case member 30 to the outside of the case member 30 and is connected to an external power source. In this embodiment, the power harness 41 is connected to a commercial AC power source. Alternatively, the power harness 41 may be connected to the power supply board of the refrigerator 2. In this case, the imaging device 3 and the refrigerator 2 may be connected via a USB (Universal Serial Bus) cable.

[0027] The base unit 32 includes a sensor board 43. The sensor board 43 is a board that controls the distance measuring sensor 44 and thermopile 45 of the imaging device 3. The sensor board 43 is located to the left of the power supply board 40. The distance measuring sensor 44 is connected to the sensor board 43 via cable 46, and the thermopile 45 is connected via cable 47. The sensor board 43 is also a board that controls the first LED (Light Emitting diode) 48, the second LED 49, and the buzzer 50. The first LED 48 is connected to the sensor board 43 via cable 51, the second LED 49 is connected via cable 52, and the buzzer 50 is connected via cable 53. In this embodiment, the distance measuring sensor 44 is an example of a "second sensor." The thermopile 45 is an example of a "first sensor."

[0028] The imaging unit 33 includes a System-on-a-Chip (SoC) board 54. The SoC board 54 is a board that controls the wide-angle camera 55 and the narrow-angle camera 56. The wide-angle camera 55 and the narrow-angle camera 56 are connected to the SoC board 54 via two Flexible Flat Cables (FFCs) 57. The SoC board 54 is an example of a "camera board". The wide-angle camera 55 is an example of a "first camera". The narrow-angle camera 56 is an example of a "second camera". Hereafter, when wide-angle camera 55 and narrow-angle camera 56 are not distinguished, the designation "58" is added, and they will be referred to as "camera 58".

[0029] Figure 4 is a plan view of the installed camera 3, seen from above. Figure 5 is a plan view of the refrigerator 2 and the camera 3 installed on the refrigerator 2, seen from the front. In Figure 5, the door 21A of the refrigerator 2 is open. Figure 6 is a plan view of the refrigerator 2 and the camera 3 installed on the refrigerator 2, seen from the right. In Figure 6, the drawer 25A of the refrigerator 2 is open, i.e., pulled out. Figure 7 is a plan view of the refrigerator 2 and the camera 3 installed on the refrigerator 2, seen from the front. In Figure 7, the door 21A of the refrigerator 2 is closed. Figure 8 is a plan view of the refrigerator 2 and the camera 3 installed on the refrigerator 2, seen from the right. In Figure 8, the drawer 26A of the refrigerator 2 is closed, i.e., not pulled out.

[0030] In the imaging device 3 shown in Figures 4 to 8, the mark 38 is located approximately in the center CT in the width direction of the refrigerator 2, and the position regulating part 37 is in contact with the front edge 20B of the main box 20.

[0031] The imaging device 3 includes a wide-angle camera 55 and a narrow-angle camera 56. The wide-angle camera 55 and the narrow-angle camera 56 are positioned in front of the door 21A when the imaging device 3 is installed and the door 21A is closed.

[0032] The wide-angle camera 55 is a camera with a wider angle than the narrow-angle camera 56. When the imaging device 3 is installed, the wide-angle camera 55 is located approximately in the center CT in the width direction of the refrigerator 2 and photographs the refrigerator compartment 21 from the front and above of the refrigerator 2. For example, the shooting range of the wide-angle camera 55 includes the range A1 shown in Figure 5 when viewed from the front of the refrigerator 2. Range A1 includes the door pockets 21D of the right door 21B and the left door 21C, and the opening of the refrigerator compartment 21, when the right door 21B and the left door 21C are in the fully open position. In addition, the shooting range of the wide-angle camera 55 includes the range A3 shown in Figure 6 when viewed from the right side of the refrigerator 2. Range A3 includes the range from the front ends of the open right door 21B and left door 21C to the front end of the shelf 21E in the front-to-back direction, and the range including the opening of the refrigerator compartment 21 in the up-to-down direction.

[0033] The narrow-angle camera 56 is a camera with a narrower field of view than the wide-angle camera 55. When the imaging device 3 is installed, the narrow-angle camera 56 is located to the left of the central CT in the width direction of the refrigerator 2 and photographs the drawer 26A from the front upper part of the refrigerator 2. For example, the shooting range of the narrow-angle camera 56 includes area A2 shown in Figure 5, when viewed from the front of the refrigerator 2. Area A2 includes the drawer 26A when viewed from the front of the refrigerator 2. Also, for example, the shooting range of the narrow-angle camera 56 includes area A4 shown in Figure 6, when viewed from the right side of the refrigerator 2. Area A4 includes the drawer 26A in its fully extended state in the front-to-back direction.

[0034] The imaging device 3 is equipped with a distance measuring sensor 44. The distance measuring sensor 44 is a sensor that detects the opening and closing of the left door 21C. When the imaging device 3 is installed, the distance measuring sensor 44 is located to the left of the central CT in the width direction of the refrigerator 2. That is, when the imaging device 3 is installed, the distance measuring sensor 44 is located on the hinge member 21C1 side of the left door 21C, above the central CT, in the width direction of the refrigerator 2. Also, when the imaging device 3 is installed, the distance measuring sensor 44 is located behind the front edge 20B of the main body 20 in the front-rear direction of the refrigerator 2. The distance measuring sensor 44 outputs a sensing element corresponding to the measurement format, such as light, millimeter waves, or ultrasound, from a window portion 59 formed on the front end surface 36 of the base portion 32, detects the distance between itself and the left door 21C, and outputs a detected value indicating the detected distance. The sensing target of the distance measuring sensor 44 is the part of the left door 21C above the upper door gasket.

[0035] The imaging device 3 is equipped with a thermopile 45. The thermopile 45 is a sensor that detects the opening and closing of the right door 21B and the drawer 26A. When the imaging device 3 is installed, the thermopile 45 is located to the right of the central CT in the width direction of the refrigerator 2. That is, when the imaging device 3 is installed, the thermopile 45 is located on the hinge member 21B1 side of the right door 21B, relative to the central CT, in the width direction of the refrigerator 2. Furthermore, when the imaging device 3 is installed, the thermopile 45 is positioned such that, when viewed from above, at least half of the thermopile 45 is located in front of the closed right door 21B. Here, "at least half of the thermopile 45 is located in front of the closed right door 21B" means that at least half of the infrared detection elements of the thermopile 45 are positioned so as not to overlap with the refrigerator 2. The thermopile 45 may also be located between the left door 21C and the right door 21B when viewing the refrigerator 2 from above.

[0036] When the imaging device 3 is installed, the thermopile 45 senses the area below the refrigerator 2. The thermopile 45 is positioned so that, when viewed from the front of the refrigerator 2, its sensing direction faces the central CT side of the refrigerator 2. The sensing range of the thermopile 45 includes the range A5 shown in Figure 7 when viewed from the front of the refrigerator 2. Range A5 is a range that extends vertically with a sensing angle of θ1. Range A5 also includes a portion of the closed right door 21B and a portion of the closed drawer 26A. Furthermore, the thermopile 45 is positioned so that, when viewed from the side of the refrigerator 2, its sensing direction is tilted from the rear to the front of the refrigerator 2. The sensing range of the thermopile 45 includes the range A6 shown in Figure 8 when viewed from the right side of the refrigerator 2. Range A6 is a range that extends approximately parallel to the Z-axis and has a sensing angle of θ2. Range A6 is the area in front of the refrigerator 2 when the door 21A and drawer 26A are closed, and is the area in which the top surface of the open drawer 26A can be detected. Range A6 is also the area in front of the refrigerator 2 when the door 21A and drawer 26A are closed, and is the area in which the door pocket 21D of the open right door 21B can be detected. Furthermore, because range A6 is the area in front of the refrigerator 2 when the door 21A and drawer 26A are closed, the thermopile 45 can detect the temperature of a hand touching the refrigerator 2.

[0037] The thermopile 45 detects the temperature of a portion of the floor FL (FLA) when the right door 21B and drawer 26A are closed and there are no objects within the sensing range. Furthermore, the thermopile 45 detects the temperature of the door pocket 21D of the right door 21B if the door pocket 21D of the right door 21B is within its sensing range. Generally, the temperature of the door pocket 21D is lower than the temperature of the area FLA. Furthermore, the thermopile 45 detects the temperature inside drawer 26A if it is within its sensing range. Generally, the temperature inside drawer 26A is lower than the temperature of region FLA. Furthermore, the thermopile 45 detects the temperature of user P's hand if the user P's hand, which is attempting to open door 21A or drawer 26A, is within its sensing range. Generally, the temperature of user P's hand is higher than the temperature of area FLA. Furthermore, the thermopile 45 detects the temperature of food items being moved in and out of refrigerator 2 if those items are within its sensing range. Generally, the temperature of food items being moved in and out of refrigerator 2 is lower than the temperature of area FLA.

[0038] The imaging device 3 is equipped with a first LED 48 and a second LED 49. The first LED 48 is located at the right front end of the imaging device 3 and illuminates the interior of the refrigerator 2. The second LED 49 is located at the left front end of the imaging device 3 and illuminates the interior of the refrigerator 2.

[0039] Figure 9 is a block diagram showing the configuration of the control system for the imaging device 3. The imaging device 3 includes a power supply board 40, a sensor board 43, an SoC board 54, and an operation display board 60.

[0040] The operation display board 60 is a board to which the first button 39A, the second button 39B, and the third button 39C are connected. The operation display board 60 includes a first processor 100 such as a CPU (Central Processing Unit), a first memory 110, a first communication module 120, and interface circuits to which other devices and sensors are connected.

[0041] The first memory 110 is a storage device that stores programs and data executed by the first processor 100. The first memory 110 is composed of non-volatile storage devices such as ROM (Read Only Memory). The first memory 110 also includes volatile storage devices that constitute the work area of ​​the first processor 100, such as RAM (Random Access Memory). The first memory 110 stores data processed by the first processor 100 and control programs 111 executed by the first processor 100.

[0042] The first communication module 120 is equipped with hardware such as communication circuits and connectors that conform to a predetermined wired communication standard, and communicates with each board of the imaging device 3 in accordance with the control of the first processor 100.

[0043] The first processor 100 communicates with each board of the imaging device 3 via the first communication module 120 by reading and executing the control program 111. When the first button 39A is operated, the first processor 100 sends a signal to the power supply board 40 indicating that the first button 39A has been operated. The power supply board 40 turns the power of the imaging device 3 on and off according to the signal sent by the operation display board 60. When the second button 39B is operated, the first processor 100 sends a signal to the SoC board 54 indicating that the second button 39B has been operated. When the third button 39C is operated, the first processor 100 sends a signal to the SoC board 54 indicating that the third button 39C has been operated.

[0044] The sensor board 43 is a board to which the distance measuring sensor 44, thermopile 45, first LED 48, second LED 49, and buzzer 50 are connected.

[0045] The sensor board 43 includes a second processor 200 such as a CPU, a second memory 210, a second communication module 220, and interface circuits to which other devices and sensors are connected.

[0046] The second memory 210 is a storage device that stores programs and data executed by the second processor 200. The second memory 210 is composed of a non-volatile storage device, such as ROM. The second memory 210 also includes a volatile storage device that constitutes the work area of ​​the second processor 200, such as RAM. The second memory 210 stores data processed by the second processor 200 and control programs 211 executed by the second processor 200.

[0047] The second communication module 220 is equipped with hardware such as communication circuits and connectors that conform to a predetermined wired communication standard, and communicates with the operation display board 60 and the SoC board 54 in accordance with the control of the second processor 200.

[0048] The second processor 200 communicates with the operation display board 60 and the SoC board 54 via the second communication module 220 by reading and executing the control program 211 stored in the second memory 210. The second processor 200 transmits the detected values ​​output by each sensor connected to the sensor board 43 to the SoC board 54 via the second communication module 220.

[0049] Furthermore, the second processor 200 controls the lighting state of the first LED 48 and the second LED 49 by reading and executing the control program 211 stored in the second memory 210. Furthermore, the second processor 200 reads and executes the control program 211 stored in the second memory 210, thereby outputting sound via the buzzer 50. The second processor 200 also outputs sound via the buzzer 50 at the timing when power is turned on to the imaging device 3 and when it is electrically connected to the refrigerator 2.

[0050] The SoC board 54 is the board to which the wide-angle camera 55, the narrow-angle camera 56, and the wireless communication module 61 are connected.

[0051] The wireless communication module 61 is a device that includes hardware such as an antenna and a wireless communication circuit.

[0052] The SoC board 54 includes a third processor 300 such as a CPU, a third memory 310, a third communication module 320, and interface circuits to which other devices and sensors are connected.

[0053] The third memory 310 is a storage device that stores programs and data executed by the third processor 300. The third memory 310 is composed of a non-volatile storage device, such as ROM. The third memory 310 also includes a volatile storage device that constitutes the work area of ​​the third processor 300, such as RAM. The third memory 310 stores data processed by the third processor 300, a control program 311 executed by the third processor 300, and an imaging device ID (Identification) 312. The imaging device ID 312 is the identification information for imaging device 3.

[0054] The third communication module 320 is equipped with hardware such as communication circuits and connectors that conform to a predetermined wired communication standard, and communicates with the operation display board 60 and the sensor board 43 in accordance with the control of the third processor 300.

[0055] The third processor 300 functions as the first communication control unit 301, the second communication control unit 302, the open / close determination unit 303, the shooting control unit 304, the recording control unit 305, and the generation unit 306 by reading and executing the control program 311 stored in the third memory 310.

[0056] The first communication control unit 301 communicates with the management server 5 via the wireless communication module 61.

[0057] The second communication control unit 302 communicates with the operation display board 60 and the sensor board 43 via the third communication module 320. The second communication control unit 302 receives detection values ​​from the sensor board 43 from the sensor board 43, which is connected to various sensors.

[0058] The opening / closing determination unit 303 determines whether the left door 21C has opened from a closed state based on the detection value of the distance measuring sensor 44 received by the second communication control unit 302. The opening / closing determination unit 303 determines that the left door 21C has opened from a closed state if the second communication control unit 302 receives a detection value different from the detection value corresponding to the closed state. On the other hand, the opening / closing determination unit 303 determines that the left door 21C has not opened from a closed state if the second communication control unit 302 receives a detection value corresponding to the closed state.

[0059] The opening / closing determination unit 303 determines whether at least one of the right door 21B and the drawer 26A has changed from a closed state to an open state, based on the value detected by the thermopile 45 received by the second communication control unit 302. The opening / closing determination unit 303 determines that at least one of the right door 21B and the drawer 26A has changed from a closed state to an open state if the value detected by the thermopile 45 has risen to a predetermined temperature or higher.

[0060] For example, suppose the temperature of region FLA is 20°C, the temperature of the door pocket 21D of the right door 21B is 5°C, and the predetermined temperature detected by the thermopile 45 is 5°C. In this example, if the door pocket 21D of the right door 21B is included in the sensing range of the thermopile 45, the temperature detected by the thermopile 45 will change by 15°C (=20°C - 5°C). Therefore, in this example, if the door pocket 21D of the right door 21B is included in the sensing range of the thermopile 45, the opening / closing determination unit 303 determines that at least one of the right door 21B and the drawer 26A has changed from a closed state to an open state.

[0061] For example, suppose the temperature of region FLA is 20°C, the temperature inside drawer 25A of the vegetable compartment 25 is 5°C, and the predetermined temperature detected by the thermopile 45 is 5°C. In this example, if drawer 25A is included in the sensing range of the thermopile 45, the temperature detected by the thermopile 45 will change by 15°C (=20°C - 5°C). Therefore, in this example, if drawer 25A is included in the sensing range of the thermopile 45, the opening / closing determination unit 303 determines that at least one of the right door 21B and drawer 26A has changed from a closed state to an open state.

[0062] For example, suppose the temperature of area FLA is 20°C, the temperature of user P's hand is 36°C, and the predetermined temperature detected by the thermopile 45 is 5°C. In this example, if user P's hand is within the sensing range of the thermopile 45, the temperature detected by the thermopile 45 will change by 16°C (=36°C - 20°C). Therefore, in this example, if user P's hand is within the sensing range of the thermopile 45, the opening / closing determination unit 303 determines that at least one of the right door 21B and the drawer 26A has changed from a closed state to an open state.

[0063] For example, suppose the temperature of area FLA is 20°C, the temperature of the food being put in and taken out of refrigerator 2 is 10°C, and the predetermined temperature detected by the thermopile 45 is 5°C. In this example, if the food is within the sensing range of the thermopile 45, the temperature detected by the thermopile 45 will change by 10°C (=20°C - 10°C). Therefore, in this example, if the food being put in or taken out is within the sensing range of the thermopile 45, the open / close determination unit 303 determines that at least one of the right door 21B and the drawer 26A has changed from a closed state to an open state.

[0064] If the opening / closing determination unit 303 determines that at least one of the door 21A and the drawer 26A is in the open state, it determines whether the door 21A and the drawer 26A have changed from the open state to the closed state based on the images captured by the wide-angle camera 55 and the narrow-angle camera 56. As will be revealed later, if the opening / closing determination unit 303 determines that the door is in the open state, the wide-angle camera 55 and the narrow-angle camera 56 take pictures. Based on the marker MK captured in the images taken by the wide-angle camera 55 and the narrow-angle camera 56, the opening / closing determination unit 303 determines whether the door 21A and the drawer 26A have changed from the open state to the closed state.

[0065] The open / close determination unit 303 determines that at least one of the door 21A and the drawer 26A is in the open state, and if the marker MK does not appear in the images captured by the wide-angle camera 55 and the narrow-angle camera 56 for a predetermined period of time, it determines that the door 21A and the drawer 26A are in the closed state. Furthermore, the opening / closing determination unit 303 determines that the door 21A and the drawer 26A are in the closed state if, after the marker MK is captured in the images taken by the wide-angle camera 55 and the narrow-angle camera 56, the marker MK is not captured in the images taken by the wide-angle camera 55 and the narrow-angle camera 56 for a predetermined period of time. Whether or not marker MK is captured in the image taken by camera 58 is determined based on pattern matching and color.

[0066] When the opening / closing determination unit 303 determines that at least one of the door 21A and the drawer 26A is open, the shooting control unit 304 causes at least one of the wide-angle camera 55 and the narrow-angle camera 56 to start shooting.

[0067] Furthermore, if the opening / closing determination unit 303 determines that the door 21A and the drawer 26A are in a closed state, the shooting control unit 304 instructs the wide-angle camera 55 and the narrow-angle camera 56 to stop shooting.

[0068] When the shooting control unit 304 instructs the camera 58 to start shooting, the recording control unit 305 starts recording the shooting results of the wide-angle camera 55 and the narrow-angle camera 56. That is, the recording control unit 305 starts recording recording data 313 to the third memory 310. The recording data 313 includes first recording data showing the shooting results of the wide-angle camera 55 and second recording data showing the shooting results of the narrow-angle camera 56.

[0069] When the shooting control unit 304 tells camera 58 to finish shooting, the recording control unit 305 stops recording the shooting results of the wide-angle camera 55 and the narrow-angle camera 56.

[0070] Furthermore, if there is already recording data 313 in the third memory 310, the recording control unit 305 will overwrite the recording data 313 stored in the third memory 310 with the new recording data 313 each time it records new recording data 313.

[0071] The generation unit 306 generates management data D1. Management data D1 is data to be uploaded to the management server 5 and is data for managing the items stored in the refrigerator 2. The first communication control unit 301 transmits the management data D1 generated by the generation unit 306 to the management server 5.

[0072] [1-1-3. Management Server Configuration] Figure 10 is a block diagram showing the configuration of the control system for terminal device 4 and management server 5. The management server 5 comprises a server processor 500, such as a CPU, server memory 510, and a server communication module 520. The server memory 510 and the server communication module 520 are connected to the server processor 500.

[0073] The server memory 510 is a storage device that stores programs and data executed by the server processor 500. The server memory 510 is composed of non-volatile storage devices. The server memory 510 also includes non-volatile storage devices that constitute the work area of ​​the server processor 500. The server memory 510 stores data processed by the server processor 500, control programs 511 executed by the server processor 500, and management DB (database) 512.

[0074] The management database 512 has one record for each imaging device 3. This record includes the imaging device ID 312, image data of the first image, and image data of the second image. The first and second images will be described later.

[0075] The server communication module 520 is equipped with communication circuits and connectors that conform to a predetermined communication standard, and communicates with the imaging device 3 and the terminal device 4 via the network NW.

[0076] The server processor 500 functions as the server communication unit 501 and the server processing unit 502 by reading and executing the control program 511 stored in the server memory 510. The server communication unit 501 communicates with the imaging device 3 and the terminal device 4 via the server communication module 520. The server processing unit 502 performs various information processing with the imaging device 3 and the terminal device 4 as clients.

[0077] [1-1-4. Terminal Device Configuration] Next, we will describe the configuration of terminal device 4. The terminal device 4 includes a terminal processor 400, such as a CPU, terminal memory 410, terminal communication module 420, and touch panel 430.

[0078] The terminal memory 410 is a storage device that non-volatilely stores programs and data executed by the terminal processor 400. The terminal memory 410 is composed of non-volatile storage devices. The terminal memory 410 also includes RAM, which constitutes the work area of ​​the terminal processor 400. The terminal memory 410 stores data processed by the terminal processor 400 and management applications 411 executed by the terminal processor 400.

[0079] The terminal communication module 420 is equipped with communication circuits and connectors that conform to a predetermined communication standard, and communicates with the management server 5 in accordance with the control of the terminal processor 400. The touch panel 430 comprises a display panel and a touch sensor that is superimposed on or integrated with the display panel.

[0080] The terminal processor 400 functions as an application execution unit 401. The application execution unit 401 communicates with the management server 5 via the terminal communication module 420. The application execution unit 401 displays a user interface related to the management of items stored in the refrigerator 2 on the touch panel 430. In the following description, this user interface will be referred to as the "application UI". In this embodiment, the application execution unit 401 displays images of the items stored in the refrigerator 2 by displaying the application UI. The application execution unit 401 receives various inputs from user P via the application UI.

[0081] [1-2. Operation] Next, the operation of each part of the management system 1 according to this embodiment will be described.

[0082] [1-2-1. Actions related to recording] Next, we will explain the operation of the recording device 3 related to recording. Figures 11 and 12 are flowcharts showing the operation of the imaging device 3.

[0083] If the SoC board 54 is high-performance, the imaging device 3 performs the operations shown in Figure 11 as part of the recording operations. Here, high performance refers to the ability to simultaneously capture images with the wide-angle camera 55 and the narrow-angle camera 56, and to simultaneously record the images captured by the wide-angle camera 55 and the narrow-angle camera 56. Furthermore, if the SoC board 54 is of low performance, the imaging device 3 performs the operation shown in Figure 12 as part of the recording operation. Here, low performance refers to a performance level that makes it impossible to simultaneously capture images with the wide-angle camera 55 and the narrow-angle camera 56, and also impossible to simultaneously record the images captured by the wide-angle camera 55 and the narrow-angle camera 56.

[0084] First, with reference to Figure 11, the operation of the imaging device 3 equipped with a high-performance SoC substrate 54 will be described.

[0085] The opening / closing determination unit 303 determines whether at least one of the door 21A and the drawer 26A has changed from a closed state to an open state (step SA1).

[0086] If the opening / closing determination unit 303 determines that at least one of the door 21A and the drawer 26A has not changed from the closed state to the open state (step SA1: NO), the determination in step SA1 is performed again.

[0087] If the opening / closing determination unit 303 determines that at least one of the door 21A and the drawer 26A has changed from a closed state to an open state (step SA1: YES), the shooting control unit 304 causes the wide-angle camera 55 and the narrow-angle camera 56 to start shooting (step SA2). From step SA2 onward, the shooting control unit 304 causes the wide-angle camera 55 and the narrow-angle camera 56 to shoot simultaneously. Note that simultaneous shooting means that the shooting timing of the wide-angle camera 55 and the shooting timing of the narrow-angle camera 56 may overlap.

[0088] Next, the recording control unit 305 starts recording the shooting results from the wide-angle camera 55 and the narrow-angle camera 56 (step SA3).

[0089] Next, the opening / closing determination unit 303 determines whether the door 21A and the drawer 26A are in the closed state (step SA4). If the opening / closing determination unit 303 determines that the door 21A and the drawer 26A are not in the closed state (step SA4: NO), it performs the determination in step SA4 again.

[0090] On the other hand, if it is determined that the door 21A and the drawer 26A are closed (step SA4: YES), the recording control unit 305 terminates recording of the shooting results (step SA5). As a result, new recording data 313 is generated in the third memory 310.

[0091] Next, the shooting control unit 304 instructs the wide-angle camera 55 and the narrow-angle camera 56 to terminate shooting (step SA6).

[0092] Next, with reference to Figure 12, the operation of the imaging device 3 equipped with a low-performance SoC substrate 54 will be described. In Figure 12, the same reference numerals are used for the same steps as in Figure 11. Furthermore, in the explanation of each step in Figure 12, steps similar to those in Figure 11 are omitted as appropriate.

[0093] If the opening / closing determination unit 303 determines that at least one of the door 21A and the drawer 26A has changed from a closed state to an open state (step SA1: YES), the shooting control unit 304 causes the wide-angle camera 55 or the narrow-angle camera 56 to start shooting (step SB1). The camera 58 that starts shooting in step SB1 may be the wide-angle camera 55 or the narrow-angle camera 56.

[0094] Next, the recording control unit 305 starts recording the results captured by the camera 58 (step SA3).

[0095] Next, the shooting control unit 304 determines whether or not to switch the camera used for shooting (step SB2).

[0096] Step SB2 will be described in detail. The shooting control unit 304 determines that if the camera 58 currently taking pictures is the wide-angle camera 55, and after α frames have been taken since the wide-angle camera 55 started shooting, it will switch the camera 58 taking pictures to the narrow-angle camera 56. Furthermore, if the camera 58 currently taking pictures is the narrow-angle camera 56, the shooting control unit 304 determines that it should switch the camera 58 to the wide-angle camera 55 after β images have been taken since the narrow-angle camera 56 started taking pictures.

[0097] The relationship between α and β described above may be either "α>β" or "α=β", but in this embodiment, the case where "β>α" is used as an example. In other words, in this embodiment, when at least one of the door 21A and the drawer 26A is open, the number of times the narrow-angle camera 56 takes pictures per unit time is greater than the number of times the wide-angle camera 55 takes pictures.

[0098] If the shooting control unit 304 determines that it does not need to switch the camera 58 that will be taking pictures (step SB2: NO), it performs the determination in step SB2 again.

[0099] On the other hand, if the shooting control unit 304 determines that it is necessary to switch the camera 58 to be used for shooting (step SB2: YES), it switches the camera 58 to be used for shooting (step SB3) and starts shooting with the switched camera 58 (step SB4).

[0100] Next, the opening / closing determination unit 303 determines whether the door 21A and the drawer 26A are in the closed state (step SB5). If the opening / closing determination unit 303 determines that the door 21A and the drawer 26A are not in the closed state (step SB5: NO), it performs the determination in step SB2 again.

[0101] On the other hand, if it is determined that the door 21A and the drawer 26A are closed (step SB5: YES), the recording control unit 305 terminates recording of the shooting results (step SA5).

[0102] Furthermore, the second processor 200 lights up the first LED 48 and the second LED 49 when the imaging device 3 starts taking images, and turns off the first LED 48 and the second LED 49 when the imaging device 3 finishes taking images. Furthermore, the second processor 200 outputs a sound via the buzzer 50 while the camera 3 is taking pictures. This increases the likelihood that user P will be able to move away from the refrigerator 2, and allows the camera 3 to take pictures of the inside of the refrigerator 2 without capturing user P in the images.

[0103] [1-2-2. Operations related to the transmission of management data] Next, we will explain the operations involved in transmitting management data D1. Figure 13 is a flowchart showing the operation of the imaging device 3 and the management server 5. In Figure 13, the flowchart on the left shows the operation of the imaging device 3, and the flowchart on the right shows the operation of the management server 5.

[0104] The generation unit 306 determines whether or not the recorded data 313 has been newly stored in the third memory 310 (step SC1). If the generation unit 306 determines that the recorded data 313 has not been newly stored (step SC1: NO), it performs the determination in step SC1 again.

[0105] On the other hand, if the generation unit 306 determines that the recording data 313 has been stored (step SC1: YES), it selects a first captured image from the video images indicated by the first recording data of the recording data 313 stored in the third memory 310 (step SC2).

[0106] Step SC2 will be described in detail. The first captured image is one taken when the right door 21B and the left door 21C are open at a rotation angle of α°≦θ≦β°, and is the image in which the user P is least visible. α° is, for example, 60°. β° is, for example, 100°.

[0107] The generation unit 306 selects a captured image that shows the first marker MK1 attached inside the refrigerator compartment 21 from the moving images shown in the first recorded data. The first marker MK1 is pre-attached so that it appears in the captured image when the right door 21B and the left door 21C are α°≦θ≦β°. Next, the generation unit 306 selects the captured image with the least amount of user P reflected in it from the selected captured images as the first captured image. The generation unit 306 recognizes user P appearing in the captured image by methods such as pattern matching, and selects the captured image with the smallest area of ​​the recognized user P's image as the captured image with the least amount of user P reflected in it.

[0108] Returning to the flowchart explanation, the generation unit 306 selects a second captured image from the video images indicated by the second recorded data of the stored recorded data 313 (step SC3).

[0109] Step SC3 will be described in detail. The second captured image is one in which the extension 26A has an extension of γ% or more, with the extension degree in the closed state being 0% and the extension degree in the fully open state being 100%. Note that the extension degree refers to the degree to which it is extended. The second captured image is also the image in which user P is least visible. γ% is, for example, 70%.

[0110] The generation unit 306 selects a captured image from the moving images shown in the second recorded data that shows the second marker MK2 attached inside the drawer 26A. The second marker MK2 is pre-attached so that it appears in the captured image when the drawer 26A is pulled out by γ% or more. Next, the generation unit 306 selects the captured image with the least amount of user P reflected in it from the selected captured images as the second captured image.

[0111] The generation unit 306 determines whether or not it was possible to select at least one of the first captured image and the second captured image (step SC4).

[0112] If the generation unit 306 determines that it could not select the first and second captured images (step SC4: NO), it performs the process in step SC1.

[0113] On the other hand, if the generation unit 306 determines that it has been able to select at least one of the first captured image and the second captured image (step SC4: YES), it generates management data D1 (step SC5).

[0114] If a first and second image can be selected, the generation unit 306 generates management data D1 including the image data of the selected first image, identification information of the refrigerator compartment 21 indicated by the first marker MK1 in the selected first image, image data of the selected second image, identification information of the drawer storage compartment 26 indicated by the second marker MK2 in the selected second image, and the imaging device ID 312. If a first image can be selected but a second image cannot be selected, the generation unit 306 generates management data D1 including the image data of the selected first image, identification information of the refrigerator 21 indicated by the first marker MK1 in the selected first image, and the imaging device ID 312. If a second image can be selected but a first image cannot, the generation unit 306 generates management data D1 including the image data of the selected second image, identification information of the drawer storage compartment 26 indicated by the second marker MK2 in the selected second image, and the imaging device ID 312.

[0115] Next, the first communication control unit 301 transmits the management data D1 generated by the generation unit 306 to the management server 5 (step SC6).

[0116] The server communication unit 501 determines whether or not it has received management data D1 from the imaging device 3 (step SD1). If the server communication unit 501 determines that it has received management data D1 (step SD1: YES), the server processing unit 502 updates the management DB 512 based on the management data D1 received by the server communication unit 501 (step SD2).

[0117] Step SD2 will be described in detail. The server processing unit 502 identifies the record containing the camera ID 312 included in the management data D1 from the management DB 512, and updates the image data of the first captured image in the identified record with the image data of the first captured image included in the management data D1. The updated image data of the first captured image is associated with the identification information of the refrigerator room 21 included in the management data D1. The server processing unit 502 identifies the record containing the imaging device ID 312 included in the management data D1 from the management DB 512, and updates the image data of the second image captured in the identified record with the image data of the second image captured included in the management data D1. The updated image data of the second image captured is associated with the identification information of the drawer storage room 26 included in the management data D1.

[0118] [1-2-7. Actions related to the app UI] Next, we will explain the operation of Management System 1 related to the app UI. Figure 14 is a flowchart showing the operation of terminal device 4 and management server 5. In Figure 14, the flowchart on the left shows the operation of terminal device 4, and the flowchart on the right shows the operation of management server 5.

[0119] The application execution unit 401 determines whether or not to request image data stored in the management DB 512 from the management server 5 (step SE1). For example, if the management application 411 is started, the application execution unit 401 makes a positive determination in step SE1.

[0120] If the application execution unit 401 determines that it needs to request image data stored in the management DB 512 (step SE1: YES), it sends request data requesting the image data to the management server 5 (step SE2). The request data includes the camera ID 312 stored in the terminal memory 410. The terminal device 4 acquires the camera ID 312 at a predetermined timing.

[0121] The server communication unit 501 receives the request data (step SF1). Next, the server processing unit 502 retrieves the image data corresponding to the imaging device ID 312 included in the request data received by the server communication unit 501 from the management DB 512 (step SF2). Next, the server communication unit 501 transmits the image data retrieved by the server processing unit 502 to the terminal device 4 as a response to the request data (step SF3).

[0122] The application execution unit 401 receives image data from the management server 5 (step SE3). Next, the application execution unit 401 displays the application UI, which includes the image indicated by the image data received in step SE3, on the touch panel 430 (step SE4).

[0123] [1-3. Effects, etc.] As described above, the imaging device 3 photographs the refrigerator 2 from the front and above. The imaging device 3 includes a camera 58 for photographing the refrigerator 2, and a thermopile 45 for detecting the opening and closing of the right door 21B located in the refrigerator compartment 21 of the refrigerator 2, and the opening and closing of the drawer 26A located in the drawer storage compartment 26 of the refrigerator 2. The camera 58 starts taking photographs when the thermopile 45 detects that at least one of the right door 21B and the drawer 26A is open.

[0124] According to this, since the thermopile 45 detects the opening and closing of the right door 21B and the drawer 26A, the number of parts can be reduced compared to a case where sensors are provided to detect each of the right door 21B and the drawer 26A. In addition, since the camera 58 starts taking pictures when the thermopile 45 detects that at least one of the right door 21B and the drawer 26A is open, the camera can start taking pictures of the inside of the refrigerator 2 at the appropriate time. Thus, the number of parts can be reduced and the inside of the refrigerator 2 can be taken pictures at the appropriate time.

[0125] With the camera 3 installed in the refrigerator 2, the thermopile 45 is positioned such that, when viewed from above, at least half of the thermopile 45 is located in front of the closed right door 21B.

[0126] According to this, even when the right door 21B is closed, the thermopile 45 can detect that the drawer 26A has opened.

[0127] The imaging device 3 is equipped with a distance measuring sensor 44 that detects the opening and closing of the left door 21C. The camera 58 starts taking pictures when the distance measuring sensor 44 detects that the left door 21C is open.

[0128] According to this, the camera 58 starts taking pictures when it detects that the left door 21C is open. Therefore, the photographing device 3 can reduce the number of parts and start taking pictures of the inside of the refrigerator 2 at the appropriate time, even if the refrigerator 2 being photographed has two doors.

[0129] The thermopile 45 is located in the width direction of the refrigerator 2, on the hinge member side 21B1 of the right door 21B from the center CT of the refrigerator 2. The distance measuring sensor 44 is located in the width direction of the refrigerator 2, on the hinge member side 21C1 of the left door 21C from the center CT of the refrigerator 2.

[0130] According to this, since the thermopile 45 and the distance sensor 44 can each detect opening and closing at a position close to the object to be detected, it is possible to suppress false detection of opening and closing by the thermopile 45 and the distance sensor 44.

[0131] With the imaging device 3 installed in the refrigerator 2, the sensing direction of the thermopile 45 is directed toward the central CT side of the refrigerator 2 when viewed from the front.

[0132] According to this, when viewing the refrigerator 2 from the front, the sensing direction of the thermopile 45 is directed toward the central CT side of the refrigerator 2. Therefore, even if the refrigerator 2 installed in the imaging device 3 is equipped with drawers 26A arranged in the width direction of the refrigerator 2, the thermopile 45 will be able to detect the open state of the drawers 26A arranged in the width direction of the refrigerator 2.

[0133] The imaging device 3 includes a wide-angle camera 55 for imaging the refrigerator compartment 21, a narrow-angle camera 56 for imaging the drawer 26A, and an SoC board 54 that controls the wide-angle camera 55 and the narrow-angle camera 56. The SoC board 54 causes the wide-angle camera 55 to take an image, then the narrow-angle camera 56 to take an image, or vice versa.

[0134] According to this, the SoC board 54 does not allow the wide-angle camera 55 and the narrow-angle camera 56 to take pictures simultaneously, thereby reducing the processing load on the SoC board 54 related to the taking of pictures by the wide-angle camera 55 and the narrow-angle camera 56.

[0135] The SoC board 54 controls the wide-angle camera 55 and the narrow-angle camera 56 so that the number of times the narrow-angle camera 56 takes pictures is greater than the number of times the wide-angle camera 55 takes pictures during the period from when the camera 58 starts taking pictures until it finishes taking pictures.

[0136] According to this, during the period from when camera 58 starts shooting until it finishes shooting, more images can be obtained from the narrow-angle camera 56 than from the wide-angle camera 55. When the shooting device 3 is installed in the refrigerator 2, the distance between the shooting device 3 and the drawer 26A is greater than the distance between the shooting device 3 and the refrigerator compartment 21. Therefore, the size of the drawer 26A captured by the narrow-angle camera 56 is smaller than the size of the refrigerator compartment 21 captured by the wide-angle camera 55. Also, the image quality of the drawer 26A captured by the narrow-angle camera 56 is lower than the image quality of the refrigerator compartment 21 captured by the wide-angle camera 55. Therefore, by making it possible to obtain more images from the narrow-angle camera 56 than from the wide-angle camera 55, as many images as possible can be processed in the processing that uses the images from the narrow-angle camera 56, and a decrease in the processing accuracy of the processing that uses the images from the narrow-angle camera 56 can be suppressed.

[0137] (Embodiment 2) Next, Embodiment 2 will be described. The description of Embodiment 2 will mainly focus on the differences from Embodiment 1. Furthermore, in the description of Embodiment 2, components identical to those in the management system 1 of Embodiment 1 will be denoted by the same reference numerals, and detailed descriptions will be omitted as appropriate.

[0138] [2-1. Structure] Figure 15 is a plan view of the installed camera 3, seen from above. Figure 16 is a plan view of the refrigerator 2 and the camera 3 installed on the refrigerator 2, seen from the front. In Figure 16, the door 21A of the refrigerator 2 is closed. Figure 17 is a plan view of the refrigerator 2 and the camera 3 installed on the refrigerator 2, seen from the right. In Figure 17, the drawer 26A of the refrigerator 2 is closed, i.e., not pulled out.

[0139] Figures 15-17 each illustrate the XYZ axes. These three axes are the same as those illustrated in Figures 2-8.

[0140] In Embodiment 2, compared to Embodiment 1, the imaging device 3 does not have a distance measuring sensor 44, but it does have a thermopile 62. In this embodiment, the thermopile 62 is an example of a "second sensor".

[0141] The thermopile 62 is a sensor that detects the opening and closing of the left door 21C and the drawer 26A. When the imaging device 3 is installed, the thermopile 62 is located to the left of the central CT in the width direction of the refrigerator 2. That is, when the imaging device 3 is installed, the thermopile 62 is located on the hinge member 21C1 side of the left door 21C, beyond the central CT, in the width direction of the refrigerator 2. Furthermore, when the imaging device 3 is installed, the thermopile 62 is positioned such that, when viewed from above the refrigerator 2, at least half of the thermopile 62 is located in front of the closed left door 21C. Note that when viewed from above the refrigerator 2, the thermopile 62 may be located between the left door 21C and the right door 21B. Here, "at least half of the thermopile 62 is located in front of the closed left door 21C" means that at least half of the infrared detection elements of the thermopile 62 are positioned so as not to overlap with the refrigerator 2.

[0142] When the imaging device 3 is installed, the thermopile 62 senses the area below the refrigerator 2. The thermopile 62 is positioned so that, when viewed from the front of the refrigerator 2, its sensing direction faces the central CT side of the refrigerator 2. The sensing range of the thermopile 62 includes the range A7 shown in Figure 16 when viewed from the front of the refrigerator 2. Range A7 has a sensing angle of θ3 and extends in the vertical direction. Range A7 also includes a portion of the closed left door 21C and a portion of the closed drawer 26A. Furthermore, the thermopile 62 is positioned so that, when viewed from the side of the refrigerator 2, its sensing direction is tilted from the rear to the front of the refrigerator 2. The sensing range of the thermopile 62 includes the range A8 shown in Figure 17 when viewed from the right side of the refrigerator 2. Range A8 extends approximately parallel to the Z-axis and has a sensing angle of θ4. Range A8 is the area in front of the refrigerator 2 when the door 21A and drawer 26A are closed, and is the area in which the top surface of the open drawer 26A can be detected. Range A8 is also the area in front of the refrigerator 2 when the door 21A and drawer 26A are closed, and is the area in which the door pocket 21D of the open left door 21C can be detected. Furthermore, because range A8 is the area in front of the refrigerator 2 when the door 21A and drawer 26A are closed, the thermopile 62 can detect the temperature of a hand touching the refrigerator 2.

[0143] The thermopile 62 detects the temperature of a portion of the floor FL, FLA1, when the left door 21C and drawer 26A are closed and there are no objects within the sensing range. Furthermore, the thermopile 62 detects the temperature of the door pocket 21D of the left door 21C if the door pocket 21D of the left door 21C is within its sensing range. Generally, the temperature of the door pocket 21D is lower than the temperature of area FLA1. Furthermore, the thermopile 62 detects the temperature inside drawer 26A if it is within its sensing range. Generally, the temperature inside drawer 26A is lower than the temperature in region FLA1. Furthermore, the thermopile 62 detects the temperature of user P's hand if the user P's hand, which is attempting to open door 21A or drawer 26A, is within its sensing range. Generally, the temperature of user P's hand is higher than the temperature of region FLA1. Furthermore, the thermopile 62 detects the temperature of food items being moved in and out of refrigerator 2 if those items are within its sensing range. Generally, the temperature of food items being moved in and out of refrigerator 2 is lower than the temperature of area FLA1.

[0144] Figure 18 is a block diagram showing the configuration related to the control system of the imaging device 3. In contrast to Embodiment 1, in Embodiment 2, a thermopile 62 is connected to the sensor board 43 instead of the distance measuring sensor 44.

[0145] Compared to Embodiment 1, the third processor 300 in Embodiment 2 functions as an open / close determination unit 303A instead of the open / close determination unit 303.

[0146] The open / close determination unit 303A determines whether at least one of the left door 21C and the drawer 26A has changed from a closed state to an open state based on the value detected by the thermopile 62 received by the second communication control unit 302. The open / close determination unit 303A determines that at least one of the left door 21C and the drawer 26A has changed from a closed state to an open state if the value detected by the thermopile 62 has risen to a predetermined temperature or higher.

[0147] For example, suppose the temperature of region FLA1 is 20°C, the temperature of the door pocket 21D of the left door 21C is 5°C, and the predetermined temperature detected by the thermopile 62 is 5°C. In this example, if the door pocket 21D of the left door 21C is included in the sensing range of the thermopile 62, the temperature detected by the thermopile 62 will change by 15°C (=20°C - 5°C). Therefore, in this example, if the door pocket 21D of the left door 21C is included in the sensing range of the thermopile 62, the opening / closing determination unit 303A determines that at least one of the left door 21C and the drawer 26A has changed from a closed state to an open state.

[0148] The opening / closing determination unit 303A determines whether at least one of the right door 21B and the drawer 26A has changed from a closed state to an open state. This determination is the same as in Embodiment 1.

[0149] The opening / closing determination unit 303A determines whether the door 21A and the drawer 26A have changed from the open state to the closed state. This determination is the same as in Embodiment 1.

[0150] Furthermore, the shooting control unit 304 controls the camera 58 in the same manner as in Embodiment 1, based on the determination result of the opening / closing determination unit 303A.

[0151] [2-2. Operation] Next, the operation of each part of the management system 1 in Embodiment 2 will be described. The operation of each part of the management system 1 in Embodiment 2 is the same as in Embodiment 1.

[0152] [2-3. Effects, etc.] According to Embodiment 2, the same effects as in Embodiment 1 are achieved.

[0153] Furthermore, the imaging device 3 of the second embodiment includes a thermopile 62 that detects the opening and closing of the left door 21C and the drawer 26A. The camera 58 starts imaging when the thermopile 62 detects that at least one of the left door 21C and the drawer 26A is open.

[0154] According to this configuration, the camera 58 starts taking pictures when it detects that the left door 21C is open. Therefore, even if the refrigerator 2 being photographed has two doors, the photographing device 3 can reduce the number of parts and start photographing the inside of the refrigerator 2 at the appropriate time. Furthermore, since the opening and closing of the drawer 26A is detected by two thermopiles 62, the open state of the drawer 26A can be detected with high accuracy, allowing for even more appropriate timing to start photographing the inside of the refrigerator 2.

[0155] (Other embodiments) As described above, Embodiments 1 and 2 have been explained as examples disclosed in this application. However, the technology in this disclosure is not limited thereto and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to combine the components described in Embodiments 1 and 2 to create new embodiments. Therefore, other embodiments are described below as examples.

[0156] In embodiments 1 and 2 described above, the camera 3 is configured to stop shooting when the door 21A and drawer 26A are closed. In other embodiments, along with or in addition to this configuration of embodiments 1 and 2, the camera 3 may stop shooting after a predetermined period (for example, 30 seconds) has elapsed since the camera started shooting. This allows the camera 3 to stop shooting even if it mistakenly starts shooting because a person or an animal other than a person momentarily enters the sensing range of the thermopiles 45 and 62.

[0157] In embodiments 1 and 2 described above, the imaging device 3 is configured to stop imaging when the door 21A and drawer 26A are closed. In another embodiment described above, imaging is configured to stop after a predetermined period of time has elapsed since the start of imaging. In another embodiment, along with or in addition to these configurations, the imaging device 3 may stop imaging after a predetermined period of time has elapsed since the marker MK is no longer visible in the images taken by the wide-angle camera 55 and the narrow-angle camera 56. This ensures that imaging is terminated when the door 21A and drawer 26A are reliably closed.

[0158] In the above-described Embodiment 1, a thermopile 45 was exemplified as the "first sensor" and a distance measuring sensor 44 was exemplified as the "second sensor". However, in other embodiments relating to Embodiment 1, the "first sensor" may be a distance measuring sensor, and the "second sensor" may be a thermopile. When the "first sensor" is a distance measuring sensor, it is preferable that the "first sensor" is located in front of the closed right door 21B in the installed state, similar to the thermopile 45. Also, when the "second sensor" is a thermopile, it is preferable that the "second sensor" is located in the same position as in Embodiment 2.

[0159] In the embodiments described above, the "first sensor" and the "second sensor" were exemplified as sensors that detect the opening and closing of the corresponding opening / closing body, that is, sensors that detect the open state and the closed state. However, the "first sensor" and the "second sensor" only need to be sensors that detect at least the open state of the corresponding opening / closing body.

[0160] In the above-described embodiment 2, a thermopile 45 was exemplified as the "first sensor" and a thermopile 62 was exemplified as the "second sensor". However, in other embodiments relating to embodiment 2, at least one of the "first sensor" and the "second sensor" may be a distance measuring sensor. In this case, it is preferable that at least one of the distance measuring sensors, the "first sensor" and the "second sensor," is located in front of the closed door 21A when installed.

[0161] In the embodiments 1 and 2 described above, examples were given in which the thermopiles 45 and 62 are located in front of the closed door 21A. In other embodiments, when the refrigerator 2 is viewed from above in its installed state, the thermopiles 45 and 62 may be positioned so as to partially overlap the closed door 21A.

[0162] In embodiments 1 and 2 described above, the sensor board 43 and the SoC board 54 are separate boards. In other embodiments, the sensor board 43 and the SoC board 54 may be made of the same board. This reduces the number of wires in the imaging device 3, thereby reducing the number of components. In addition, since the number of wires connecting the boards can be reduced, the generation of noise within the imaging device 3 can be suppressed.

[0163] In the embodiment described above, a refrigerator 2 was illustrated in which the widths of the right door 21B and the left door 21C are different. In other embodiments, the widths of the right door 21B and the left door 21C may be the same, or the left door 21C may be wider than the right door 21B.

[0164] In embodiments 1 and 2 described above, the refrigerator 2 and the imaging device 3 are separate components. In other embodiments, the refrigerator 2 may include the imaging device 3. That is, in other embodiments, the refrigerator 2 and the imaging device 3 may be integrated into a single unit. Even in this case, the same effects as in embodiments 1 and 2 are achieved.

[0165] In another embodiment, instead of the recorded video data 313, still images captured by each of the cameras 58 may be stored in the third memory 310. In this other embodiment, there may be multiple captured images for each of the cameras 58 stored in the third memory 310.

[0166] In the above-described embodiment 1, the first LED 48, thermopile 45, wide-angle camera 55, narrow-angle camera 56, and second LED 49 are arranged in this order at the front end of the imaging device. However, this arrangement is merely an example, and the arrangement of these devices is not limited to embodiments 1 and 2.

[0167] In other embodiments, the camera 58 may be configured to stop recording or taking pictures after a predetermined time has elapsed since the door 21A or drawer 26A was opened. For example, the predetermined time is 1 minute. However, if it is detected that the door 21A or drawer 26A is open, that a person is present around the refrigerator 2, or that user P is taking items in or out, the predetermined time may be extended. Furthermore, if it is detected that the door 21A or drawer 26A is open, that a person is present around the refrigerator 2, or that user P is taking items in or out, the camera 58 may stop recording or taking pictures after a predetermined time has elapsed since the detection.

[0168] In other embodiments, in addition to the functional units of the third processor 300 in embodiments 1 and 2, the third processor 300 may also function as a detection unit for detecting articles.

[0169] In other embodiments, in addition to the functional units of the third processor 300 in embodiments 1 and 2, the third processor 300 may also function as a person presence determination unit. In this case, the shooting device 3 is equipped with a human presence sensor. The person presence determination unit determines whether or not a person is present around the refrigerator 2 based on the detection value of the human presence sensor. More specifically, the person presence determination unit determines that a person is present around the refrigerator 2 if the detection value of the human presence sensor corresponds to the detection value of a person, and determines that no person is present around the refrigerator 2 otherwise. When the third processor 300 functions as a person presence determination unit, the shooting control unit 304 causes at least one of the wide-angle camera 55 and the narrow-angle camera 56 to start shooting if the person presence determination unit determines that a person is present, and causes the wide-angle camera 55 and the narrow-angle camera 56 to stop shooting if the person presence determination unit determines that no person is present. Furthermore, when the third processor 300 functions as a person presence detection unit, the recording control unit 305 starts recording when the person presence detection unit determines that a person is present, and stops recording when the person presence detection unit determines that no person is present. This ensures that even if the opening / closing detection unit 303 makes a false determination, the shooting results from the wide-angle camera 55 and the narrow-angle camera 56 can be properly recorded.

[0170] In other embodiments, in addition to the functional units of the third processor 300 in embodiments 1 and 2, the third processor 300 may also function as an in / out determination unit. In this case, the in / out determination unit changes the boundary line used for determining in / out according to the opening and closing angles of the left door 21C and the right door 21B, and determines the in / out of goods. Furthermore, the in / out determination unit changes the boundary line used for determining in / out according to the degree to which the drawer 26A is pulled out, and determines the in / out of goods.

[0171] In other embodiments, the first captured image may be image-corrected and uploaded to the management server 5. An example of image correction is correction of viewpoint transformation.

[0172] In the embodiment described above, an example was given in which the first and second captured images are used for display purposes on the terminal device 4. However, the first and second captured images may also be used for other purposes, such as detecting items stored in the refrigerator 2.

[0173] In other embodiments, the main body 20 of the refrigerator 2 may have more or fewer storage compartments than the five types described above. Also, in other embodiments, the number of doors (rotary opening / closing bodies) provided at the opening of the refrigerator compartment 21 may be just one. If there is only one rotary opening / closing body at the opening of the refrigerator compartment 21, the imaging device 3 does not need to be equipped with a distance measuring sensor 44 or a thermopile 62.

[0174] In the embodiment described above, the camera 3 is configured to be applied to the refrigerator 2, but the application of the camera 3 is not limited to the refrigerator 2, and may be to other home appliances, furniture, etc. For example, the camera 3 may be applied to an induction cooktop. In this case, the camera 3 is installed in the range hood of the induction cooktop and detects the cooking state using either the thermopile 45 or 62. For example, by detecting the temperature of the pot and its surroundings using either the thermopile 45 or 62, spills can be detected. When a spill occurs, the temperature around the pot rises rapidly, so by utilizing the detection by the thermopile 45 or 62, cooking can be stopped quickly. Furthermore, for example, the camera 3 may be applied to a cupboard. In this case, the camera 3 can detect the opening and closing of the cupboard door and photograph the items stored in the cupboard.

[0175] The first processor 100, the second processor 200, and the third processor 300 may be composed of a single processor or multiple processors. These processors may also be hardware programmed to implement the corresponding functional units. That is, these processors may be composed of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0176] The configurations of the various parts of the management system 1 shown in Figures 9, 10, and 18 are merely examples, and the specific implementation is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each part be implemented individually; it is also possible to configure the system so that a single processor executes a program to realize the functions of each part. Furthermore, some of the functions realized by software in the above-described embodiment may be implemented by hardware, or vice versa.

[0177] The operational step units shown in Figures 11, 12, 13, and 14 are divided according to the main processing content to facilitate understanding of the operation, and the operation is not limited by the way the processing units are divided or the names of the processing units. Depending on the processing content, it may be further divided into more step units. Alternatively, it may be divided so that one step unit includes even more processing. Furthermore, the order of the steps may be changed as appropriate, as long as it does not hinder the intent of this disclosure.

[0178] Since the embodiments described above are for illustrative purposes of the technology described herein, various modifications, substitutions, additions, omissions, etc., can be made within the claims or their equivalents.

[0179] (Note) Based on the above description of embodiments, the following technologies are disclosed.

[0180] (Technology 1) A photographing device for photographing a refrigerator from the front and above, comprising: a camera for photographing the refrigerator; and a first sensor for detecting at least an open state of a rotary opening / closing body provided in a first storage compartment of the refrigerator, and at least an open state of a pull-out opening / closing body provided in a second storage compartment of the refrigerator, wherein the camera starts taking photographs when the first sensor detects that at least one of the rotary opening / closing body and the pull-out opening / closing body is open. According to this, since the first sensor detects the opening and closing of the rotary opening / closing body and the pull-out opening / closing body, the number of parts can be reduced compared to a case where there are separate sensors to detect each of the rotary opening / closing body and the pull-out opening / closing body. Also, since the camera starts taking pictures when the first sensor detects that at least one of the rotary opening / closing body and the pull-out opening / closing body is open, the camera can start taking pictures of the inside of the refrigerator at the appropriate time. Thus, the number of parts can be reduced and the inside of the refrigerator can be taken pictures at the appropriate time.

[0181] (Technology 2) The imaging device according to Technology 1, wherein, when the imaging device is installed in the refrigerator, the first sensor is positioned such that, when viewed from above the refrigerator, at least half of the first sensor is located in front of the closed rotating opening / closing body. According to this, even when the rotary opening / closing mechanism is in the closed position, the first sensor can detect that the pull-out opening / closing mechanism has opened.

[0182] (Technology 3) The first storage chamber is provided with two rotary opening / closing bodies, a first rotary opening / closing body and a second rotary opening / closing body, and is equipped with a second sensor that detects at least the open state of the second rotary opening / closing body, the first sensor detects at least the open state of the first rotary opening / closing body and at least the open state of the pull-out opening / closing body, and the camera starts taking pictures when the second sensor detects the open state of the second rotary opening / closing body, according to the imaging device according to Technology 1 or Technology 2. According to this, the camera starts taking pictures when it detects that the second rotary opening / closing body is open. Therefore, even if the refrigerator being photographed has two rotary opening / closing bodies, the photographing device can reduce the number of parts and start photographing the inside of the refrigerator at the appropriate time.

[0183] (Technology 4) The first storage chamber is provided with two rotary opening / closing bodies, a first rotary opening / closing body and a second rotary opening / closing body, and is equipped with a second sensor that detects at least the open state of the second rotary opening / closing body and at least the open state of the pull-out opening / closing body, the first sensor detects at least the open state of the first rotary opening / closing body and at least the open state of the pull-out opening / closing body, and the camera starts taking pictures when the second sensor detects that at least one of the second rotary opening / closing body and the pull-out opening / closing body is open, as described in Technology 1 or Technology 2. According to this design, the camera starts taking pictures when it detects that the second rotary opening / closing body is open. Therefore, even if the refrigerator being photographed has two rotary opening / closing bodies, the photographing device can reduce the number of parts and start taking pictures of the inside of the refrigerator at the appropriate time. Furthermore, since the configuration uses two sensors to detect the opening and closing of the pull-out opening / closing body, the open state of the pull-out opening / closing body can be detected with high accuracy, allowing for even more appropriate timing to start taking pictures of the inside of the refrigerator.

[0184] (Technology 5) The imaging device according to Technology 3 or Technology 4, wherein, when the imaging device is installed in the refrigerator, the first sensor is located in the width direction of the refrigerator toward the hinge member of the first rotary opening / closing body from the center of the refrigerator, and the second sensor is located in the width direction of the refrigerator toward the hinge member of the second rotary opening / closing body from the center of the refrigerator. According to this, since the first sensor and the second sensor can each detect opening and closing at a position close to the object being detected, it is possible to suppress false detections of opening and closing by the first sensor and the second sensor.

[0185] (Technology 6) The imaging device according to Technical Reference 5, wherein, when the imaging device is installed in the refrigerator, the sensing direction of the first sensor is directed toward the center of the refrigerator when viewed from the front. According to this, when viewing the refrigerator from the front, the sensing direction of the first sensor is directed towards the center of the refrigerator. Therefore, even if the refrigerator installed in the imaging device has pull-out drawers arranged in the width direction of the refrigerator, the first sensor will be able to detect the open state of the pull-out drawers arranged in the width direction of the refrigerator.

[0186] (Technology 7) A photographing device according to any one of the technologies 1 to 6, comprising: a first camera which is the camera for photographing the first containment chamber; a second camera which is the camera for photographing the pull-out type opening / closing body; and a camera board which controls the first camera and the second camera, wherein the camera board causes the first camera to take a photograph and then the second camera to take a photograph, or the second camera to take a photograph and then the first camera to take a photograph. According to this, the camera board prevents the first camera and the second camera from capturing images simultaneously, thus reducing the processing load on the camera board involved in capturing images with the first and second cameras.

[0187] (Technology 8) The imaging apparatus according to Technical Reference 7, wherein the camera substrate controls the first camera and the second camera such that the number of times the second camera takes pictures is greater than the number of times the first camera takes pictures during the period from when the camera starts taking pictures until it stops taking pictures. According to this, more images can be obtained from the second camera than from the first camera during the period from when the camera starts shooting until it stops shooting. When the shooting device is installed in a refrigerator, the distance between the shooting device 3 and the pull-out opening / closing body is generally greater than the distance between the shooting device and the first storage compartment. Therefore, the size of the pull-out opening / closing body captured by the second camera is smaller than that of the first storage compartment captured by the first camera. Also, the image quality of the pull-out opening / closing body captured by the second camera is lower than that of the first storage compartment captured by the first camera. Therefore, by making it possible to obtain more images from the second camera than from the first camera, as many images as possible can be processed in the processing that uses the images from the second camera, and a decrease in the processing accuracy of the processing that uses the images from the second camera can be suppressed. [Industrial applicability]

[0188] As described above, the photographic device according to the present invention can be used for photographing a refrigerator from the front and above. [Explanation of Symbols]

[0189] 1 Management System 2 Refrigerator 3. Imaging device 4 Terminal devices 5. Management Server 6. Communication equipment 20 Main box 20A Top 20B Front edge 21. Refrigerated Room (Storage Room 1) 21A Door (rotary opening / closing mechanism) 21B Right door (first rotation type opening / closing mechanism) 21B1, 21C1 Hinge Members 21C Left door (second rotating opening / closing mechanism) 21D Door Pocket 21E shelf 22 Ice maker 23 Fresh freezing room 24 Freezer 25 Vegetable compartment 26. Drawer storage room (Second storage room) 22A~25A drawer 26A drawer (pull-out type opening / closing body) 30 Case components 31 Cover component 32 Base 33 Photography Department 34 Bottom 35 Legs 36 Front end surface 37 Position regulation part 38 Mark 39 Control section 39A First button 39B Second button 39C Third Button 40 Power supply board 41 Power harness 43 Sensor board 44 Distance measuring sensor (second sensor) 45 Thermopile (1st sensor) 46, 47, 51, 52, 53 Cables 48 1st LED 49. Second LED 50 Buzzer 54 SoC board (camera board) 55 Wide-angle camera (first camera) 56 Narrow-angle camera (second camera) 57 FFC 58 Cameras 59 Window section 60 Operation display board 61 Wireless communication module 62 Thermopile (Second Sensor) 100 First Processor 110 First Memory 111, 211, 311, 511 control programs 120 First Communication Module 200 Second Processor 210 Second Memory 220 Second communication module 300 Third Processor 301 First Communication Control Unit 302 Second Communication Control Unit 303, 303A open / close judgment section 304 Imaging Control Unit 305 Recording Control Unit 306 Generation part 310 Third Memory 312 Imaging device ID 313 Recorded data 320 Third Communication Module 400 terminal processors 401 Application Execution Unit 410 Terminal memory 411 Management App 420 Terminal communication module 430 Touch Panel 500 Server Processors 501 Server Communications Unit 502 Server Processing Unit 510 Server Memory 512 Management DB 520 Server Communication Module Range A1-A8 CT center D1 Management Data FL floor FLA, FLA1 area H Home MK Marker MK1 First Marker MK2 Second Marker NW Network

Claims

1. A photographic device for photographing the refrigerator from the front and above, A camera for photographing the aforementioned refrigerator, The refrigerator comprises a first sensor that detects at least the open state of a rotary opening / closing body provided in the first storage compartment of the refrigerator, and at least the open state of a pull-out opening / closing body provided in the second storage compartment of the refrigerator. The camera starts taking pictures when the first sensor detects that at least one of the rotary opening / closing body and the pull-out opening / closing body is in an open state. A photographic device.

2. With the aforementioned imaging device installed in the refrigerator, The first sensor is positioned such that, when viewed from above the refrigerator, at least half of the first sensor is located in front of the closed rotary opening / closing body. The imaging apparatus according to claim 1.

3. The first containment chamber is provided with two rotary opening / closing bodies, a first rotary opening / closing body and a second rotary opening / closing body. The second rotary opening / closing body is equipped with a second sensor that detects at least the open state, The first sensor detects at least the open state of the first rotary opening / closing body and at least the open state of the pull-out opening / closing body, The camera starts taking pictures when the second sensor detects that the second rotary opening / closing body is in the open state. The imaging apparatus according to claim 1 or 2.

4. The first containment chamber is provided with two rotary opening / closing bodies, a first rotary opening / closing body and a second rotary opening / closing body. The system includes a second sensor that detects at least the open state of the second rotary opening / closing body and at least the open state of the pull-out opening / closing body, The first sensor detects at least the open state of the first rotary opening / closing body and at least the open state of the pull-out opening / closing body, The camera starts taking pictures when the second sensor detects that at least one of the second rotary opening / closing body and the pull-out opening / closing body is in an open state. The imaging apparatus according to claim 1 or 2.

5. With the aforementioned imaging device installed in the refrigerator, The first sensor is located in the width direction of the refrigerator, on the side of the hinge member of the first rotary opening / closing body from the center of the refrigerator, The second sensor is located in the width direction of the refrigerator, on the side of the hinge member of the second rotary opening / closing body from the center of the refrigerator. The imaging apparatus according to claim 3.

6. With the aforementioned imaging device installed in the refrigerator, The sensing direction of the first sensor is such that, when viewed from the front of the refrigerator, it faces towards the center of the refrigerator. The imaging apparatus according to claim 5.

7. The first camera, which is the camera that photographs the first containment chamber, The second camera is the camera that photographs the aforementioned pull-out type opening / closing body, The system comprises a camera board for controlling the first camera and the second camera, The camera board is configured to allow the first camera to take an image, then the second camera to take an image, or to allow the second camera to take an image, then the first camera to take an image. The imaging apparatus according to claim 1 or 2.

8. The camera board controls the first camera and the second camera such that the number of times the second camera takes pictures is greater than the number of times the first camera takes pictures during the period from when the first camera starts taking pictures until it stops taking pictures. The photographic apparatus according to claim 7.

Images