storage facility

Abstract

To provide a storage house that allows a storage room, etc. to be imaged appropriately.SOLUTION: A storage house includes: a housing having a storage room; a door closing the opening of the storage room; a door sensor for detecting opening / closing of the door; and an imaging unit for imaging at least the storage room. The storage house also includes a control unit for executing an imaging command to the imaging unit while an opening operation is detected after the door opening operation is detected by the door sensor, and recognizing an opening degree of the door on the basis of the image captured by the imaging unit. Also, the control unit determines storage or disposal of the latest captured image on the basis of the opening degree of the door.SELECTED DRAWING: Figure 8A

Description

[Technical Field]

[0001] The present disclosure relates to a reservoir. [Background technology]

[0002] There is known a technology for taking pictures of the storage compartment of a refrigerator with a camera. Patent Document 1 discloses that, in principle, the selection of the photographs taken is performed according to the angle of the door (0278), but if a photo selection command is not issued, a first-in, first-out method is used to temporarily store a small number of photographs (0096). It also discloses that, in consideration of cost, the time interval between photographs taken by a camera that takes continuous photographs can be made longer than 1 / 30 seconds (0271, 0274). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] European Patent No. 2909549 (EP, B1) Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 does not disclose in detail the relationship between the setting of the photographing time interval and the timing of selecting the photographs taken. [Means for solving the problem]

[0005] The reservoir according to the present disclosure comprises: a housing having a storage chamber; a door that closes the opening of the storage chamber; a door sensor that detects whether the door is open or closed; and an imaging unit that images at least the storage chamber, After the door opening operation is detected by the door sensor, during the detection of the door opening operation, 、 issuing an imaging command to the imaging unit; a control unit that determines whether to save or discard the latest captured image based on the degree to which the door is opened, and, in accordance with the determination, completes the saving if the image is to be saved so that it can be selected as an image that can be sent to the server, and completes the discarding if the image is to be discarded because it cannot be selected as an image that can be sent to the server, and then repeats the determination of whether to shoot, save, and discard the next image, and the execution of saving or discarding, until the open door is closed; The criteria for the determination are not to unconditionally determine whether to store the latest captured image, but rather there is a possibility of either storing or discarding the image. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a front view of a refrigerator according to a first embodiment. [Figure 2] FIG. 1 is a side view of a refrigerator according to a first embodiment. [Figure 3] FIG. 1 is a front view of a refrigerator according to a first embodiment with the left and right refrigerator compartment doors open. [Figure 4] FIG. 1 is a plan view of a refrigerator according to a first embodiment with the left and right refrigerator compartment doors open. [Figure 5] 1 is a perspective view of a camera unit of a refrigerator according to a first embodiment, viewed from diagonally below. FIG. [Figure 6A] FIG. 2 is a front view of the camera unit of the refrigerator according to the first embodiment. [Figure 6B] 6B is a cross-sectional view of the refrigerator according to the first embodiment taken along the line II-II in FIG. 6A. FIG. [Figure 7] FIG. 1 is a system configuration diagram including a camera unit of a refrigerator according to a first embodiment. [Figure 8A] 10 is a flowchart of an automatic photography process executed by a control unit of the refrigerator according to the first embodiment. [Figure 8B] 10 is a flowchart of an automatic photography process executed by a control unit of the refrigerator according to the first embodiment. [Figure 9] 2A to 2C are explanatory diagrams showing in time sequence the process of opening and closing the refrigerator compartment door of the refrigerator according to the first embodiment. [Figure 10] 10 is an example of a photograph taken by a camera unit of a refrigerator according to the first embodiment, in a state where no image processing has been performed. [Figure 11] 10 is an example of a photograph taken by a camera unit of the refrigerator according to the first embodiment, in which image processing has been performed. [Figure 12A] FIG. 2 is an explanatory diagram showing candidates for areas that serve as a reference when adjusting exposure in the refrigerator according to the first embodiment. [Figure 12B]FIG. 10 is an explanatory diagram showing a reference area for adjusting exposure in a refrigerator according to a comparative example. [Figure 13] 10A and 10B are a front view and a bottom view of left and right refrigerator compartment doors of a refrigerator according to a second embodiment. [Figure 14A] FIG. 10 is a cross-sectional perspective view of the lower part of the right-side refrigerator compartment door of the refrigerator according to the second embodiment. [Figure 14B] FIG. 10 is a cross-sectional view of the lower part of the right-side refrigerator compartment door of the refrigerator according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] First Embodiment FIG. 1 is a front view of a refrigerator 100 according to the first embodiment. Refrigerator 100 is an appliance for cooling food and the like, and includes a housing 1, doors such as refrigerator compartment doors 211 and 212, and a camera unit 3. Housing 1 has multiple storage compartments inside. In the example of Fig. 1, the storage compartments of refrigerator 100 are, from top to bottom, refrigerator compartment 21, ice making compartment 22 and upper freezer compartment 23 lined up on the left and right, as well as vegetable compartment 24 and lower freezer compartment 25.

[0008] The housing 1 is configured such that an outer box 11 made of steel plate and an inner box (not shown) made of resin are separated by a heat insulating material (not shown) such as urethane foam. A plurality of openings 10 (see FIGS. 2 and 4) corresponding to each compartment are provided on the front side (front face side) of the housing 1. For example, when food or the like is placed through opening 10 of refrigerator compartment 21, refrigerator compartment doors 211, 212 are opened. When refrigerator compartment doors 211, 212 are closed, opening 10 of refrigerator compartment 21 is blocked. In this way, refrigerator compartment doors 211, 212 (doors) have the function of closing opening 10 of the housing 1. The same applies to the other doors.

[0009] Refrigerator 100 is equipped with left and right refrigerator compartment doors 211, 212 as French-style doors that, together with housing 1, form refrigerator compartment 21. Left refrigerator compartment door 211 (left door) is rotatable around the axis of hinge 211a (see FIG. 4) at the left end. The same is true for right refrigerator compartment door 212 (right door). Refrigerator 100 is also equipped with drawer-type doors, such as ice-making compartment door 221 and upper freezer compartment door 231, as well as vegetable compartment door 241 and lower freezer compartment door 251.

[0010] Refrigerating compartment 21 is provided with a plurality of shelves 213 (see FIG. 3) that divides refrigerating compartment 21 into predetermined sections. The interior of left-side refrigerator compartment door 211 is provided with a plurality of door pockets 211c (see FIG. 3) for storing food and the like (the same applies to right-side refrigerator compartment door 212). Ice-making compartment 22 is provided with an ice-making compartment container (not shown) that is pulled out together with ice-making compartment door 221. Similarly, upper freezer compartment 23 is provided with an upper freezer container (not shown), vegetable compartment 24 is provided with a vegetable container (not shown), and lower freezer compartment 25 is provided with a lower freezer container (not shown).

[0011] Although not shown, the refrigerator 100 also includes a compressor, a radiator (condenser), a capillary tube (throttle mechanism), and an evaporator. A refrigerant circulates through the compressor, the radiator, the capillary tube, and the evaporator in this order, and the air in the storage compartment is cooled by heat exchange with the refrigerant flowing through the evaporator. The camera unit 3 shown in Fig. 1 captures images of the storage compartment and the like of the refrigerator 100, and is installed on the top surface of the housing 1.

[0012] FIG. 2 is a side view of the refrigerator 100. 2, the camera unit 3 includes a main body 31 and a support 32. The main body 31 has a function of capturing images of a storage compartment, etc. The support 32 supports the main body 31 and is installed on the top surface of the housing 1.

[0013] A lens 31a (see also FIG. 5) is provided near the front end of the main body 31. The lens 31a is an optical element that refracts and focuses light. For example, a fisheye lens is used as this lens 31a. When the refrigerator compartment doors 211, 212 (see FIG. 1) are opened, the lens 31a faces downward so that the camera unit 3 can capture an image of the refrigerator compartment 21 (see FIG. 1).

[0014] As shown in FIG. 2, lens 31a is located forward of the front end of housing 1 (opening 10 of housing 1). More preferably, lens 31a is located further forward than the front surface of refrigerator compartment doors 211, 212 (see FIG. 1) in the closed state. This makes it easier for refrigerator compartment 21 to be included in the field of view of lens 31a when refrigerator compartment doors 211, 212 are opened, for example. Note that it is also possible for camera unit 3 to capture images of vegetable compartment 24 and the like (see FIG. 1) when a drawer-type door such as vegetable compartment door 241 is open. In this way, lens 31a is provided in a position that allows for a view down on each storage compartment from above.

[0015] FIG. 3 is a front view of the refrigerator 100 with the left and right refrigerator compartment doors 211, 212 open. As described above, the inner panel 211b of the left refrigerator compartment door 211 is provided with a plurality of door pockets 211c for storing food and the like (the same applies to the right refrigerator compartment door 212). When the left and right refrigerator compartment doors 211, 212 are opened, the food and the like in the refrigerator compartment 21 and the door pockets 211c, 212c come into the field of view of the lens 31a of the camera unit 3 in a bird's-eye view (see also Figure 10).

[0016] Rotating partition 211d shown in Fig. 3 is a partition for preventing cold air from leaking from the gap between refrigerator compartment doors 211 and 212. In the example of Fig. 3, rotating partition 211d is provided at the end of left refrigerator compartment door 211 opposite to the axis of hinge 211a (see Fig. 4), and is designed to rotate in a predetermined manner as refrigerator compartment door 211 is opened or closed.

[0017] Furthermore, when manually photographing refrigerator compartment 21 using camera unit 3, one photographing button 7 that is pressed by the user is provided on each of the left and right refrigerator compartment doors 211, 212. In the example of Fig. 3, the photographing button 7 is provided on the left refrigerator compartment door 211 (door) at the bottom of the surface opposite hinge 211a (see Fig. 4) of this refrigerator compartment door 211 (the same is true for the right refrigerator compartment door 212). From another perspective, when the left and right refrigerator compartment doors 211, 212 are closed (see Fig. 1), the photographing button 7 is provided on the surface of one of the refrigerator compartment doors 211, 212 facing the other (near rotating partition 211d in Fig. 3).

[0018] Shooting performed when at least one of these two shooting buttons 7 is pressed by the user is called "manual shooting." Shooting performed by the control unit 8 (see FIG. 7) based on the elapsed time from the timing when the refrigerator compartment doors 211, 212 start to open is called "automatic shooting."

[0019] FIG. 4 is a plan view of the refrigerator 100 with the left and right refrigerator compartment doors 211, 212 open. In the example of Figure 4, main body 31 of camera unit 3 is arranged slightly left of the center of housing 1 of refrigerator 100 in the left-right direction. To explain in more detail, main body 31 of camera unit 3 is arranged directly above the joint between refrigerator compartment doors 211, 212 in the closed state (see also Figure 1). By arranging main body 31 in this way, for example, when refrigerator compartment doors 211, 212 are opened, food and the like in door pockets 211c, 212c are more likely to come into the field of view of camera unit 3.

[0020] Furthermore, when using refrigerator 100, the user is often conscious of the seam (boundary) of refrigerator compartment doors 211, 212. Therefore, by arranging main body 31 directly above the seam of refrigerator compartment doors 211, 212, the user is less likely to feel uncomfortable compared to when main body 31 is positioned off the seam in the left-right direction.

[0021] FIG. 5 is a perspective view of the camera unit 3 when viewed from diagonally below. The main body 31 of the camera unit 3 shown in Fig. 5 includes the lens 31a (see also Fig. 2) described above, as well as a case 31b and a camera LED 31c (Light Emitting Diode: illumination unit). In the example of Fig. 5, the case 31b of the main body 31 is generally a rectangular parallelepiped that is elongated in the front-to-rear direction. A circular hole (not shown) is provided on the underside near the front end of the case 31b, and the lens 31a is exposed through this hole. The camera LED 31c is provided on the rear side (rear side) of the lens 31a.

[0022] Camera LED 31c is a light source that irradiates refrigerator compartment 21 (see FIG. 3) and door pockets 211c, 212c (see FIG. 3) with light so that camera unit 3 can photograph refrigerator compartment 21 (see FIG. 3) and the like under appropriate brightness. Camera LED 31c (illumination unit) is provided on the outside of housing 1 (see FIG. 2) of refrigerator 100. Details will be described later, but in the first embodiment, camera unit 3 photographs refrigerator compartment 21 (see FIG. 3) with interior light 4 (see FIG. 7) of refrigerator 100 turned off and camera LED 31c turned on.

[0023] As shown in Fig. 5, by providing lens 31a in front of camera LED 31c, reflected light from refrigerator compartment 21 and the like (see Fig. 3) can easily enter lens 31a when refrigerator compartment doors 211, 212 are open. This makes it possible to properly photograph food stored in refrigerator compartment 21 and the like. Note that camera LED 31c is preferably positioned forward of the front end (opening 10 of housing 1) of housing 1 of refrigerator 100 (see Fig. 2). This makes it easier for light emitted from camera LED 31c to enter refrigerator compartment 21 and the like, which in turn makes it easier to obtain clear photographic results.

[0024] FIG. 6A is a front view of the camera unit 3. FIG. 6A, the main body 31 of the camera unit 3 is placed above the support part 32. The main body 31 is placed near the center of the support part 32 in the left-right direction.

[0025] FIG. 6B is a cross-sectional view taken along line II-II in FIG. 6A. As shown in Fig. 6B, camera unit 3 includes lens 31a (see also Fig. 5), camera LED 31c (see also Fig. 5), case 31b, and circuit board 31d. Circuit board 31d is a printed circuit board on which an image sensor 31e (see Fig. 7) and a camera / communication control SoC 31f (System-on-a-Chip: see Fig. 7), which will be described later, are mounted. By housing lens 31a, camera LED 31c, and circuit board 31d in a single case 31b in this way, the camera unit 3 can be made more compact and the length of the wiring can be shortened compared to when these components are provided separately.

[0026] FIG. 7 is a diagram showing the system configuration including the camera unit 3 of the refrigerator 100. As shown in Fig. 7, refrigerator 100 includes camera unit 3, as well as interior light 4, buzzer 5, door sensor 6, image capture button 7, and control unit 8. Interior light 4 is a light source that irradiates light into the interior of refrigerator 100, and is provided inside housing 1. Buzzer 5 is provided in a predetermined location in housing 1 and sounds when interior light 4 is turned off, for example, during image capture by camera unit 3.

[0027] Door sensor 6 outputs a predetermined signal indicating the open / closed state of refrigerator compartment doors 211, 212 to control unit 8. Although one door sensor 6 is shown in Fig. 7, in reality, a door sensor 6 corresponding to left refrigerator compartment door 211 (see Fig. 1) and another door sensor 6 corresponding to right refrigerator compartment door 212 (see Fig. 1) are provided. For example, when left refrigerator compartment door 211 is opened, an open signal is output from door sensor 6 corresponding to refrigerator compartment door 211 to control unit 8. As described above, photographing button 7 (see also Fig. 3) is a button that is pressed by the user when manually photographing using camera unit 3.

[0028] The control unit 8 is, for example, a microcomputer, and although not shown, is configured to include electronic circuits such as a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), and various interfaces. The CPU reads out a program stored in the ROM and loads it into the RAM, and the CPU executes various processes. In addition to performing the "automatic shooting" described above, the control unit 8 also performs "manual shooting" when the shooting button 7 is pressed. Details of the processing performed by the control unit 8 will be described later.

[0029] The camera unit 3 includes a lens 31a (see also FIG. 5) and a camera LED 31c (see also FIG. 5), as well as an image sensor 31e, a camera / communication control SoC 31f, and a wireless LAN unit 31g. The image sensor 31e photoelectrically converts light incident through the lens 31a to generate captured image data. For example, a CCD sensor (Charge Coupled Device) or a CMOS sensor (Complementary Metal Oxide Semiconductor) is used as the image sensor 31e.

[0030] The camera / communication control SoC31f is an integrated circuit designed to integrate circuits having microcomputer functions and other applied functions on a single chip and function in cooperation with each other. The camera / communication control SoC31f communicates with the control unit 8 in a predetermined manner and outputs a shooting command to the image sensor 31e. As a result, captured image data is input from the image sensor 31e to the camera / communication control SoC31f. Note that the camera / communication control SoC31f is an example of an integrated circuit, and other types of integrated circuits may also be used.

[0031] 7 has a function of capturing images of at least the refrigerator compartment 21 (storage compartment) and is provided on the outside of the housing 1 (see FIG. 1) of the refrigerator 100. As shown in FIG. 7, the photographing unit 30 includes a lens 31a, an image sensor 31e, and a camera / communication control SoC 31f.

[0032] Wireless LAN unit 31g transmits captured image data output from camera / communication control SoC 31f to router 51. Router 51 is a communication relay device and transmits the captured image data received from wireless LAN unit 31g to server 53 via network 52. Server 53 performs predetermined processing on the captured image data and stores it in association with the identification information of refrigerator 100. Furthermore, when server 53 receives a request signal for an image of the interior of refrigerator 100 from a user's mobile terminal 54, it transmits the captured image data to mobile terminal 54. As such mobile terminal 54, a mobile phone, a smartphone, a tablet, a wearable terminal, etc. may be used.

[0033] <Processing of control section> 8A and 8B are flowcharts of the automatic photography process executed by the control unit of the refrigerator (see FIG. 7 as appropriate). The following describes, as an example, the process in which camera unit 3 captures an image of refrigerator compartment 21 when at least one of refrigerator compartment doors 211, 212 (see FIG. 3) is open. At "START" in FIG. 8A, it is assumed that both left and right refrigerator compartment doors 211, 212 (see FIG. 3) are closed. In addition, the user can switch the automatic photography setting and the manual photography setting on and off by operating mobile terminal 54 (see FIG. 7). In the examples of FIGS. 8A and 8B, it is assumed that the automatic photography setting is on (automatic photography possible) and that the manual photography setting is also on (manual photography possible).

[0034] In step S101, the control unit 8 determines whether or not an open signal has been input from the door sensor 6. That is, the control unit 8 determines whether or not at least one of the left and right refrigerator compartment doors 211, 212 (see FIG. 3) has been opened. In step S101, if an open signal has not been input from the door sensor 6 (S101: No), the control unit 8 repeats the processing of step S101. On the other hand, in step S101, if an open signal has been input from the door sensor 6 (S101: Yes), the processing of the control unit 8 proceeds to step S102.

[0035] In step S102, the control unit 8 turns on the interior light 4 of the refrigerator 100 and also turns on the camera LED 31c. That is, immediately after at least one of the refrigerator compartment doors 211, 212 (doors) is opened, the control unit 8 turns on the interior light 4 and also turns on the camera LED 31c (illumination unit). As will be described later, the control unit 8 turns off the interior light 4 after a predetermined time has elapsed since the open signal was input from the door sensor 6 (S103: Yes, S104), but until then the interior light 4 is kept on. This irradiates the refrigerator compartment 21 with light, making it easier for the user to see the refrigerator compartment 21. Furthermore, this reduces the sense of discomfort felt by the user compared to when the interior light 4 is turned off when the user opens at least one of the refrigerator compartment doors 211, 212.

[0036] Furthermore, the control unit 8 turns on the camera LED 31c together with the interior light 4 (S102), thereby preparing for the subsequent photographing (S106). Also, the time required to adjust the exposure during photographing can be shortened compared to when the interior light 4 and the camera LED 31c are turned on at different times. The camera LED 31c remains lit until both the left and right refrigerator compartment doors 211, 212 are closed.

[0037] Next, in step S103, the control unit 8 determines whether or not a predetermined time has elapsed since the open signal was input from the door sensor 6. This predetermined time (for example, several seconds) is the time until the interior light 4 is turned off (S104), and is set in advance. For example, the predetermined time in step S103 is set in advance, taking into consideration the time required for the subsequent exposure adjustment (S105), so that there is a high possibility that the refrigerator compartment doors 211, 212 will be opened to a predetermined angle or more when the image is taken (S106).

[0038] In step S103, if the predetermined time has not elapsed since the open signal was input from the door sensor 6 (S103: No), the control unit 8 repeats the process of step S103. On the other hand, if the predetermined time has elapsed since the open signal was input from the door sensor 6 (S103: Yes), the process of the control unit 8 proceeds to step S104.

[0039] In step S104, the control unit 8 turns off the interior light 4 and sounds the buzzer 5. That is, the control unit 8 sounds the buzzer 5 when turning off the interior light 4 after at least one of the refrigerator compartment doors 211, 212 (doors) has been opened. By sounding the buzzer 5 in this way, it is possible to reduce the sense of discomfort felt by the user when the interior light 4 is turned off, and it is also possible for the user to easily understand that the light has been turned off to take a photo.

[0040] Next, in step S105, the control unit 8 performs exposure adjustment. That is, the control unit 8 adjusts the gain (sensitivity to brightness) when amplifying the electrical signal output from the image sensor 31e, as well as adjusting the shutter speed of the camera unit 3, etc. For example, the control unit 8 adjusts the gain so that the average brightness of the captured image data falls within a predetermined range. By performing exposure adjustment in this manner, it is possible to suppress blown-out highlights (halation) and crushed shadows in the captured image. Note that image input and exposure adjustment may be repeated in a predetermined manner from when at least one of the refrigerator compartment doors 211, 212 is opened until both refrigerator compartment doors 211, 212 are closed.

[0041] Next, in step S106, control unit 8 outputs a photography command. That is, control unit 8 captures, as photographed image data, the electrical signal that is output from image sensor 31e by photoelectric conversion and further amplified by a predetermined gain. In this way, after at least one of refrigerator compartment doors 211, 212 (doors) is opened (S101: Yes), control unit 8 turns off interior light 4 and further causes photographing unit 30 to take a photograph with camera LED 31c (illumination unit) turned on (S102, S104, S106).

[0042] 8A and 8B, it is preferable that the control unit 8 causes the photographing unit 30 to photograph after a predetermined time (for example, several seconds) has elapsed since the interior light 4 was turned off. By providing a predetermined time between the turning off of the interior light 4 and the photographing, exposure adjustment (S105) can be performed appropriately.

[0043] 8A and 8B, the control unit 8 may take a photograph every time a predetermined time has elapsed since the first photographing. The length of this predetermined time may be the same as or different from the predetermined time in step S103.

[0044] In step S107, control unit 8 recognizes the open angle of refrigerator compartment doors 211, 212. For example, control unit 8 performs image processing such as edge detection to recognize the open angle of one of refrigerator compartment doors 211, 212 that is open. Note that the "open angle" is the angle at which refrigerator compartment doors 211, 212 are open, and is recognized, for example, based on the front end of housing 1 (see FIG. 4) when refrigerator 100 is viewed from above.

[0045] Next, in step S108 of Fig. 8B, control unit 8 determines whether the open angle of refrigerator compartment doors 211, 212 is equal to or greater than a predetermined value. This predetermined value is a preset threshold value of the open angle that serves as a criterion for determining whether to save the photographed result (S109) or discard it (S110).

[0046] In step S108, if the open angle of at least one of refrigerator compartment doors 211, 212 is equal to or greater than the predetermined value (S108: Yes), the process by control unit 8 proceeds to step S109. In step S109, control unit 8 saves the photographing results and proceeds to the process of step S112. This is because if the open angle of refrigerator compartment doors 211, 212 is equal to or greater than the predetermined value, there is a high possibility that the refrigerator compartment 21 etc. has been photographed appropriately.

[0047] The photographing results when only the left refrigerator compartment door 211 is open, the photographing results when only the right refrigerator compartment door 212 is open, and the photographing results when both the left and right refrigerator compartment doors 211, 212 are open may be stored in different storage areas.

[0048] Furthermore, in step S108, if the open angles of the refrigerator compartment doors 211, 212 are both less than the predetermined value (S108: No), the process by the control unit 8 proceeds to step S110. In step S110, the control unit 8 discards the image capture results. This is because if the open angles of the refrigerator compartment doors 211, 212 are less than the predetermined value, it is highly likely that the refrigerator compartment 21, etc. have not been properly imaged.

[0049] Then, in step S111, control unit 8 sounds buzzer 5. That is, if an image is captured when the open angle of refrigerator compartment doors 211, 212 (doors) is less than a predetermined value (S108: No), control unit 8 sounds buzzer 5 (S111). This notifies the user that the image capture result has been discarded. Note that the sound of buzzer 5 in step S111 may be different from the sound of buzzer 5 in step S104 (when interior light 4 is turned off) so that the user can distinguish between the two. After processing step S111, control unit 8 proceeds to step S112. Instead of sounding buzzer 5, the refrigerator or mobile terminal 54 used by the user may output a light, vibration, or other output that the user can perceive.

[0050] In step S112, control unit 8 turns on interior light 4. That is, control unit 8 turns on interior light 4 while leaving camera LED 31c turned on. In this way, after control unit 8 causes photographing unit 30 to take an image (S106 in FIG. 8A), it turns on interior light 4 (S112 in FIG. 8B). This brightly illuminates refrigerator compartment 21, making it easier for the user to see refrigerator compartment 21.

[0051] Next, in step S113, the control unit 8 determines whether or not a close signal has been input from the door sensor 6. If a close signal has not been input from the door sensor 6 (S113: No), the control unit 8 returns to step S102. On the other hand, if a close signal has been input from the door sensor 6 (S113: Yes), the process by the control unit 8 proceeds to step S114.

[0052] In step S114, the control unit 8 turns off the interior light 4 and the camera LED 31c. In step S115, the control unit 8 selects a photographing result. For example, the control unit 8 selects the most recent of the multiple photographing results stored as the photographing result to be transmitted to the server 53. In this way, by completing the saving or discarding of the photographing result for each photograph, it is possible to reduce the amount of storage required by the control unit 8, reduce the amount of data that needs to be transmitted to the server 53, or, in the case of discarding, to notify the user accordingly. Of course, it is also possible to notify the user of the fact that the photographing result has been saved. It is sufficient that the user can recognize whether the photographing result has been saved or discarded. To make it easier for the user to recognize each photographic result, the time interval between photographs can be, for example, 3 seconds or more. In this way, when an output is generated that identifies a certain photographic result, the user can easily understand that the output is for that photographic result. For example, if the time interval is less than 3 seconds, there is a risk that the user will have difficulty distinguishing whether the output is for a certain photographic result or for a photographic result one or more years before or after that. On the other hand, if the time interval is too long, the refrigerator compartment doors 211, 212 will be open for a long time, causing a large amount of cold air to flow out of the refrigerator compartment, so the time interval is 8 seconds or less, preferably 5 seconds or less. If step S111 is not executed, shooting may be performed multiple times in succession in step S106. In this case, in steps S108 to S110, processing can be performed on each of the multiple image data.

[0053] In step S116, the control unit 8 transmits the photographed result to the server 53. That is, the control unit 8 transmits the photographed image data to the server 53 sequentially via the wireless LAN unit 31g, the router 51, and the network 52 shown in Fig. 7. After performing the process of step S116, the control unit 8 ends the series of processes (END).

[0054] In addition to the right and left areas of refrigerator compartment 21 (see FIG. 3), server 53 extracts the area of ​​left refrigerator compartment door 211 (see FIG. 3) and the area of ​​right refrigerator compartment door 212 (see FIG. 3) from the photographed results. Then, server 53 individually updates the images of each area to the latest ones. Furthermore, in response to a request signal from mobile terminal 54, server 53 transmits the photographed image data to mobile terminal 54, and the image is displayed in a predetermined manner on a display (not shown) of mobile terminal 54.

[0055] FIG. 9 is an explanatory diagram showing the process of opening and closing the refrigerator compartment doors 211, 212 in time sequence. 9 chronologically shows the states of refrigerator compartment doors 211, 212 as well as the states of interior light 4 and camera LED 31c in the alphabetical order A, B, C, . . . , J, K, and L. In addition, in FIG. 9 , when interior light 4 of refrigerator 100 is on, interior light 4 is shown (for example, state B), and when interior light 4 is off, interior light 4 is not shown (for example, state A). Similarly, in FIG. 9 , when camera LED 31c is on, camera LED 31c is shown (for example, state B), and when camera LED 31c is off, camera LED 31c is not shown (for example, state A).

[0056] In the example of FIG. 9, from State A where both the left and right refrigerator compartment doors 211, 212 are closed, the user opens the left refrigerator compartment door 211 (States B, C, and D). After this, the camera LED 31c remains lit until both the left and right refrigerator compartment doors 211, 212 are closed (S102 in FIG. 8A). Furthermore, as shown in States B and C, both the interior light 4 and the camera LED 31c remain lit (S102) until a predetermined time has elapsed since the left refrigerator compartment door 211 began to be opened (S103: No). Then, when the predetermined time has elapsed since the left refrigerator compartment door 211 began to be opened (S103: Yes in FIG. 8A), the interior light 4 is turned off and automatic photography is performed with the camera LED 31c lit, as shown in State D (S104 to S106). Thereafter, in State E, the interior light 4 is turned on again (S112 in FIG. 8B).

[0057] In states E and F, the left refrigerator door 211 is closed while the right refrigerator door 212 is open. In this case, as shown in state F, the interior light 4 is turned off and automatic photography is performed again with the camera LED 31c turned on (S104 to S106 in FIG. 8A). Thereafter, in the example of FIG. 9, as shown in states G, H, and I, the user closes the left refrigerator door 211 and then opens it again.

[0058] Even in such a case, as shown in state I, automatic photography is performed again with the interior light 4 turned off and the camera LED 31c turned on (S104 to S106 in FIG. 8A). This allows photography to be performed with both the left and right refrigerator compartment doors 211, 212 open. Thereafter, as shown in state J, the interior light 4 is turned on again (S112 in FIG. 8B). In this way, every time at least one of the left and right refrigerator compartment doors 211, 212 is opened, the control unit 8 causes the camera unit 3 to photograph the refrigerator compartment 21, etc.

[0059] Furthermore, as shown in state K, when the user presses the photographing button 7 (see FIG. 3), the control unit 8 performs manual photographing. In this case, although not specifically shown in the flowcharts of FIGS. 8A and 8B, the control unit 8 performs the following process. That is, when the photographing button 7 is pressed after at least one of the refrigerator compartment doors 211, 212 (doors) is opened, the control unit 8 turns off the interior light 4 and causes the photographing unit 30 to photograph with the camera LED 31c (illumination unit) turned on (state K). Note that when the interior light 4 is turned off, the control unit 8 may sound the buzzer 5 to reduce any sense of discomfort felt by the user. After performing manual photographing, the control unit 8 turns the interior light 4 on again as shown in state L.

[0060] When photographing is performed by operating the photographing button 7 (manual photographing), it is preferable that the control unit 8 displays the photographing result on the mobile terminal 54 (see FIG. 7) in priority to other photographing results (automatic photographing results). This allows the user to check the photographing result obtained by pressing the photographing button 7 on their own mobile terminal 54 (see FIG. 7).

[0061] Furthermore, it is preferable that the control unit 8 not take another photograph (automatic photograph) after photographing based on the operation of the photographing button 7 (after manual photographing) while at least one of the refrigerator compartment doors 211, 212 (doors) is open. This is because if the result of manual photographing is sent preferentially, there is no particular need to take an automatic photograph thereafter.

[0062] Furthermore, if the user does not close at least one of refrigerator doors 211, 212 securely, leaving the door ajar, automatic photography is performed at predetermined time intervals, but the photographed images are not sent to server 53 (see FIG. 7). This is because, in the case of ajar doors, the open angle of refrigerator doors 211, 212 is usually less than a predetermined value (S108: No in FIG. 8B), and the photographed images are discarded (S110).

[0063] FIG. 10 shows an example of the photographic result of the camera unit, without image processing. In Fig. 10, the upper side of the page is the front and the lower side is the rear, so the left and right are reversed compared to Fig. 3 (the same applies to Figs. 11, 12A, and 12B). In the example of Fig. 10, both left and right refrigerator compartment doors 211, 212 are open, and in addition to refrigerator compartment 21, door pocket 211c of left refrigerator compartment door 211 and door pocket 212c of right refrigerator compartment door 212 are also photographed.

[0064] In particular, a fisheye lens is used as lens 31a of camera unit 3 (see FIG. 5), allowing for imaging with a wide angle of view. However, without image processing, ridgelines that are actually straight appear curved, as shown in FIG. 10, and even for ridgelines of the same length, the number of pixels in the captured image decreases as the ridgeline moves farther from camera unit 3.

[0065] FIG. 11 shows an example of the photographic result of the camera unit 3 after image processing. For example, predetermined image processing is performed by server 53 (see FIG. 7) based on the photographed result of Fig. 10, and the images are converted into images that appear as if they were taken from the front of refrigerator compartment 21 and left and right refrigerator compartment doors 211, 212. Such images are displayed on mobile terminal 54 (see FIG. 7), allowing the user to see at a glance what foods are stored in refrigerator compartment 21, etc. Next, the details of the exposure adjustment process (S105 in FIG. 8A) by the control unit 8 will be described in detail with reference to FIG. 12A (first embodiment) and FIG. 12B (comparative example).

[0066] FIG. 12A is an explanatory diagram showing candidates for areas that serve as a reference when adjusting exposure in the refrigerator according to the first embodiment. As described above, in the exposure adjustment process (S105 in FIG. 8A), the gain used to amplify the electrical signal output from the image sensor 31e (see FIG. 7) is adjusted. In the first embodiment, at least one of the three rectangular areas 61 to 63 shown in FIG. 12A is used as a reference for indicating the degree of image brightness in exposure adjustment. That is, at least one of the area 61 on the left refrigerator compartment door 211, the area 62 on the right refrigerator compartment door 212, and a predetermined area 63 between these areas 61 and 62 is used. The positions and sizes of the areas 61 to 63 may be changed as appropriate based on the opening angles of the refrigerator compartment doors 211 and 212.

[0067] In the example of FIG. 12A, the left refrigerator compartment door 211 is open, but the right refrigerator compartment door 212 is closed. In this case, the control unit 8 adjusts the exposure based on the average luminance value of the area 61 corresponding to the left refrigerator compartment door 211. That is, the control unit 8 adjusts the gain used when amplifying the electrical signal output from the image sensor 31e (see FIG. 7) so that the average luminance value of the area 61 falls within a predetermined range. As a result, the area 62 of the right refrigerator compartment door 212, which is closed, and the area 63, part of which is outside the left refrigerator compartment door 211, are excluded from the reference for adjusting the gain. Therefore, even when the area around the refrigerator compartment is dark, the control unit 8 can appropriately adjust the exposure without increasing the gain too much.

[0068] In this way, when one of the right refrigerator compartment door 212 (right door) and the left refrigerator compartment door 211 (left door) is opened, the control unit 8 adjusts the exposure gain during shooting based on the area corresponding to the door within the field of view of the shooting unit 30. Note that signals indicating the open / closed states of the left and right refrigerator compartment doors 211, 212 are output to the control unit 8 from the left and right door sensors 6 (see FIG. 7), respectively.

[0069] Furthermore, exposure adjustment may be performed using as a reference the area with the highest average image brightness among the three areas 61 to 63. That is, the control unit 8 adjusts the exposure gain during shooting using as a reference the area with the highest average image brightness among the multiple areas 61 to 63 included in the field of view of the shooting unit 30. This prevents the gain from becoming too high, and ultimately reduces blown-out highlights (halation) in the shot image.

[0070] Furthermore, exposure adjustment may be performed based on the area of ​​the left and right refrigerator compartment doors 211, 212 that includes at least a part of the door to which the open signal is input from the door sensor 6 and that has the highest average image brightness. This allows the control unit 8 to appropriately set the gain for exposure adjustment even in a situation where, for example, light irradiated from outside the refrigerator 100 is reflected on the surface of the refrigerator compartment door 212 in the closed state. This makes it possible to prevent overexposure and underexposure from occurring in the captured image.

[0071] FIG. 12B is an explanatory diagram showing an area that serves as a reference for brightness when adjusting exposure in a refrigerator according to a comparative example. In the comparative example of FIG. 12B , a single rectangular area 60 is set as a reference for adjusting exposure. For example, when the left and right refrigerator compartment doors 211, 212 are open, rectangular area 60 is set to include both of these refrigerator compartment doors 211, 212. When a single large area 60 like this is used and the surroundings of refrigerator 100 are dark, right refrigerator compartment door 212 appears dark in the image captured, so a higher gain is set to increase sensitivity to light. As a result, overexposure is likely to occur in the left refrigerator compartment door 211 in the open state and in the left half of refrigerator compartment 21 in the captured image.

[0072] In contrast to this, in the first embodiment (see FIG. 12A), for example, the control unit 8 selects an area that serves as a reference for adjusting exposure based on the open / closed state of the refrigerator compartment doors 211, 212. This makes it possible to suppress overexposure in the captured image and display a clear image that is easy for the user to view on the mobile terminal 54 (see FIG. 7).

[0073] The wall surfaces of refrigerator compartment 21 (storage compartment) and inner panels 211b, 212b (see FIG. 3) of refrigerator compartment doors 211, 212 (doors) may each be white. Here, "white" refers to a color that reflects light with a reflectance of 50% or more across substantially the entire visible light spectrum. From another perspective, "white" refers to a color in which the lightness of each element of R (Red), G (Green), and B (Blue) in the RGB color system is substantially the same and that reflects light with a reflectance of 50% or more.

[0074] When the wall surfaces of refrigerator compartment 21, etc. are white, the captured image tends to be overexposed, but control unit 8 can suppress overexposure by appropriately selecting an area that serves as a reference when adjusting exposure. As described above, according to the first embodiment, refrigerator compartment 21, etc. can be appropriately photographed using camera unit 3. This makes it easier for the user to view the state of refrigerator compartment 21, etc. by looking at mobile terminal 54.

[0075] Second Embodiment The second embodiment differs from the first embodiment in that a photo button 7A (see FIG. 13) is provided near handles 211e, 212e (see FIG. 13) provided on the undersides of left and right refrigerator compartment doors 211, 212 (see FIG. 13). The rest of the second embodiment is the same as the first embodiment. Therefore, only the parts that differ from the first embodiment will be described, and a description of the overlapping parts will be omitted.

[0076] FIG. 13 is a front view and a bottom view of left and right refrigerator compartment doors 211, 212 included in a refrigerator 100A according to the second embodiment. In Fig. 13, a bottom view of refrigerator compartment doors 211, 212 is shown below the front view of the doors. As shown in Fig. 13, a handle portion 211e that is recessed upward is provided on the bottom surface of left-side refrigerator compartment door 211, extending over a predetermined length in the horizontal direction.

[0077] Furthermore, on the underside of the left refrigerator compartment door 211, inside the handle 211e (on the opposite side from the hinge shaft 211f), there is provided a photographing button 7A that is pressed by the user when taking a manual photograph. In other words, the photographing button 7A is provided outside the handle 211e of the refrigerator compartment door 211 (door) and near this handle 211e. Similarly, a handle 212e and a photographing button 7A are provided on the underside of the right refrigerator compartment door 212.

[0078] By arranging the photographing button 7A in this manner, for example, when a user places their hand on the handle 212e to open the right refrigerator compartment door 212, it is possible to prevent the user from accidentally pressing the photographing button 7A unintentionally. When a user places their right hand on the right handle 211e, the palm of their hand faces upward and the thumb of their right hand is positioned on the outer side (right side). In the example of FIG. 13, the photographing button 7A is provided on the inner side (left side) of the handle 212e. Therefore, when a user places their right hand on the handle 212e, it is possible to prevent the thumb of their right hand from accidentally touching the photographing button 7A. The same can be said for the photographing button 7A on the left refrigerator compartment door 211.

[0079] Furthermore, since the photographing button 7A is provided near the handle portion 212e, the user can place their hand on the refrigerator compartment door 212 and move their hand to press the photographing button 7A by the feel of their fingertips. Therefore, the user does not need to release their hand from the refrigerator compartment door 212 when pressing the photographing button 7A, which improves operability. Furthermore, since the user can take a photograph with the refrigerator compartment door 212 opened to a predetermined position and the rotation of the refrigerator compartment door 212 stopped, blurring of the photographed image can be reduced. Furthermore, compared to when the photographing button 7A is provided on the inside of the refrigerator compartment door 212 or on the wall of the refrigerator compartment 21, it is possible to reduce the likelihood of the user's hand appearing in the photographed image. Furthermore, by providing the photographing button 7A near the handle portion 212e, it is possible to share the wiring for the rotating partition 211d (see FIG. 3) and the operation panel (not shown) with the wiring for the photographing button 7A. The processing of the control unit 8 (see FIG. 7) based on the operation of the shooting button 7A is the same as that in the first embodiment (see FIGS. 8A and 8B), and therefore will not be described here.

[0080] FIG. 14A is a cross-sectional perspective view of the lower part of right refrigerator compartment door 212 provided in the refrigerator. The configuration in Fig. 14A corresponds to Fig. 13. As shown in Fig. 14A, the photo capture button 7A on the right-side refrigerator compartment door 212 includes a resin switch portion 71A that is pressed by the user, a laminate film 72A that is attached to the outside of the switch portion 71A, and a switch board 73A that generates a predetermined electrical signal when the switch portion 71A is pressed. A plurality of wires 80 are connected to the switch board 73A. These wires 80 include power lines and signal lines. The left-side refrigerator compartment door 211 (see Fig. 13) has a similar configuration.

[0081] FIG. 14B is a cross-sectional view of the lower part of the right refrigerator compartment door 212 of the refrigerator. The configuration in Fig. 14B corresponds to Fig. 13 and Fig. 14A. As shown in Fig. 14B, a partition wall 92 is provided inside refrigerator compartment door 212 (door) to separate area 91a around photograph button 7A from the rest. Partition wall 92 is provided with an insertion hole 93 for wiring 80 connected to photograph button 7A. A seal member 94 that closes insertion hole 93 is provided in insertion hole 93. For example, urethane foam is used as such seal member 94. Then, wiring 80 is inserted through seal member 94. In other words, with wiring 80 inserted through insertion hole 93, insertion hole 93 is closed by seal member 94.

[0082] By providing such sealing member 94, when a heat insulating material such as urethane is injected (filled) into the internal space of refrigerator compartment door 212 during manufacturing, it is possible to prevent the heat insulating material from leaking toward the photographing button 7A through insertion hole 93. In addition, it becomes possible to draw out wiring 80 connected to photographing button 7A to area 91b above partition wall 92.

[0083] <<Variations>> The refrigerator 100 and the like according to the present disclosure have been described above in accordance with the various embodiments, but they are not limited to these descriptions and can be modified in various ways. For example, in each embodiment, the camera unit 3 is provided on the top surface of the housing 1 of the refrigerator 100 (see FIG. 2), but this is not limiting. That is, the camera unit 3 may be provided at another predetermined position outside the housing 1.

[0084] In addition, in each embodiment, a configuration has been described in which the camera LED 31c (see FIG. 5) is disposed behind the lens 31a (see FIG. 5), but this is not limiting. For example, the camera LED may be disposed in front of the lens 31a. Furthermore, multiple camera LEDs may be disposed in a predetermined arrangement.

[0085] Furthermore, in each embodiment, as an example, a case has been described in which camera unit 3 photographs the interior of refrigerator compartment doors 211, 212 in addition to refrigerator compartment 21, but this is not limited to this. For example, camera unit 3 may photograph at least one of drawer-type ice-making compartment door 221 (door: see FIG. 1), upper freezer compartment door 231 (door), vegetable compartment door 241 (door), and lower freezer compartment door 251 (door).

[0086] In addition, in each embodiment, a configuration has been described in which the photo capture button 7 (see FIG. 3) is provided on the left and right refrigerator compartment doors 211, 212, but this is not limiting. For example, a photo capture button (not shown) may be provided on at least one of the drawer-type ice compartment door 221 (see FIG. 1), upper freezer compartment door 231, vegetable compartment door 241, and lower freezer compartment door 251.

[0087] Furthermore, in each embodiment, the process (S102) in which the control unit 8 turns on the interior light 4 and the camera LED 31c when at least one of the refrigerator compartment doors 211, 212 is opened (S101: Yes in Fig. 8A) has been described, but this is not limiting. That is, when at least one of the refrigerator compartment doors 211, 212 is opened, the control unit 8 may turn on the interior light 4 and the camera LED 31c at a predetermined staggered timing.

[0088] In addition, in each embodiment, the case where the control unit 8 keeps the camera LED 31c on from when at least one of the left and right refrigerator compartment doors 211, 212 is opened until both refrigerator compartment doors 211, 212 are closed has been described, but this is not limiting. For example, when exposure adjustment (S105 in FIG. 8A) is not being performed, the control unit 8 may turn off the camera LED 31c at a predetermined timing.

[0089] In the second embodiment, a configuration has been described in which recessed handles 211e, 212e are provided on the undersides of refrigerator compartment doors 211, 212 (see FIG. 13), but this is not limiting. For example, a configuration in which a predetermined door handle (not shown) is provided on refrigerator compartment doors 211, 212 may also be used. In this case, a photographing button (not shown) may be provided on the outside of the door handle, or a photographing button (not shown) may be provided at a predetermined position on the door handle itself.

[0090] Furthermore, the refrigerator 100 and the like described in each embodiment are merely examples, and can be applied to other types of refrigerators or storage cabinets. For example, each embodiment can be applied to a refrigerator with a single-wing refrigerator compartment door (not shown) or a portable refrigerator (not shown). The image capturing device can receive information from a computing device or sensor provided in the storage cabinet via wired or wireless communication and perform image capturing, and may be a separate device from the storage cabinet.

[0091] Furthermore, each embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to having all of the configurations described. Furthermore, some of the configurations of the embodiments can be added to, deleted from, or replaced with other configurations. Furthermore, the mechanisms and configurations described above are those that are considered necessary for the explanation, and do not necessarily represent all mechanisms and configurations of the product. [Explanation of symbols]

[0092] 100,100A refrigerator (storage) 1 chassis 10 aperture 21 Refrigerator (storage room) 211 Refrigerator door (door, first door) 212 Refrigerator door (door, first door) 211a, 212a hinge 211b,212b Inner plate 211e, 212e Handle 22 Ice making room (storage room) 221 Ice Room Door (Door, Second Door) 23 Upper freezer compartment (storage compartment) 231 Upper freezer door (door, second door) 24 Vegetable compartment (storage compartment) 241 Vegetable compartment door (door, second door) 25 Lower freezer compartment (storage compartment) 251 Lower freezer door (door, second door) 3 Camera Unit 4 Interior light 5 Buzzer 6 Door Sensor 7,7A Shooting button 8 Control Unit 31c Camera LED 30 Photography Department 54 Mobile Devices 61,62,63 area 80 Wiring 91a area 92 Bulkhead 93 Insertion hole 94 Sealing material

Claims

1. a housing having a storage chamber; a door that closes the opening of the storage chamber; a door sensor that detects whether the door is open or closed; and an imaging unit that images at least the storage chamber, a control unit that, after the door opening operation is detected by the door sensor, issues a command to the photographing unit to photograph while the door opening operation is being detected, determines whether to save or discard the most recently photographed image based on the degree of opening of the door, and, in accordance with the determination, completes the saving if the image is to be saved so that it can be selected as an image that can be sent to the server, or completes the discarding if the image is to be discarded because it cannot be selected as an image that can be sent to the server, and then repeats the next photographing, decision on whether to save and discard, and execution of saving or discarding until the open door is closed; The criteria for the above-mentioned judgment are not to unconditionally judge whether the latest captured image should be saved, but rather there is a possibility of either saving or discarding the image. A storage facility characterized by:

2. The doors include a hinged first door and a drawer-type second door; The second door is lower in height than the first door, An area in the field of view of the photographing unit in which the second door is photographed is set to be outside the scope of exposure adjustment in the photographing unit when the first door is opened. The storage facility according to claim 1, characterized in that

3. The control unit recognizes the opening angle of the door as the degree of opening of the door, The criteria for the judgment are based on the result of comparing the opening angle with a preset criterion. The storage facility according to claim 1, characterized in that

Images