Storage house

The refrigerator system addresses the lack of detail in existing technologies by using a door sensor and photographing unit to optimize imaging based on door opening, ensuring clear and efficient storage compartment photography.

JP2024169495AActive Publication Date: 2024-12-05HITACHI GLOBAL LIFE SOLUTIONS INC
View PDF 16 Cites 0 Cited by

Patent Information

Application Number
JP2024159605
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-12-05
Estimated Expiration
2041-10-19

AI Technical Summary

Technical Problem

Existing technologies do not detail the relationship between the setting of the photographing time interval and the timing of selecting photographed photographs, particularly in refrigerator storage compartment imaging.

Method used

A refrigerator system with a casing, door sensor, and photographing unit that issues a photographing command upon door opening, recognizing the door's opening degree based on the photographed image, and adjusts exposure settings accordingly.

Benefits of technology

Ensures proper imaging of refrigerator contents by adjusting exposure based on door opening angles, reducing unnecessary images and enhancing image clarity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024169495000001_ABST
    Figure 2024169495000001_ABST
Patent Text Reader

Abstract

To provide a storage house that allows a storage room, etc. to be imaged appropriately.SOLUTION: A storage house includes: a housing having a storage room; a door closing the opening of the storage room; a door sensor for detecting opening / closing of the door; and an imaging unit for imaging at least the storage room. The storage house also includes a control unit for executing an imaging command to the imaging unit while an opening operation is detected after the door opening operation is detected by the door sensor, and recognizing an opening degree of the door on the basis of the image captured by the imaging unit. Also, the control unit determines storage or disposal of the latest captured image on the basis of the opening degree of the door.SELECTED DRAWING: Figure 8A
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a reservoir. [Background technology]

[0002] There is known a technology for taking pictures of the storage compartment of a refrigerator with a camera. Patent Document 1 discloses that, in principle, the selection of the photographs taken is performed according to the angle of the door (0278), but when a photo selection command is not issued, a method is used to temporarily store a small number of photographs using a first-in, first-out method (0096). It also discloses that, taking into account costs, the time interval between photographs taken by a camera that takes continuous photographs can be made longer than 1 / 30 seconds (0271, 0274). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] European Patent No. 2909549 (EP, B1) Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 does not disclose in detail the relationship between the setting of the shooting time interval and the selection timing of the shot photos. [Means for solving the problem]

[0005] The storage facility according to the present disclosure comprises: a housing having a storage chamber; A door that closes an opening of the storage chamber; A door sensor that detects whether the door is open or closed; An imaging unit that images at least the storage chamber, After the door opening operation is detected by the door sensor, a control unit issues an image capture command to the photographing unit while the door opening operation is being detected, and recognizes the degree of opening of the door based on the image captured by the photographing unit. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a front view of a refrigerator according to a first embodiment. [Diagram 2] FIG. 1 is a side view of a refrigerator according to a first embodiment. [Diagram 3] FIG. 1 is a front view of a refrigerator according to a first embodiment with left and right refrigerator compartment doors open. [Figure 4] FIG. 1 is a plan view of a refrigerator according to a first embodiment with left and right refrigerator compartment doors open. [Diagram 5] 1 is a perspective view of a camera unit of a refrigerator according to a first embodiment when viewed from diagonally below. [Figure 6A] FIG. 2 is a front view of a camera unit of the refrigerator according to the first embodiment. [Figure 6B] 6B is a cross-sectional view taken along line II-II of FIG. 6A in the refrigerator according to the first embodiment. [Figure 7] FIG. 1 is a system configuration diagram including a camera unit of a refrigerator according to a first embodiment. [Figure 8A] 5 is a flowchart of an automatic photography process executed by a control unit of the refrigerator according to the first embodiment. [Figure 8B] 5 is a flowchart of an automatic photography process executed by a control unit of the refrigerator according to the first embodiment. [Figure 9] FIG. 2 is an explanatory diagram showing a time sequence of a process in which a refrigerator compartment door of the refrigerator according to the first embodiment is opened and closed. [Figure 10] 1 is an example of a photographing result of a camera unit of a refrigerator according to a first embodiment, in which no image processing is performed. [Figure 11] 4 is an example of a result of photographing by a camera unit of the refrigerator according to the first embodiment, in which image processing has been performed. [Figure 12A] FIG. 2 is an explanatory diagram showing candidates for an area serving as a reference when performing exposure adjustment in the refrigerator according to the first embodiment. [Figure 12B]FIG. 11 is an explanatory diagram showing a reference area for adjusting exposure in a refrigerator according to a comparative example. [Figure 13] 11A and 11B are a front view and a bottom view of left and right refrigerator compartment doors of a refrigerator according to a second embodiment. [Figure 14A] FIG. 11 is a cross-sectional oblique view of the lower part of a right-side refrigerator compartment door of a refrigerator according to a second embodiment. [Figure 14B] 11 is a cross-sectional view of the lower part of a right-side refrigerator compartment door of a refrigerator according to a second embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] First Embodiment FIG. 1 is a front view of a refrigerator 100 according to the first embodiment. Refrigerator 100 is an appliance for cooling food and the like, and includes a housing 1, doors such as refrigerator compartment doors 211, 212, and a camera unit 3. Housing 1 has multiple storage compartments therein. In the example of Fig. 1, the storage compartments of refrigerator 100 include, from the top, refrigerator compartment 21, ice-making compartment 22 and upper freezer compartment 23 arranged on the left and right, as well as vegetable compartment 24 and lower freezer compartment 25.

[0008] The housing 1 is configured such that a space between an outer box 11 made of a steel plate and an inner box (not shown) made of resin is filled with a heat insulating material (not shown) such as urethane foam. A plurality of openings 10 (see Figs. 2 and 4) corresponding to each chamber are provided on the front side (front surface side) of the housing 1. For example, when food or the like is placed through the opening 10 of the refrigerator compartment 21, the refrigerator compartment doors 211, 212 are opened. When the refrigerator compartment doors 211, 212 are closed, the opening 10 of the refrigerator compartment 21 is closed. In this way, the refrigerator compartment doors 211, 212 (doors) have a function of closing the opening 10 of the housing 1. The same applies to the other doors.

[0009] Refrigerator 100 is equipped with left and right refrigerator compartment doors 211, 212 as French style doors which form refrigerator compartment 21 together with housing 1. Left refrigerator compartment door 211 (left door) is rotatable about the axis of hinge 211a (see FIG. 4) at the left end. The same is true for right refrigerator compartment door 212 (right door). Refrigerator 100 is also equipped with ice-making compartment door 221, upper freezer compartment door 231, as well as vegetable compartment door 241 and lower freezer compartment door 251 as drawer-type doors.

[0010] Refrigerator compartment 21 is provided with a plurality of shelves 213 (see FIG. 3) that divides refrigerator compartment 21 into predetermined sections. A plurality of door pockets 211c (see FIG. 3) for storing food and the like are provided on the inside of left refrigerator compartment door 211 (similar to right refrigerator compartment door 212). Ice-making compartment 22 is provided with an ice-making compartment container (not shown) that is pulled out together with ice-making compartment door 221. Similarly, upper freezer compartment 23 is provided with an upper freezer compartment container (not shown), vegetable compartment 24 is provided with a vegetable compartment container (not shown), and lower freezer compartment 25 is provided with a lower freezer compartment container (not shown).

[0011] Although not shown, the refrigerator 100 also includes a compressor, a radiator (condenser), a capillary tube (throttle mechanism), and an evaporator. A refrigerant circulates through the compressor, the radiator, the capillary tube, and the evaporator in this order, and the air in the storage compartment is cooled by heat exchange with the refrigerant flowing through the evaporator. The camera unit 3 shown in Fig. 1 captures an image of the storage compartment, etc. of the refrigerator 100, and is installed on the top surface of the housing 1.

[0012] FIG. 2 is a side view of the refrigerator 100. 2, the camera unit 3 includes a main body 31 and a support 32. The main body 31 has a function of capturing an image of a storage room, etc. The support 32 supports the main body 31 and is installed on the top surface of the housing 1.

[0013] A lens 31a (see also FIG. 5) is provided near the front end of the main body 31. The lens 31a is an optical element that refracts and focuses light. For example, a fisheye lens is used as the lens 31a. When the refrigerator compartment doors 211, 212 (see FIG. 1) are opened, the lens 31a faces downward so that the camera unit 3 can capture an image of the refrigerator compartment 21 (see FIG. 1).

[0014] As shown in FIG. 2, lens 31a is located forward of the front end of housing 1 (opening 10 of housing 1). More preferably, lens 31a is located further forward of the front surface of refrigerator compartment doors 211, 212 (see FIG. 1) in the closed state. This makes it easier for refrigerator compartment 21 to be within the field of view of lens 31a when refrigerator compartment doors 211, 212 are opened, for example. It is also possible for camera unit 3 to capture images of vegetable compartment 24, etc. (see FIG. 1) when a drawer-type door such as vegetable compartment door 241 is open. In this way, lens 31a is provided in a position that allows each storage compartment to be viewed from above.

[0015] FIG. 3 is a front view of refrigerator 100 with left and right refrigerator compartment doors 211, 212 open. As described above, the inner panel 211b of the left refrigerator compartment door 211 is provided with a plurality of door pockets 211c for storing food and the like (the same is true for the right refrigerator compartment door 212). When the left and right refrigerator compartment doors 211, 212 are opened, the food and the like in the refrigerator compartment 21 and the door pockets 211c, 212c are viewed from above within the field of view of the lens 31a of the camera unit 3 (see also FIG. 10).

[0016] Rotating partition 211d shown in Fig. 3 is a partition for preventing cold air from leaking from the gap between refrigerator compartment doors 211, 212. In the example of Fig. 3, rotating partition 211d is provided at the end opposite to the axis of hinge 211a (see Fig. 4) in left refrigerator compartment door 211, and is configured to rotate in a predetermined manner when refrigerator compartment door 211 is opened or closed.

[0017] Furthermore, when manually photographing the refrigerator compartment 21 using the camera unit 3, the left and right refrigerator compartment doors 211, 212 are provided with one photographing button 7 pressed by the user. In the example of Fig. 3, the left refrigerator compartment door 211 (door) has the photographing button 7 provided at the bottom of the surface opposite to the hinge 211a (see Fig. 4) of this refrigerator compartment door 211 (similar to the right refrigerator compartment door 212). From another perspective, when the left and right refrigerator compartment doors 211, 212 are closed (see Fig. 1), the photographing button 7 is provided on the surface where one of the refrigerator compartment doors 211, 212 faces the other (near the rotating partition 211d in Fig. 3).

[0018] Shooting when at least one of these two shooting buttons 7 is pressed by the user is called "manual shooting." Moreover, shooting performed by the control unit 8 (see FIG. 7) based on the elapsed time from the timing when the refrigerator compartment doors 211, 212 start to open is called "automatic shooting."

[0019] FIG. 4 is a plan view of the refrigerator 100 with the left and right refrigerator compartment doors 211, 212 open. In the example of Fig. 4, the main body 31 of the camera unit 3 is disposed slightly to the left of the center of the housing 1 of the refrigerator 100 in the left-right direction. To explain in more detail, the main body 31 of the camera unit 3 is disposed directly above the joint of the refrigerator compartment doors 211, 212 in the closed state (see also Fig. 1). By disposing the main body 31 in this manner, for example, when the refrigerator compartment doors 211, 212 are opened, food etc. in the door pockets 211c, 212c are more likely to come into the field of view of the camera unit 3.

[0020] Furthermore, when using refrigerator 100, the user is often conscious of the seam (boundary) of refrigerator compartment doors 211, 212. Therefore, by arranging main body 31 directly above the seam of refrigerator compartment doors 211, 212, the user can be prevented from feeling uncomfortable compared to when main body 31 is positioned off the seam in the left-right direction.

[0021] FIG. 5 is a perspective view of the camera unit 3 as viewed obliquely from below. The main body 31 of the camera unit 3 shown in Fig. 5 includes the lens 31a (see also Fig. 2) described above, as well as a case 31b and a camera LED 31c (Light Emitting Diode: illumination unit). In the example of Fig. 5, the case 31b of the main body 31 is generally a rectangular parallelepiped elongated in the front-rear direction. A circular hole (not shown) is provided on the underside near the front end of the case 31b, and the lens 31a is exposed through this hole. The camera LED 31c is provided on the rear side (rear side) of the lens 31a.

[0022] The camera LED 31c is a light source that irradiates light onto the refrigerator compartment 21 (see FIG. 3) and door pockets 211c, 212c (see FIG. 3) so that the camera unit 3 can photograph the refrigerator compartment 21 and the like under appropriate brightness. The camera LED 31c (illumination unit) is provided on the outside of the housing 1 (see FIG. 2) of the refrigerator 100. Although details will be described later, in the first embodiment, the camera unit 3 photographs the refrigerator compartment 21 and the like (see FIG. 3) with the interior light 4 (see FIG. 7) of the refrigerator 100 turned off and the camera LED 31c turned on.

[0023] As shown in Fig. 5, by providing lens 31a in front of camera LED 31c, reflected light from refrigerator compartment 21, etc. (see Fig. 3) can easily enter lens 31a when refrigerator compartment doors 211, 212 are open. Therefore, food stored in refrigerator compartment 21, etc. can be properly photographed. Note that camera LED 31c is preferably positioned forward of the front end (opening 10 of housing 1) of housing 1 (see Fig. 2) of refrigerator 100. This makes it easier for light emitted from camera LED 31c to enter refrigerator compartment 21, etc., and ultimately makes it easier to obtain clear photographing results.

[0024] FIG. 6A is a front view of the camera unit 3. FIG. 6A, the main body 31 of the camera unit 3 is disposed above the support portion 32. Moreover, the main body 31 is disposed near the center of the support portion 32 in the left-right direction.

[0025] FIG. 6B is a cross-sectional view taken along line II-II of FIG. 6A. As shown in Fig. 6B, the camera unit 3 includes a lens 31a (see Fig. 5), a camera LED 31c (see Fig. 5), a case 31b, and a circuit board 31d. The circuit board 31d is a printed circuit board on which an image sensor 31e (see Fig. 7) and a camera / communication control SoC 31f (System-on-a-Chip: see Fig. 7) are mounted. By housing the lens 31a, the camera LED 31c, and the circuit board 31d in one case 31b in this way, the camera unit 3 can be made smaller and the length of the wiring can be shortened, compared to when these are provided separately.

[0026] FIG. 7 is a diagram showing the system configuration including the camera unit 3 of the refrigerator 100. 7, in addition to the camera unit 3, refrigerator 100 is equipped with an interior light 4, a buzzer 5, a door sensor 6, a photography button 7, and a control unit 8. The interior light 4 is a light source that irradiates light into the interior of refrigerator 100, and is provided inside the housing 1. The buzzer 5 is sounded when the interior light 4 is turned off, for example, during photography by the camera unit 3, and is provided at a predetermined location in the housing 1.

[0027] The door sensor 6 outputs a predetermined signal indicating the open / closed state of the refrigerator compartment doors 211, 212 to the control unit 8. Although one door sensor 6 is shown in Fig. 7, in reality, a door sensor 6 corresponding to the left refrigerator compartment door 211 (see Fig. 1) and another door sensor 6 corresponding to the right refrigerator compartment door 212 (see Fig. 1) are provided. For example, when the left refrigerator compartment door 211 is opened, an open signal is output from the door sensor 6 corresponding to this refrigerator compartment door 211 to the control unit 8. The shooting button 7 (see also Fig. 3) is a button that is pressed by the user when manually shooting using the camera unit 3, as described above.

[0028] The control unit 8 is, for example, a microcomputer, and although not shown, is configured to include electronic circuits such as a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and various interfaces. The CPU reads out a program stored in the ROM and loads it into the RAM, and executes various processes. The control unit 8 performs the above-mentioned "automatic shooting", and also performs "manual shooting" when the shooting button 7 is pressed. Details of the processes performed by the control unit 8 will be described later.

[0029] The camera unit 3 includes a lens 31a (see FIG. 5) and a camera LED 31c (see FIG. 5), as well as an imaging element 31e, a camera / communication control SoC 31f, and a wireless LAN unit 31g. The imaging element 31e is an element that photoelectrically converts light incident through the lens 31a and generates captured image data. For example, a CCD sensor (Charge Coupled Device) or a CMOS sensor (Complementary Metal Oxide Semiconductor) is used as the imaging element 31e.

[0030] The camera / communication control SoC31f is an integrated circuit designed to integrate circuits having microcomputer functions and other applied functions on a single chip and function in cooperation with each other. The camera / communication control SoC31f communicates with the control unit 8 in a predetermined manner and outputs an image capture command to the image sensor 31e. As a result, captured image data is input from the image sensor 31e to the camera / communication control SoC31f. Note that the camera / communication control SoC31f is an example of an integrated circuit, and other types of integrated circuits may be used.

[0031] The photographing unit 30 shown in Fig. 7 has a function of photographing at least the refrigerator compartment 21 (storage compartment) and is provided outside the housing 1 (see Fig. 1) of the refrigerator 100. As shown in Fig. 7, the photographing unit 30 includes a lens 31a, an image sensor 31e, and a camera / communication control SoC 31f.

[0032] The wireless LAN unit 31g transmits the captured image data output from the camera / communication control SoC 31f to the router 51. The router 51 is a communication relay device, and transmits the captured image data received from the wireless LAN unit 31g to the server 53 via the network 52. The server 53 performs a predetermined process on the captured image data and stores it in association with the identification information of the refrigerator 100. Furthermore, when the server 53 receives a request signal for an image of the interior of the refrigerator 100 from the user's mobile terminal 54, it transmits the captured image data to the mobile terminal 54. As such a mobile terminal 54, a mobile phone, a smartphone, a tablet, a wearable terminal, or the like may be used.

[0033] <Processing of control unit> 8A and 8B are flowcharts of the automatic photography process executed by the control unit of the refrigerator (see FIG. 7 as appropriate). In the following, as an example, a process in which the camera unit 3 captures an image of the refrigerator compartment 21 when at least one of the refrigerator compartment doors 211, 212 (see FIG. 3) is open will be described. Note that at the time of "START" in FIG. 8A, the left and right refrigerator compartment doors 211, 212 (see FIG. 3) are assumed to be in a closed state. Also, the user can operate the mobile terminal 54 (see FIG. 7) to switch the automatic shooting setting on / off and the manual shooting setting on / off. In the examples of FIGS. 8A and 8B, it is assumed that the automatic shooting setting is on (automatic shooting possible) and further that the manual shooting setting is also on (manual shooting possible).

[0034] In step S101, the control unit 8 determines whether or not an open signal has been input from the door sensor 6. That is, the control unit 8 determines whether or not at least one of the left and right refrigerator compartment doors 211, 212 (see FIG. 3) has been opened. In step S101, if an open signal has not been input from the door sensor 6 (S101: No), the control unit 8 repeats the process of step S101. On the other hand, in step S101, if an open signal has been input from the door sensor 6 (S101: Yes), the process of the control unit 8 proceeds to step S102.

[0035] In step S102, the control unit 8 turns on the interior light 4 of the refrigerator 100 and also turns on the camera LED 31c. That is, the control unit 8 turns on the interior light 4 and also turns on the camera LED 31c (illumination unit) immediately after at least one of the refrigerator compartment doors 211, 212 (doors) is opened. As described later, the control unit 8 turns off the interior light 4 after a predetermined time has elapsed since an open signal is input from the door sensor 6 (S103: Yes, S104), but until then, the interior light 4 is turned on. This allows light to be irradiated onto the refrigerator compartment 21, making it easier for the user to view the refrigerator compartment 21. In addition, the user is less likely to feel uncomfortable compared to when the interior light 4 is turned off when the user opens at least one of the refrigerator compartment doors 211, 212.

[0036] Furthermore, the control unit 8 turns on the camera LED 31c together with the interior light 4 (S102), thereby preparing for the subsequent photographing (S106). Also, the time required for exposure adjustment during photographing can be shortened compared to when the interior light 4 and the camera LED 31c are turned on at different times. The camera LED 31c remains lit until both the left and right refrigerator compartment doors 211, 212 are closed.

[0037] Next, in step S103, the control unit 8 determines whether or not a predetermined time has elapsed since the open signal was input from the door sensor 6. This predetermined time (e.g., several seconds) is the time until the interior light 4 is turned off (S104), and is set in advance. For example, the predetermined time in step S103 is set in advance, taking into consideration the time for the subsequent exposure adjustment (S105) so that there is a high possibility that the refrigerator compartment doors 211, 212 are opened to a predetermined opening angle or more when the image is captured (S106).

[0038] In step S103, if the predetermined time has not elapsed since the open signal was input from the door sensor 6 (S103: No), the control unit 8 repeats the process of step S103. On the other hand, if the predetermined time has elapsed since the open signal was input from the door sensor 6 (S103: Yes), the control unit 8 proceeds to the process of step S104.

[0039] In step S104, the control unit 8 turns off the interior light 4 and sounds the buzzer 5. That is, the control unit 8 sounds the buzzer 5 when turning off the interior light 4 after at least one of the refrigerator compartment doors 211, 212 (doors) is opened. By sounding the buzzer 5 in this manner, it is possible to reduce the sense of discomfort felt by the user when the interior light 4 is turned off, and it is also possible for the user to easily understand that the light is turned off to take a picture.

[0040] Next, in step S105, the control unit 8 performs exposure adjustment. That is, the control unit 8 adjusts the gain (sensitivity to brightness) when amplifying the electric signal output from the image sensor 31e, and also adjusts the shutter speed of the camera unit 3, etc. For example, the control unit 8 adjusts the gain so that the average value of the luminance of the captured image data falls within a predetermined range. By performing exposure adjustment in this manner, it is possible to suppress blown-out highlights (halation) and crushed shadows in the captured image. Note that image input and exposure adjustment may be repeated in a predetermined manner from when at least one of the refrigerator compartment doors 211, 212 is opened until both refrigerator compartment doors 211, 212 are closed.

[0041] Next, in step S106, the control unit 8 outputs a shooting command. That is, the control unit 8 captures, as the shot image data, an electrical signal that is output from the image sensor 31e by photoelectric conversion and further amplified by a predetermined gain. In this way, after at least one of the refrigerator compartment doors 211, 212 (doors) is opened (S101: Yes), the control unit 8 turns off the interior light 4 and further causes the shooting unit 30 to shoot with the camera LED 31c (illumination unit) turned on (S102, S104, S106).

[0042] 8A and 8B, it is preferable that the control unit 8 causes the photographing unit 30 to photograph after a predetermined time (e.g., several seconds) has elapsed since the interior light 4 was turned off. By providing a predetermined time between the turning off of the interior light 4 and photographing in this manner, exposure adjustment (S105) can be appropriately performed.

[0043] 8A and 8B, the control unit 8 may take a photograph every time a predetermined time has elapsed since the first photographing. The length of this predetermined time may be the same as the predetermined time in step S103, or may be different.

[0044] In step S107, control unit 8 recognizes the open angle of refrigerator compartment doors 211, 212. For example, control unit 8 performs image processing such as edge detection to recognize the open angle of the refrigerator compartment door 211, 212 that is open. Note that the "open angle" is the angle at which refrigerator compartment doors 211, 212 are open, and is recognized, for example, based on the front end of housing 1 (see FIG. 4) when refrigerator 100 is viewed in plan.

[0045] Next, in step S108 in Fig. 8B, the control unit 8 judges whether the open angle of the refrigerator compartment doors 211, 212 is equal to or greater than a predetermined value. This predetermined value is a threshold value of the open angle that is a criterion for deciding whether to save the photographed result (S109) or discard it (S110), and is set in advance.

[0046] In step S108, if the opening angle of at least one of the refrigerator compartment doors 211, 212 is equal to or greater than the predetermined value (S108: Yes), the control unit 8 proceeds to step S109. In step S109, the control unit 8 stores the image capturing result, and proceeds to step S112. This is because, if the opening angle of the refrigerator compartment doors 211, 212 is equal to or greater than the predetermined value, there is a high possibility that the refrigerator compartment 21, etc. have been appropriately captured.

[0047] The photographing results when only the left refrigerator compartment door 211 is open, the photographing results when only the right refrigerator compartment door 212 is open, and the photographing results when both the left and right refrigerator compartment doors 211, 212 are open may be stored in different memory areas.

[0048] Furthermore, in step S108, if the opening angles of the refrigerator compartment doors 211, 212 are both less than the predetermined value (S108: No), the process by the control unit 8 proceeds to step S110. In step S110, the control unit 8 discards the image capture result. This is because, if the opening angles of the refrigerator compartment doors 211, 212 are less than the predetermined value, there is a high possibility that the refrigerator compartment 21 etc. have not been properly imaged.

[0049] Then, in step S111, the control unit 8 sounds the buzzer 5. That is, when an image is captured with the opening angle of the refrigerator door 211, 212 (door) less than a predetermined value (S108: No), the control unit 8 sounds the buzzer 5 (S111). This notifies the user that the current captured image has been discarded. The sound of the buzzer 5 in step S111 and the sound of the buzzer 5 in step S104 (when the interior light 4 is turned off) may be different so that the user can distinguish between the two. After the process of step S111, the process of the control unit 8 proceeds to step S112. Instead of sounding the buzzer 5, the refrigerator or the mobile terminal 54 used by the user may output light, vibration, or other output that the user can perceive.

[0050] In step S112, the control unit 8 turns on the interior light 4. That is, the control unit 8 turns on the interior light 4 while keeping the camera LED 31c turned on. In this way, after the control unit 8 causes the photographing unit 30 to photograph (S106 in FIG. 8A), it turns on the interior light 4 (S112 in FIG. 8B). This brightly illuminates the refrigerator compartment 21, making it easier for the user to view the refrigerator compartment 21.

[0051] Next, in step S113, the control unit 8 determines whether or not a close signal has been input from the door sensor 6. If a close signal has not been input from the door sensor 6 (S113: No), the control unit 8 returns to step S102. On the other hand, if a close signal has been input from the door sensor 6 (S113: Yes), the process of the control unit 8 proceeds to step S114.

[0052] In step S114, the control unit 8 turns off the interior light 4 and the camera LED 31c. In step S115, the control unit 8 selects a shooting result. For example, the control unit 8 selects the most recent of the multiple saved shooting results as the shooting result to be transmitted to the server 53. In this way, by completing the saving or discarding of the shooting result for each shooting, it is possible to reduce the amount of storage required by the control unit 8, reduce the amount of data required to be transmitted to the server 53, or, in the case of discarding, to appropriately notify the user of that fact. Of course, in the case of saving, it is also possible to notify the user of that fact. It is sufficient that the user can recognize whether the shooting result has been processed by saving or discarding. In order to make it easier for the user to recognize each shooting result, the time interval between shootings can be, for example, 3 seconds or more. In this way, when an output is made that allows the user to recognize a certain shooting result, the user can easily understand that the output is for that shooting result. For example, if the time interval is less than 3 seconds, there is a risk that when an output is made, it will be difficult for the user to distinguish whether the output is for a certain shooting result or an output for a shooting result one or more before or after that. On the other hand, if the time interval is too long, the open time of the refrigerator compartment doors 211, 212 will be long, causing a large amount of cold air to flow out of the refrigerator compartment, so the time interval is 8 seconds or less, preferably 5 seconds or less. If step S111 is not executed, shooting may be performed multiple times in succession in step S106. In this case, in steps S108 to S110, processing can be performed on each of the multiple pieces of image data.

[0053] In step S116, the control unit 8 transmits the photographed result to the server 53. That is, the control unit 8 transmits the photographed image data to the server 53 via the wireless LAN unit 31g, the router 51, and the network 52 shown in Fig. 7 in this order. After performing the process of step S116, the control unit 8 ends the series of processes (END).

[0054] In addition to the right and left areas of refrigerator compartment 21 (see FIG. 3), server 53 extracts the area of ​​left refrigerator compartment door 211 (see FIG. 3) and the area of ​​right refrigerator compartment door 212 (see FIG. 3) from the photographed results. Then, server 53 updates the images of each area individually to the latest ones. Furthermore, in response to a request signal from mobile terminal 54, captured image data is transmitted from server 53 to mobile terminal 54, and is displayed in a predetermined manner on a display (not shown) of mobile terminal 54.

[0055] FIG. 9 is an explanatory diagram showing the process in which the refrigerator compartment doors 211, 212 are opened and closed in time sequence. In addition, in Fig. 9, the states of the refrigerator compartment doors 211, 212, as well as the states of the interior light 4 and the camera LED 31c are shown in chronological order in the alphabetical order of A, B, C, . . . , J, K, and L. In Fig. 9, when the interior light 4 of the refrigerator 100 is on, the interior light 4 is shown (for example, state B), and when the interior light 4 is off, the interior light 4 is not shown (for example, state A). Similarly, in Fig. 9, when the camera LED 31c is on, the camera LED 31c is shown (for example, state B), and when the camera LED 31c is off, the camera LED 31c is not shown (for example, state A).

[0056] In the example of FIG. 9, the user opens the left refrigerator door 211 from state A where both the left and right refrigerator doors 211, 212 are closed (states B, C, and D). After this, the camera LED 31c is maintained in a lit state until both the left and right refrigerator doors 211, 212 are closed (S102 in FIG. 8A). Also, until a predetermined time has elapsed since the left refrigerator door 211 began to be opened (S103: No), both the interior light 4 and the camera LED 31c are lit (S102) as shown in states B and C. Then, when a predetermined time has elapsed since the left refrigerator door 211 began to be opened (S103: Yes in FIG. 8A), as shown in state D, the interior light 4 is turned off and automatic photography is performed with the camera LED 31c turned on (S104 to S106). After that, in state E, the interior light 4 is turned on again (S112 in FIG. 8B).

[0057] In states E and F, the left refrigerator door 211 is closed while the right refrigerator door 212 is open. In this case, as shown in state F, the interior light 4 is turned off and the camera LED 31c is turned on, and automatic photography is performed again (S104 to S106 in FIG. 8A). Thereafter, in the example of FIG. 9, as shown in states G, H, and I, the user closes the left refrigerator door 211 once and then opens it again.

[0058] Even in such a case, as shown in state I, automatic photography is performed again with the interior light 4 turned off and the camera LED 31c turned on (S104 to S106 in FIG. 8A). This makes it possible to photograph with both the left and right refrigerator compartment doors 211, 212 open. Thereafter, as shown in state J, the interior light 4 is turned on again (S112 in FIG. 8B). In this way, every time at least one of the left and right refrigerator compartment doors 211, 212 is opened, the control unit 8 photographs the refrigerator compartment 21 etc. with the camera unit 3.

[0059] Furthermore, as shown in state K, when the user presses the photographing button 7 (see FIG. 3), the control unit 8 performs manual photographing. Although this case is not particularly shown in the flowcharts of FIG. 8A and FIG. 8B, the control unit 8 performs the following process. That is, when the photographing button 7 is pressed after at least one of the refrigerator compartment doors 211, 212 (doors) is opened, the control unit 8 turns off the interior light 4 and further causes the photographing unit 30 to photograph with the camera LED 31c (illumination unit) turned on (state K). Note that when turning off the interior light 4, the control unit 8 may sound the buzzer 5 in order to reduce the user's sense of discomfort. After performing manual photographing, the control unit 8 turns on the interior light 4 again as shown in state L.

[0060] When photographing based on the operation of the photographing button 7 (manual photographing), it is preferable that the control unit 8 displays the photographing result on the mobile terminal 54 (see FIG. 7) in priority to other photographing results (automatic photographing results). This allows the user to check the result of photographing by pressing the photographing button 7 on his / her own mobile terminal 54 (see FIG. 7).

[0061] Furthermore, it is preferable that the control unit 8 does not capture an image again (automatic capture) after capturing an image based on the operation of the capture button 7 (after manual capture) while at least one of the refrigerator compartment doors 211, 212 (doors) is open. This is because when the result of manual capture is preferentially transmitted, there is no particular need to capture an image automatically thereafter.

[0062] Furthermore, if the user does not completely close at least one of refrigerator doors 211, 212, leaving the door half-open, automatic photography is performed at predetermined time intervals, but the photographed results are not sent to server 53 (see FIG. 7). This is because, in the case of a half-open door, the open angle of refrigerator doors 211, 212 is usually less than a predetermined value (S108: No in FIG. 8B), and the photographed results are discarded (S110).

[0063] FIG. 10 shows an example of the result of shooting with the camera unit without image processing. In Fig. 10, the upper side of the page is the front and the lower side is the rear, so the left and right are reversed compared to Fig. 3 (the same applies to Figs. 11, 12A, and 12B). In the example of Fig. 10, both the left and right refrigerator compartment doors 211, 212 are open, and in addition to the refrigerator compartment 21, door pocket 211c of the left refrigerator compartment door 211 and door pocket 212c of the right refrigerator compartment door 212 are also photographed.

[0064] In particular, a fisheye lens is used as lens 31a of camera unit 3 (see FIG. 5), allowing imaging with a wide angle of view. However, without image processing, as shown in FIG. 10, ridgelines that are actually straight appear curved, and even for ridgelines of the same length, the number of pixels in the captured image decreases as the ridgeline moves away from camera unit 3.

[0065] FIG. 11 shows an example of the result of photographing by the camera unit 3 after image processing. For example, predetermined image processing is performed by server 53 (see FIG. 7) based on the photographed result of FIG. 10, converting the images into images as if they were seen from the front of refrigerator compartment 21 and left and right refrigerator compartment doors 211, 212. By displaying such images on mobile terminal 54 (see FIG. 7), the user can see at a glance what foods are stored in refrigerator compartment 21 etc. Next, the details of the exposure adjustment process (S105 in FIG. 8A) by the control unit 8 will be described in detail with reference to FIG. 12A (first embodiment) and FIG. 12B (comparative example).

[0066] FIG. 12A is an explanatory diagram showing candidates for areas serving as a reference when performing exposure adjustment in the refrigerator according to the first embodiment. As described above, in the exposure adjustment process (S105 in FIG. 8A), the gain for amplifying the electrical signal output from the image sensor 31e (see FIG. 7) is adjusted. In the first embodiment, at least one of the three rectangular regions 61-63 shown in FIG. 12A is used as a reference for indicating the degree of image brightness in exposure adjustment. That is, at least one of the region 61 of the left refrigerator compartment door 211, the region 62 of the right refrigerator compartment door 212, and a predetermined region 63 between these regions 61, 62 is used. The positions and sizes of the regions 61-63 may be changed as appropriate based on the opening angles of the refrigerator compartment doors 211, 212.

[0067] In the example of FIG. 12A, the left refrigerator door 211 is open, but the right refrigerator door 212 is closed. In such a case, the control unit 8 performs exposure adjustment based on the average luminance value of the area 61 corresponding to the left refrigerator door 211. That is, the control unit 8 adjusts the gain when amplifying the electric signal output from the image sensor 31e (see FIG. 7) so that the average luminance value of the area 61 falls within a predetermined range. As a result, the area 62 of the right refrigerator door 212, which is in the closed state, and the area 63, part of which is outside the left refrigerator door 211, are excluded from the standard when adjusting the gain. Therefore, even if the surroundings of the refrigerator are dark, the control unit 8 can appropriately adjust the exposure without increasing the gain too much.

[0068] In this way, when one of the right refrigerator compartment door 212 (right door) and the left refrigerator compartment door 211 (left door) is opened, the control unit 8 adjusts the exposure gain during shooting based on the area corresponding to the one door within the field of view of the shooting unit 30. Note that signals indicating the open / closed states of the left and right refrigerator compartment doors 211, 212 are output to the control unit 8 from the left and right door sensors 6 (see FIG. 7), respectively.

[0069] Also, exposure adjustment may be performed based on the area with the highest average image brightness among the three areas 61-63. That is, the control unit 8 adjusts the exposure gain during shooting based on the area with the highest average image brightness among the multiple areas 61-63 included in the field of view of the shooting unit 30. This makes it possible to prevent the gain from becoming too high, and thus to prevent blown-out highlights (halation) in the shot image.

[0070] Furthermore, exposure adjustment may be performed based on an area of ​​the left and right refrigerator compartment doors 211, 212 that includes at least a part of the door to which an open signal is input from the door sensor 6 and that has the highest average image brightness. This allows the control unit 8 to appropriately set the gain in exposure adjustment even in a situation where light irradiated from outside the refrigerator 100 is reflected on the surface of the refrigerator compartment door 212 in the closed state, for example. This makes it possible to prevent whiteout and blackout from occurring in the captured image.

[0071] FIG. 12B is an explanatory diagram showing an area serving as a luminance reference when performing exposure adjustment in the refrigerator according to the comparative example. In the comparative example of FIG. 12B, one rectangular area 60 is set as a reference for adjusting the exposure. For example, when the left and right refrigerator compartment doors 211, 212 are open, the rectangular area 60 is set to include both of these refrigerator compartment doors 211, 212. When one large area 60 is used in this manner, in a situation where the surroundings of the refrigerator 100 are dark, the right refrigerator compartment door 212 appears dark in the photograph, so the gain is set higher to increase the sensitivity to light. As a result, whiteout is likely to occur in the left refrigerator compartment door 211 in the open state and in the left half of the refrigerator compartment 21 in the photographed image.

[0072] In contrast, in the first embodiment (see FIG. 12A), for example, the control unit 8 selects an area that is used as a reference for adjusting the exposure based on the open / closed state of the refrigerator compartment doors 211, 212. This makes it possible to suppress whiteout in the captured image and display a clear image that is easy for the user to view on the mobile terminal 54 (see FIG. 7).

[0073] The wall surface of the refrigerator compartment 21 (storage compartment) and the inner panels 211b, 212b (see FIG. 3) of the refrigerator compartment doors 211, 212 (doors) may each be white. Here, "white" refers to a color that reflects at a reflectance of 50% or more across substantially the entire visible light spectrum. From another perspective, "white" refers to a color in which the brightness of each element of R (Red), G (Green), and B (Blue) in the RGB color system is substantially the same and that reflects at a reflectance of 50% or more.

[0074] When the wall surfaces of the refrigerator compartment 21, etc. are white, the captured image tends to be overexposed, but the controller 8 can suppress overexposure by appropriately selecting an area that serves as a reference for adjusting the exposure. In this way, according to the first embodiment, the refrigerator compartment 21, etc. can be appropriately photographed using the camera unit 3. This makes it easier for the user to view the state of the refrigerator compartment 21, etc. by looking at the mobile terminal 54.

[0075] Second Embodiment The second embodiment differs from the first embodiment in that a photo capture button 7A (see FIG. 13) is provided near handles 211e, 212e (see FIG. 13) provided on the undersides of the left and right refrigerator compartment doors 211, 212 (see FIG. 13). The rest of the second embodiment is similar to the first embodiment. Therefore, only the parts that differ from the first embodiment will be described, and a description of the overlapping parts will be omitted.

[0076] FIG. 13 is a front view and a bottom view of left and right refrigerator compartment doors 211, 212 included in a refrigerator 100A according to the second embodiment. In Fig. 13, a bottom view of the refrigerator compartment doors 211, 212 is shown below the front view of the refrigerator compartment doors 211, 212. As shown in Fig. 13, a handle portion 211e that is recessed upward is provided on the bottom surface of the left refrigerator compartment door 211 over a predetermined length in the horizontal direction.

[0077] Also, on the underside of the left refrigerator compartment door 211, inside the handle portion 211e (opposite the hinge shaft 211f), there is provided a photography button 7A that is pressed by the user when taking a manual photograph. That is, the photography button 7A is provided outside the handle portion 211e of the refrigerator compartment door 211 (door) and near this handle portion 211e. Similarly, a handle portion 212e and photography button 7A are provided on the underside of the right refrigerator compartment door 212.

[0078] By arranging the photographing button 7A in this manner, for example, when a user places his / her hand on the handle 212e to open the right refrigerator door 212, it is possible to prevent the user from accidentally pressing the photographing button 7A unintentionally. When a user places his / her right hand on the right handle 211e, the palm of the hand faces upward and the thumb of the right hand is positioned on the outer side (right side). In the example of FIG. 13, the photographing button 7A is provided on the inner side (left side) of the handle 212e. Therefore, when a user places his / her right hand on the handle 212e, it is possible to prevent the thumb of the right hand from accidentally touching the photographing button 7A. The same can be said about the photographing button 7A of the left refrigerator door 211.

[0079] In addition, since the photographing button 7A is provided near the handle portion 212e, the user can move his / her hand while placing his / her hand on the refrigerator compartment door 212 and press the photographing button 7A by the touch of his / her fingertip. Therefore, when the user presses the photographing button 7A, there is no particular need to release the hand placed on the refrigerator compartment door 212, and operability can be improved. In addition, since the user can take a photograph with the refrigerator compartment door 212 opened to a predetermined position and the rotation of the refrigerator compartment door 212 stopped, blurring of the photographed image can be suppressed. Furthermore, compared to the case where the photographing button 7A is provided on the inside of the refrigerator compartment door 212 or on the wall surface of the refrigerator compartment 21, it is possible to suppress the user's hand from appearing in the photographed image. In addition, by providing the photographing button 7A near the handle portion 212e, it is also possible to share the wiring of the rotating partition 211d (see FIG. 3) and the operation panel (not shown) with the wiring of the photographing button 7A. The process of the control unit 8 (see FIG. 7) based on the operation of the shooting button 7A is similar to that of the first embodiment (see FIGS. 8A and 8B), and therefore will not be described.

[0080] FIG. 14A is a cross-sectional perspective view of the lower part of right refrigerator compartment door 212 equipped in the refrigerator. The configuration in Fig. 14A corresponds to Fig. 13. As shown in Fig. 14A, the photographing button 7A of the right-side refrigerator door 212 includes a resin switch portion 71A that is pressed by the user, a laminate film 72A that is attached to the outside of the switch portion 71A, and a switch board 73A that generates a predetermined electric signal when the switch portion 71A is pressed. A plurality of wires 80 are connected to the switch board 73A. These wires 80 include power lines and signal lines. The left-side refrigerator door 211 (see Fig. 13) also has a similar configuration.

[0081] FIG. 14B is a cross-sectional view of the lower part of right refrigerator compartment door 212 equipped in the refrigerator. The configuration in FIG. 14B corresponds to FIG. 13 and FIG. 14A. As shown in FIG. 14B, a partition wall 92 is provided inside the refrigerator compartment door 212 (door) to separate an area 91a around the photographing button 7A from the rest. The partition wall 92 is provided with an insertion hole 93 for the wiring 80 connected to the photographing button 7A. A seal member 94 that closes the insertion hole 93 is provided in the insertion hole 93. For example, urethane foam is used as the seal member 94. The wiring 80 is inserted through the seal member 94. That is, the insertion hole 93 is closed by the seal member 94 with the wiring 80 inserted through the insertion hole 93.

[0082] By providing such a seal member 94, when a heat insulating material such as urethane is injected (filled) into the internal space of the refrigerator compartment door 212 during manufacturing, it is possible to prevent the heat insulating material from leaking toward the photographing button 7A side through the insertion hole 93. In addition, it becomes possible to draw the wiring 80 connected to the photographing button 7A to the area 91b above the partition wall 92.

[0083] <<Variations>> Although the refrigerator 100 and the like according to the present disclosure have been described above with reference to the respective embodiments, the refrigerator 100 and the like are not limited to these descriptions and may be modified in various ways. For example, in each embodiment, the camera unit 3 is provided on the top surface of the housing 1 of the refrigerator 100 (see FIG. 2), but the present invention is not limited to this. That is, the camera unit 3 may be provided at another predetermined position on the outside of the housing 1.

[0084] In addition, in each embodiment, the camera LED 31c (see FIG. 5) is disposed on the rear side of the lens 31a (see FIG. 5), but the present invention is not limited to this. For example, the camera LED may be disposed on the front side of the lens 31a. Furthermore, a plurality of camera LEDs may be disposed in a predetermined position.

[0085] In each embodiment, as an example, a case has been described in which the camera unit 3 photographs the interior of the refrigerator compartment doors 211, 212 in addition to the refrigerator compartment 21, but this is not limited to this. For example, the camera unit 3 may photograph at least one of the drawer-type ice-making compartment door 221 (door: see FIG. 1), the upper freezer compartment door 231 (door), the vegetable compartment door 241 (door), and the lower freezer compartment door 251 (door).

[0086] In each embodiment, the configuration has been described in which the photo-taking button 7 (see FIG. 3) is provided on the left and right refrigerator compartment doors 211, 212, but this is not limited to the above. For example, a photo-taking button (not shown) may be provided on at least one of the drawer-type ice-making compartment door 221 (see FIG. 1), the upper freezer compartment door 231, the vegetable compartment door 241, and the lower freezer compartment door 251.

[0087] In each embodiment, the control unit 8 turns on the interior light 4 and the camera LED 31c (S102) when at least one of the refrigerator doors 211, 212 is opened (S101: Yes in FIG. 8A), but this is not limiting. That is, when at least one of the refrigerator doors 211, 212 is opened, the control unit 8 may turn on the interior light 4 and the camera LED 31c with a predetermined lag.

[0088] In each embodiment, the case where the control unit 8 keeps the camera LED 31c on from when at least one of the left and right refrigerator compartment doors 211, 212 is opened until both refrigerator compartment doors 211, 212 are closed has been described, but this is not limiting. For example, when exposure adjustment (S105 in FIG. 8A) is not being performed, the control unit 8 may turn off the camera LED 31c at a predetermined timing.

[0089] In the second embodiment, the configuration in which the concave handles 211e, 212e are provided on the undersides of the refrigerator compartment doors 211, 212 (see FIG. 13) has been described, but the present invention is not limited to this. For example, the refrigerator compartment doors 211, 212 may be provided with a predetermined door handle (not shown). In this case, a photographing button (not shown) may be provided on the outside of the door handle, or a photographing button (not shown) may be provided at a predetermined position on the door handle itself.

[0090] The refrigerator 100 and the like described in each embodiment are merely examples, and can be applied to other types of refrigerators or storage units. For example, each embodiment can be applied to a refrigerator with a single-opening refrigerator compartment door (not shown) or a portable refrigerator (not shown). The image capturing device can receive information from a computing device or a sensor provided in the storage unit by wire or wireless communication to capture images, and can be a separate image capturing device from the storage unit.

[0091] In addition, each embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to having all of the configurations described. In addition, it is possible to add, delete, or replace part of the configuration of an embodiment with other configurations. Furthermore, the above-mentioned mechanisms and configurations are those considered necessary for the explanation, and do not necessarily show all mechanisms and configurations of the product. [Explanation of symbols]

[0092] 100,100A Refrigerator (storage) 1 Case 10 aperture 21 Refrigerator (storage room) 211 Refrigerator door (door, first door) 212 Refrigerator door (door, first door) 211a, 212a Hinge 211b,212b Inner plate 211e, 212e Handle section 22 Ice room (storage room) 221 Ice Room Door (Door, Second Door) 23 Upper freezer (storage room) 231 Upper freezer door (door, second door) 24 Vegetable room (storage room) 241 Vegetable compartment door (door, second door) 25 Lower freezer (storage room) 251 Lower freezer door (door, second door) 3 Camera unit 4 Interior light 5 Buzzer 6 Door Sensor 7,7A Shooting button 8. Control Unit 31c Camera LED 30 Photography Department 54 Mobile Devices 61,62,63 area 80 Wiring 91a area 92 Bulkhead 93 Insertion hole 94 Sealing material

Claims

1. a housing having a storage chamber; A door that closes an opening of the storage chamber; A door sensor that detects whether the door is open or closed; An imaging unit that images at least the storage chamber, The storage facility is provided with a control unit that, after the door opening operation is detected by the door sensor, issues a photographing command to the photographing unit while the opening operation is being detected, and recognizes the degree to which the door is opened based on the image captured by the photographing unit.

2. The control unit is A determination is made as to whether to store or discard the latest photographed image based on the degree of opening of the door, and in accordance with the determination, if the image is to be stored, the next photograph is taken after the storage is completed, or if the image is to be discarded, the next photograph is taken after the discard is completed until the open door is closed; The criteria for the judgment are not to unconditionally judge whether the latest captured image should be stored, but rather, there is a possibility of either storing or discarding the image. The storage facility according to claim 1 ,

3. The doors include a first hinged door and a second drawer door; The second door is lower in height than the first door, An area in the field of view of the photographing unit in which the second door is photographed is set to be outside the scope of exposure adjustment in the photographing unit when the first door is opened. The storage facility according to claim 1 or 2, characterized in that

Citation Information

Patent Citations

  • Refrigerator appearance control method

    CN113048713A

  • Refrigerator and control method for the same

    EP2909549A1

  • Refrigerator

    JP2003042626A

  • Heat insulating door and its manufacturing method

    JP2004076989A

  • Sensor device, door device and refrigerator

    JP2006084132A