Refrigerator, image generation method, and program

Abstract

The present invention pertains to a refrigerator (100) in which at least one photographing device is arranged so that all storage conditions in the refrigerator can be confirmed.

Description

Refrigerator, image generation method, and program

[0001] The present disclosure relates to refrigerators and the like.

[0002] Patent Documents 1 and 2 disclose refrigerators that are provided with a camera unit that is provided on the outside of the housing and that takes pictures of the refrigerator compartment.

[0003] JP 2023-115562 A JP 2021-196069 A

[0004] However, there is room for improvement in the refrigerators disclosed in Patent Documents 1 and 2.

[0005] Therefore, the present disclosure provides a refrigerator that can be further improved.

[0006] A refrigerator according to one aspect of the present disclosure is a refrigerator in which at least one photographing device is arranged so that the storage status of the entire interior can be confirmed.

[0007] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium. The recording medium may also be a non-transitory recording medium.

[0008] The refrigerator of the present disclosure can be further improved.

[0009] Further advantages and effects of one aspect of the present disclosure will become apparent from the specification and drawings. Such advantages and / or effects are provided by some of the embodiments and configurations described in the specification and drawings, but not all of the configurations are necessarily required to obtain the advantages and effects.

[0010] FIG. 1 is a diagram showing an example of the configuration of a refrigerator in embodiment 1. FIG. 2 is a diagram showing a front view of the refrigerator in embodiment 1. FIG. 3 is a diagram showing how a refrigerator compartment and a freezer compartment are photographed in embodiment 1. FIG. 4 is a diagram showing how a vegetable compartment is photographed in embodiment 1. FIG. 5 is a diagram showing how a door pocket is photographed in embodiment 1. FIG. 6 is a diagram showing an example of a display displayed on a display of the refrigerator in embodiment 1. FIG. 7 is a diagram showing a comparison between a conventional example and embodiment 1. FIG. 8 is a diagram showing the configuration of a refrigerator in embodiment 2. FIG. 9 is a diagram showing the configuration of a conventional refrigerator.

[0011] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0012] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0013] In addition, each drawing is a schematic diagram and is not necessarily an exact illustration. In addition, the same components are denoted by the same reference numerals in each drawing.

[0014] (Background to One Aspect of the Present Disclosure) In conventional refrigerators, users had to open the refrigerator door to check the refrigerator's storage status. However, opening the refrigerator door causes fluctuations in the temperature and humidity inside the refrigerator, which can accelerate food deterioration and lead to food waste. Furthermore, repeatedly opening the door to check whether food is present wastes energy and increases CO2 emissions. Furthermore, users cannot instantly grasp ingredient information, particularly whether ingredients are present. This requires time to create recipes, which can extend the overall cooking time.

[0015] To solve these problems, the inventors of the present invention have investigated a new method for enabling a user standing in front of a refrigerator to view the interior of the refrigerator without opening the door. As a result, they have devised a technology for installing a display on the refrigerator and displaying the status of the interior through the display. Furthermore, they have discovered that in order to reduce the frequency with which users have to open the refrigerator door, it is effective to not only simply display an image taken by a camera, but also to display the interior in a way that minimizes blind spots.

[0016] One aspect of the present disclosure has been discovered based on the above findings.

[0017] First Embodiment FIG. 1 is a diagram showing a configuration example of a refrigerator 100 according to the present disclosure.

[0018] The refrigerator 100 of the present disclosure includes a refrigerator compartment 10, a vegetable compartment 30 located below the refrigerator compartment 10, a freezer compartment 40 located below the vegetable compartment 30, a first door 60 that swings both ways and is provided corresponding to the refrigerator compartment 10 and the vegetable compartment 30, and a second door 70 that is a drawer and is provided corresponding to the freezer compartment 40. The vegetable compartment 30 includes a container for storing food. The container may include an upper container C1 located on an upper level and a lower container C2 located on a lower level. Furthermore, the containers are configured to be movable forward and backward. The configuration for allowing the containers to be moved forward and backward may be realized, for example, by a rail device (not shown) or a sliding structure (not shown) that allows the containers to slide forward and backward. As shown in FIG. 1 , the upper container C1 and the lower container C2 may each be configured to be able to move forward and backward independently.

[0019] The freezing compartment 40 may include a container for storing food. The containers may include an upper container C3 located on an upper level of the freezing compartment 40 and a lower container C4 located on a lower level. The upper container C3 and the lower container C4 may be configured to be independently movable back and forth.

[0020] Alternatively, the lower container C4 may be configured to move forward and backward in conjunction with the movement of the second door 70, while the upper container C3 may be configured to move forward and backward independently. In this case, when the second door 70 is pulled forward, the lower container C4 moves forward together with the second door 70, allowing the user to first check the contents of the lower container C4. Then, when the upper container C3, which remains within the freezer compartment 40, is pulled forward, the user can check the contents of the upper container C3. The configuration for moving the containers in the freezer compartment 40 back and forth may be implemented, for example, by a rail device (not shown) or a sliding structure (not shown) that allows them to slide back and forth.

[0021] However, these are merely examples, and the upper container C1 and lower container C2 of the vegetable compartment 30 and the upper container C3 and lower container C4 of the freezer compartment 40 may be configured in any manner. In the present disclosure, each of the containers in the vegetable compartment 30 and the freezer compartment 40 has a two-layer structure including an upper container and a lower container, but the containers may have any number of layers, such as a three-layer structure or a four-layer structure. Alternatively, each of the containers in the vegetable compartment 30 and the freezer compartment 40 may have a single-layer structure.

[0022] The refrigerator 100 of the present disclosure further includes a plurality of photographing devices. The plurality of photographing devices include first photographing devices 1 and 2 that are arranged inside the first door 60 and that photograph the interior of the refrigerator, second photographing devices 3 and 4 that are arranged on the top surface 91 outside the refrigerator and that photograph the downward direction, and a third photographing device 5 that is arranged below the first door 60 and that photographs the downward direction.

[0023] For example, the refrigerator compartment 10 can be photographed by the first photographing devices 1 and 2, and the door pockets 63, 64, and 65 (see FIG. 5) of the first door 60 can be photographed by the second photographing device 4. Furthermore, food stored in containers in the vegetable compartment 30 can be photographed by the second photographing device 3 when the containers are moved forward, and the freezer compartment 40 can be photographed by the third photographing device 5.

[0024] The first photographing devices 1 and 2 may be a refrigerator photographing device for photographing the refrigerator compartment 10. The second photographing device 3 may be a vegetable compartment photographing device for photographing the vegetable compartment 30. The second photographing device 4 may be a door pocket photographing device for photographing the door pockets 63, 64, and 65. The second photographing device may be provided as a vegetable compartment photographing device and a door pocket photographing device separately, or may be configured as a single photographing device. Furthermore, the second photographing devices 3 and 4 may be configured as a photographing device unit 80 by being housed in a single housing as shown in FIG. 1. The third photographing device 5 may be a freezer photographing device for photographing the freezer compartment 40.

[0025] Each of the plurality of imaging devices may be configured to include an image sensor equipped with, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge-Coupled Device) sensor.

[0026] A viewing angle line F1 indicates the boundary line of the image capturing range of the first image capturing device 1. A viewing angle line F2 indicates the boundary line of the image capturing range of the first image capturing device 2. A viewing angle line F3 indicates the boundary line of the image capturing range of the second image capturing device 3. A viewing angle line F4 indicates the boundary line of the image capturing range of the second image capturing device 4. A viewing angle line F5 indicates the boundary line of the image capturing range of the third image capturing device 5.

[0027] As described above, the refrigerator 100 of the present disclosure is equipped with at least one camera so that the storage status of all items within the refrigerator can be confirmed. By enabling the user to check the storage status of all items within the refrigerator using at least one camera, the user can minimize the number of times the refrigerator 100 door is opened when creating recipes or preparing meals. This minimizes temperature fluctuations within the refrigerator and slows food deterioration, contributing to a reduction in food waste. Furthermore, reducing the number of times the refrigerator 100 door is opened and closed also leads to reduced energy consumption and reduced CO2 emissions. Furthermore, being able to instantly check the availability of ingredients shortens the overall cooking time and enables more efficient meal preparation. This improves the efficiency of food management and retrieval in the user's daily life, thereby reducing environmental impact.

[0028] The ability to check the storage status of the entire interior of the refrigerator means that even if there is no food stored in the storage compartment, the status of all storage compartments within the refrigerator can be checked. In this disclosure, the storage compartment refers to the refrigerator compartment, vegetable compartment, freezer compartment, and door pocket.

[0029] The refrigerator 100 of the present disclosure may further include a deep-freeze compartment 20 and a machine compartment 50, as shown in FIG.

[0030] Here, the reason why it would be difficult for a person skilled in the art who has read Patent Documents 1 and 2 to come up with the refrigerator 100 of the present disclosure will be explained below.

[0031] In the refrigerator of Patent Document 1, an external camera on the top surface is used to capture images of the interior of the refrigerator compartment, but it is not possible to capture images above the top shelf of the refrigerator compartment. As a result, the captured images contain blind spots. Furthermore, in the refrigerator of Patent Document 1, the captured images are supposed to be sent to the user's information terminal, such as a smartphone, so that the user can check the status of the interior of the refrigerator on the information terminal.

[0032] In the refrigerator of Patent Document 2, a mirror is attached near the exterior camera on the top surface, and a virtual image reflected in the mirror is used to expand the shooting range, but the captured image still includes blind spots. Furthermore, in the refrigerator of Patent Document 2, the captured image is sent to the user's information terminal such as a smartphone, and it is assumed that the user can check the status inside the refrigerator on the information terminal.

[0033] As described above, the prior art only considers the screen to be displayed to a user who is located far from the refrigerator. In other words, it is assumed that the captured image is sent to the user's information terminal such as a smartphone, and the user checks the status of the refrigerator interior on the information terminal, but there is no mention or suggestion of recognizing the issue of what kind of screen should be displayed to a user who is standing in front of the refrigerator, that is, within reach of the refrigerator, in order to reduce the frequency of opening the refrigerator door.

[0034] In contrast, the present disclosure was conceived based on the recognition of a new problem: what should be done to reduce the frequency with which a user opens the refrigerator door 100 while standing in front of the refrigerator 100? The present disclosure was conceived based on the discovery that a display method that displays all compartments in the refrigerator, rather than simply displaying images captured by a camera, is effective. This discovery is not mentioned or suggested in the literature describing the prior art. Rather, it is expected that displaying all compartments in the refrigerator on a smartphone, which has a limited display area, would make it difficult to distinguish between ingredients. For this reason, it is believed that a person skilled in the art who has come into contact with the prior art would be unlikely to be motivated to modify a prior art refrigerator to include at least one camera so that the storage status of the entire refrigerator can be confirmed.

[0035] For the above reasons, it is believed that it would be difficult for a person skilled in the art who had read Patent Documents 1 and 2 to arrive at the present disclosure.

[0036] FIG. 2 is a diagram showing a front view of the refrigerator 100 of the present disclosure.

[0037] As described above, the refrigerator 100 of the present disclosure includes a refrigerator compartment 10, a vegetable compartment 30 located below the refrigerator compartment 10, a freezer compartment 40 located below the vegetable compartment 30, a first door 60 that swings double doors corresponding to the refrigerator compartment 10 and the vegetable compartment 30, and a second door 70 that is a drawer and corresponds to the freezer compartment 40. This reduces the number of dividing lines when the refrigerator 100 is viewed from the front while realizing the functions of a refrigerator compartment, a vegetable compartment, and a freezer compartment. In other words, the refrigerator 100 has a simple design divided into three sections on the outside, but functionally maintains at least the functions of refrigeration, vegetable storage, and freezing, similar to conventional refrigerators. This makes it possible to realize a refrigerator that combines design and functionality.

[0038] 3 is a diagram showing the refrigerator compartment 10 being photographed by the first photographing devices 1 and 2 and the freezer compartment 40 being photographed by the third photographing device 5 in the refrigerator 100 of the present disclosure. The first photographing devices 1 and 2 are disposed within the refrigerator compartment 10 in positions that allow them to capture an overall view of the shelves 11, 12, and 13 and the storage space within the refrigerator compartment 10, allowing them to record in detail the arrangement of food items within the refrigerator compartment 10. The third photographing device 5 is disposed on the lower edge of the first door 60 in a position that allows them to capture an overall view of the storage space within the freezer compartment 40, allowing them to record in detail the arrangement of food items within the freezer compartment 40. For example, when the second door 70 is pulled forward and the upper container C3 is positioned forward, the contents of the upper container C3 can be photographed, and when the upper container C3 is positioned rearward, the contents of the lower container C4 can be photographed.

[0039] FIG. 4 is a diagram showing the vegetable compartment 30 being photographed by the second photographing device 3 in the refrigerator 100 of the present disclosure. Note that the first door 60 is omitted from FIG. 4 . The second photographing device 3 is positioned above the refrigerator 100 in a position that allows it to capture the entire storage space in the vegetable compartment 30, allowing it to record the food arrangement in the vegetable compartment 30 in detail. As shown in FIG. 4 , for example, the vegetable compartment 30 may be photographed by the second photographing device 3 when the vegetable compartment 30 is pulled out to the front of the refrigerator 100. Furthermore, as shown in FIG. 4 , for example, when the upper container C1 is positioned in the front, the contents of the upper container C1 may be photographed. Furthermore, when the upper container C1 is positioned in the rear and the lower container C2 is positioned in the front, the contents of the lower container C2 may be photographed. FIG. 5 is a diagram showing the door pockets 63, 64, and 65 of the left door 62 of the first door 60 being photographed by the second photographing device 4 when the left door 62 is open in the refrigerator 100 of the present disclosure. The second photographing device 4 is placed at a position from above the refrigerator 100 where it can capture the entire storage space of the door pockets 63, 64, 65 of the first door 60, and can record in detail the arrangement of food items in the door pockets 63, 64, 65 of the first door 60.

[0040] As shown in FIG. 5 , for example, when the left door 62 is open at a predetermined angle, the second camera 4 may capture an image of the door pockets 63, 64, and 65 of the left door 62. The same applies to capturing images of the door pockets 63, 64, and 65 of the right door 61 of the first door 60. Although the door pockets 63, 64, and 65 of the right door 61 are not shown, for example, the door pockets 63, 64, and 65 may be located at the same height as the left door 62 shown in FIG. 5 . Then, for example, when the right door 61 is open at a predetermined angle, the second camera 4 may capture an image of the door pockets 63, 64, and 65 of the right door 61. The predetermined angle may be any angle within a range that allows the storage space of the door pockets 63, 64, and 65 to be captured. For example, it may be 30 degrees, 60 degrees, 90 degrees, or 120 degrees.

[0041] FIG. 6 is a diagram showing an example of a display displayed on the display 90 in the refrigerator 100 of the present disclosure.

[0042] 6, a composite image generated based on images captured by a plurality of image capturing devices is displayed on display 90 of refrigerator 100. The composite image displays images of door pockets 63, 64, and 65 of first door 60 corresponding to image areas A1 and A3, an image of refrigerator compartment 10 corresponding to image area A2, an image of vegetable compartment 30 corresponding to image areas A4 and A5, and an image of freezer compartment 40 corresponding to image areas A6 and A7.

[0043] Specifically, the images of the door pockets 63, 64, 65 of the first door 60 include images of the door pockets 63, 64, 65 of the right door 61 and images of the door pockets 63, 64, 65 of the left door 62. In addition, in the composite image displayed on the display 90 of the refrigerator 100, to the right of the image of the refrigerator compartment 10 corresponding to image area A2 is located an image of the door pockets 63, 64, and 65 of the right door 61 of the first door 60 corresponding to image area A3, to the left of the image of the refrigerator compartment 10 corresponding to image area A2 is located an image of the door pockets 63, 64, and 65 of the left door 62 of the first door 60 corresponding to image area A1, below the image of the refrigerator compartment 10 corresponding to image area A2 is located an image of the vegetable compartment 30 corresponding to image areas A4 and A5, and below the image of the vegetable compartment 30 corresponding to image areas A4 and A5 is located an image of the freezer compartment 40 corresponding to image areas A6 and A7. More specifically, image area A4 corresponds to upper container C1 located on the upper level of vegetable compartment 30, image area A5 corresponds to lower container C2 located on the lower level of vegetable compartment 30, image area A6 corresponds to upper container C3 located on the upper level of freezer compartment 40, and image area A7 corresponds to lower container C4 located on the lower level of freezer compartment 40.

[0044] This allows the user to check the composite image displayed on the display 90 of the refrigerator 100, allowing them to see all the compartments in the refrigerator in one image. As described above, this allows for a refrigerator that can reduce food waste and CO2 emissions while also reducing cooking time. Furthermore, in the composite image, the images of the refrigerator compartment 10, the door pockets 63, 64, and 65, the vegetable compartment 30, and the freezer compartment 40 are displayed in a manner that corresponds to the physical layout of the actual refrigerator 100. Displaying the images in this manner gives the user the impression that they are actually opening the door of the refrigerator 100 and looking inside. Furthermore, the user can intuitively and quickly determine where in the actual refrigerator 100 a particular ingredient seen in the composite image is located.

[0045] Note that, in generating the composite image shown in FIG. 6 , the configuration of refrigerator 100, the number of camera devices, and the arrangement of the camera devices are not limited to the embodiment exemplified in this disclosure. The structure of refrigerator 100, the number of camera devices, and the arrangement of the camera devices may be any configuration that allows for capturing images of all compartments within refrigerator 100. The specific configuration is not important as long as it is possible to create a composite image displaying all compartments within the refrigerator based on the technical concept of this disclosure. For example, even if an additional storage compartment is provided within refrigerator 100 or the camera devices are located in different positions, the arrangement is within the technical scope of this disclosure as long as the camera devices are arranged to capture images of all compartments within the refrigerator. As such, this disclosure relates to a composite image displaying all compartments within refrigerator 100, and the applicable refrigerator 100 configuration and camera device arrangement are wide-ranging.

[0046] Fig. 7 is a diagram comparing the conventional examples with the first embodiment. Specifically, Fig. 7 shows the number of cameras installed in the refrigerators of the first to fourth conventional examples and refrigerator 100 of the first embodiment of the present disclosure, and the percentage of compartments that can be confirmed from the interior images acquired by the installed cameras (visible compartments). Each refrigerator had a total of 10 storage compartments: four refrigerator compartments, two door pocket compartments, two vegetable compartments, and two freezer compartments.

[0047] The refrigerator of Conventional Example 1 had one camera and 7.5 of the 10 compartments were visible. The refrigerator of Conventional Example 2 had one camera and 5.5 of the 10 compartments were visible. The refrigerator of Conventional Example 3 had three cameras and 5 of the 10 compartments were visible. The refrigerator of Conventional Example 4 had three cameras and 4 of the 10 compartments were visible. In contrast, refrigerator 100 of Embodiment 1 according to the present disclosure had five cameras and 10 of the 10 compartments were visible. In this way, refrigerator 100 of Embodiment 1 according to the present disclosure is equipped with five cameras, and it is possible to check the storage status of the entire interior of the refrigerator from images of the interior acquired by the five cameras.

[0048] (Embodiment 2) Fig. 8 is a diagram showing the configuration of a refrigerator 101 according to embodiment 2. Fig. 9 is a diagram showing the configuration of a conventional refrigerator.

[0049] In the conventional refrigerator configuration shown in Figure 9, the lens of the camera device 6 located on the ceiling surface 91 outside the refrigerator faces downward. Furthermore, when the camera unit 81 including the camera device 6 and the door 66 are viewed vertically from above, there is a distance (offset) between the tip of the camera unit 81 and the front of the door 66. When the door 66 is closed and viewed vertically from above, the lens does not overlap with the door 66 and the lens is exposed. For this reason, a user approaching the periphery of the refrigerator may feel uneasy that they are being photographed by the camera device 6.

[0050] In contrast, refrigerator 101 of the present disclosure shown in Fig. 8 includes a storage compartment (e.g., refrigeration compartment 10) with a door 66, and an image capturing device 6 arranged on a top surface 91 outside the refrigerator. In refrigerator 101, the lens of image capturing device 6 arranged on top surface 91 outside the refrigerator faces downward, and when door 66 is closed, the lens overlaps with door 66 when viewed vertically from above, with the lens facing an upper edge 66e of door 66. On the other hand, when door 66 is open by a predetermined angle or more, the lens does not overlap with door 66 when viewed vertically from above.

[0051] This prevents the user from seeing the lens of the camera device 6 when door 66 of refrigerator 101 is closed. In this way, the present disclosure makes it possible to realize refrigerator 101 that allows the user to use camera device 6 when necessary, while giving the user the peace of mind that the surroundings of refrigerator 101 are not being photographed by the camera device. In the example of Figure 8, refrigerator 101 includes not only door 66, but also doors 67, 68, and 69.

[0052] In the refrigerator 101 of the present disclosure, when the door 66 is closed and the photographing device unit 81 including the photographing device 6 and the door 66 are viewed vertically from above, the tip of the photographing device unit 81 overlaps with the front surface of the door 66.

[0053] According to this, when door 66 is closed, there is no structure that protrudes forward from the position of door 66. In other words, there is no offset as shown in Fig. 9. This improves the sense of unity of the entire structure, and makes it possible to realize a refrigerator with a sophisticated design.

[0054] (Other Embodiments) The display 90 of the refrigerator 100 according to the present disclosure may be configured to be able to display a composite image in response to a touch operation or air operation by the user. This allows the user to check the status of the interior of the refrigerator 100 with a simple action. For example, even when the user's hands are full while cooking, the user can check the status of the interior of the refrigerator 100 with a simple movement without opening the door.

[0055] The refrigerator 100 of the present disclosure may further include a control unit and a communication unit. Furthermore, the control unit may control the communication unit to transmit information indicating a composite image generated based on images captured by the multiple image capture devices to an information terminal. The composite image may include images of the door pockets 63, 64, and 65 of the first door 60, an image of the refrigerator compartment 10, an image of the vegetable compartment 30, and an image of the freezer compartment 40. This allows the user to check the inventory status of the refrigerator 100 via the information terminal regardless of location. Therefore, even when the user is outdoors, the user can check the inventory information of the refrigerator 100 and easily manage food ingredients.

[0056] The refrigerator 100 of the present disclosure is a refrigerator equipped with a first storage compartment, a second storage compartment, a first door 60 corresponding to the first storage compartment, a second door 70 corresponding to the second storage compartment, a plurality of photographing devices, and a display 90 that displays a composite image generated based on images photographed by at least one of the photographing devices, and the composite image may include an image photographed of the first storage compartment and an image photographed of the second storage compartment.

[0057] This allows the user to check the status of the refrigerator interior by checking the composite image displayed on the display 90 of the refrigerator 100.

[0058] The first storage compartment may be the refrigerator compartment 10 or the vegetable compartment 30. The second storage compartment may be the freezer compartment 40.

[0059] In the refrigerator 100 of the present disclosure, the first door 60 is a double door, and the second door 70 is a drawer door. The composite image may further include an image of the door pockets 63, 64, 65 of the right door 61 of the first door 60 and an image of the door pockets 63, 64, 65 of the left door 62 of the first door 60.

[0060] By checking the composite image, the status of the door pockets 63, 64, 65 of the right door 61 and the left door 62 of the first door 60 can also be confirmed.

[0061] The image generation method of the present disclosure is for generating an image showing the interior of a refrigerator 100, and generates a composite image based on images captured by multiple image capture devices. The refrigerator 100 includes a first storage compartment, a second storage compartment, a first door 60 that swings both ways and is provided corresponding to the first storage compartment, a second door 70 that is a drawer and is provided corresponding to the second storage compartment, and multiple image capture devices. The composite image may include images of door pockets 63, 64, and 65 on a right door 61 of the first door 60, images of door pockets 63, 64, and 65 on a left door 62 of the first door 60, an image of the first storage compartment, and an image of the second storage compartment.

[0062] This makes it possible to generate an image that allows the user to check the status inside the refrigerator 100. The first storage compartment may be the refrigerator compartment 10 or the vegetable compartment 30. The second storage compartment may be the freezer compartment 40.

[0063] The location where the image processing for generating a composite image based on image data obtained from multiple image capturing devices is performed is not particularly limited. This processing can be performed by any computing device, such as a computer installed in refrigerator 100, an external server, or a user's information terminal.

[0064] The program of the present disclosure may be a program for causing a computer to execute the image generation method described above.

[0065] (Note) The present disclosure will be described below.

[0066] [Item 1] A refrigerator equipped with at least one imaging device so that the storage status of the entire interior can be confirmed.

[0067] [Item 2] A refrigerator comprising: a refrigerator compartment; a vegetable compartment located below the refrigerator compartment; a freezer compartment located below the vegetable compartment; a first door that swings both ways and is provided corresponding to the refrigerator compartment and the vegetable compartment; and a second door that is a drawer and is provided corresponding to the freezer compartment, wherein the vegetable compartment includes a container for storing food, and the container is configured to be movable back and forth.

[0068] [Item 3] The refrigerator according to Item 2, further comprising a plurality of camera devices, the plurality of camera devices including: a first camera device disposed inside the first door for photographing the interior of the refrigerator; a second camera device disposed on the ceiling outside the refrigerator for photographing the downward direction; and a third camera device disposed below the first door for photographing the downward direction.

[0069] [Item 4] The refrigerator according to Item 3, further comprising: a display that displays a composite image generated based on images captured by the plurality of image capturing devices, wherein the composite image includes an image of a door pocket of the first door, an image of the refrigerator compartment, an image of the vegetable compartment, and an image of the freezer compartment.

[0070] [Item 5] The refrigerator according to Item 4, wherein the image of the door pocket of the first door includes an image of the door pocket of the right door and an image of the door pocket of the left door, and in the composite image, the image of the door pocket of the right door is located to the right of the image of the refrigerator compartment, the image of the door pocket of the left door is located to the left of the image of the refrigerator compartment, the image of the vegetable compartment is located below the image of the refrigerator compartment, and the image of the freezer compartment is located below the image of the vegetable compartment.

[0071] [Item 6] The refrigerator according to Item 4, wherein the display is configured to be able to display the composite image in response to a touch operation or an air operation by a user.

[0072] [Item 7] The refrigerator according to Item 3 further includes a control unit and a communication unit, wherein the control unit controls the communication unit to transmit, to an information terminal, information indicating a composite image generated based on images captured by the plurality of image capturing devices, and the composite image includes an image of a door pocket of the first door, an image of the refrigerator compartment, an image of the vegetable compartment, and an image of the freezer compartment.

[0073] [Item 8] A refrigerator comprising: a first storage compartment; a second storage compartment; a first door provided corresponding to the first storage compartment; a second door provided corresponding to the second storage compartment; a plurality of image capturing devices; and a display that displays a composite image generated based on images captured by the plurality of image capturing devices, wherein the composite image includes an image captured of the first storage compartment and an image captured of the second storage compartment.

[0074] [Item 9] The refrigerator according to Item 8, wherein the first door is a double door, the second door is a drawer door, and the composite image further includes an image of a door pocket on the right side of the first door and an image of a door pocket on the left side of the first door.

[0075] [Item 10] A refrigerator comprising: a storage compartment with a door; and an imaging device arranged on the ceiling outside the storage compartment, wherein a lens of the imaging device faces downward; when the door is closed, the lens and the door overlap when viewed from above vertically, and the lens faces the upper edge of the door; and when the door is open by a predetermined angle or more, the lens and the door do not overlap when viewed from above vertically.

[0076] [Item 11] The refrigerator according to Item 10, wherein, when the door is closed and the photographing device unit including the photographing device and the door are viewed vertically from above, a tip of the photographing device unit overlaps with a front surface of the door.

[0077] [Item 12] An image generation method for generating an image showing the interior of a refrigerator, the image generation method generating a composite image based on images captured by a plurality of image capture devices, the refrigerator comprising: a first storage compartment; a second storage compartment; a first door that swings both ways and is provided corresponding to the first storage compartment; a second door that is a drawer and is provided corresponding to the second storage compartment; and the plurality of image capture devices, the composite image including an image of a door pocket on the right door of the first door, an image of a door pocket on the left door of the first door, an image of the first storage compartment, and an image of the second storage compartment.

[0078] [Item 13] A program for causing a computer to execute the image generating method according to Item 12.

[0079] Each of the above items may be subordinate to any other item.

[0080] (Other Aspects) In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0081] The following cases are also included in this disclosure:

[0082] (1) The at least one device or system is specifically a computer system comprising a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The at least one device or system achieves its function when the microprocessor operates in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands for a computer to achieve a predetermined function.

[0083] (2) Some or all of the components constituting at least one of the above devices or systems may be configured as a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured to include a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating in accordance with the computer program.

[0084] (3) Some or all of the components constituting at least one of the above devices or systems may be configured as an IC card or a standalone module that can be attached to or detached from the device or system. The IC card or module is a computer system configured with a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates in accordance with a computer program. This IC card or module may be tamper-resistant.

[0085] (4) The present disclosure may be embodied as the methods described above, a computer program that implements these methods on a computer, or a digital signal that includes the computer program.

[0086] The present disclosure may also be a computer program or a digital signal recorded on a computer-readable recording medium, such as a flexible disk, a hard disk, a CD (Compact Disc)-ROM, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, etc. Alternatively, the present disclosure may be a digital signal recorded on such a recording medium.

[0087] The present disclosure may also be applied to transmitting a computer program or digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, or the like.

[0088] Furthermore, the program or digital signal may be recorded on a recording medium and transferred, or the program or digital signal may be transferred via a network or the like, so that the program or digital signal may be implemented by another independent computer system.

[0089] According to an example of the refrigerator of the present disclosure, it can be applied to a refrigerator that can reduce cooking time.

[0090] 1, 2 First imaging device (photographing device for refrigerator compartment) 3 Second imaging device (photographing device for vegetable compartment) 4 Second imaging device (photographing device for door pocket) 5 Third imaging device (photographing device for freezer compartment) 10 Refrigerator compartment 11, 12, 13 Shelf 20 Quick freezing compartment 30 Vegetable compartment 40 Freezer compartment 50 Machine compartment 60 First door 61 Right door 62 Left door 63, 64, 65 Door pocket 66, 67, 68, 69 Door 70 Second door 80, 81 Imaging device unit 90 Display 100, 101 Refrigerator F1, F2, F3, F4, F5 Viewing angle C1, C3 Upper container C2, C4 Lower container A1, A2, A3, A4, A5, A6, A7 Image area

Claims

1. A refrigerator equipped with at least one imaging device that allows the storage status of all items within the refrigerator to be confirmed.

2. A refrigerator comprising: a refrigerator compartment; a vegetable compartment located below the refrigerator compartment; a freezer compartment located below the vegetable compartment; a first door that opens both ways and is provided corresponding to the refrigerator compartment and the vegetable compartment; and a second door that is a pull-out door and is provided corresponding to the freezer compartment, wherein the vegetable compartment includes a container for storing food, and the container is configured to be movable back and forth.

3. The refrigerator according to claim 2, further comprising a plurality of photographing devices, the plurality of photographing devices including: a first photographing device arranged inside the first door for photographing the interior of the refrigerator; a second photographing device arranged on the top surface outside the refrigerator for photographing the downward direction; and a third photographing device arranged below the first door for photographing the downward direction.

4. The refrigerator according to claim 3, further comprising a display for displaying a composite image generated based on images captured by the plurality of photographing devices, the composite image including an image of the door pocket of the first door, an image of the refrigerator compartment, an image of the vegetable compartment, and an image of the freezer compartment.

5. The refrigerator of claim 4, wherein the image of the door pocket of the first door includes an image of the door pocket of the right door and an image of the door pocket of the left door, and in the composite image, the image of the door pocket of the right door is located to the right of the image of the refrigerator compartment, the image of the door pocket of the left door is located to the left of the image of the refrigerator compartment, the image of the vegetable compartment is located below the image of the refrigerator compartment, and the image of the freezer compartment is located below the image of the vegetable compartment.

6. The refrigerator according to claim 4, wherein the display is configured to be able to display the composite image in response to a touch operation or an air operation by a user.

7. The refrigerator according to claim 3, further comprising a control unit and a communication unit, wherein the control unit controls the communication unit to transmit to an information terminal information indicating a composite image generated based on images captured by the plurality of photographing devices, and the composite image includes an image of the door pocket of the first door, an image of the refrigerator compartment, an image of the vegetable compartment, and an image of the freezer compartment.

8. A refrigerator comprising: a first storage compartment; a second storage compartment; a first door provided corresponding to the first storage compartment; a second door provided corresponding to the second storage compartment; a plurality of photographing devices; and a display that displays a composite image generated based on images photographed by the plurality of photographing devices, wherein the composite image includes an image photographed of the first storage compartment and an image photographed of the second storage compartment.

9. The refrigerator according to claim 8, wherein the first door is a double door, the second door is a drawer door, and the composite image further includes an image of a door pocket on the right side of the first door and an image of a door pocket on the left side of the first door.

10. A refrigerator comprising: a storage compartment with a door; and a camera arranged on the ceiling outside the compartment, wherein the lens of the camera faces downward; when the door is closed, the lens and the door overlap when viewed from above vertically, and the lens faces the upper edge of the door; and when the door is open by a predetermined angle or more, the lens and the door do not overlap when viewed from above vertically.

11. The refrigerator according to claim 10, wherein when the door is closed and the photographing device unit including the photographing device and the door are viewed vertically from above, the tip of the photographing device unit overlaps with the front surface of the door.

12. An image generation method for generating an image showing the interior of a refrigerator, comprising: generating a composite image based on images captured by a plurality of photographing devices; the refrigerator comprising: a first storage compartment; a second storage compartment; a first double-hinged door corresponding to the first storage compartment; a second drawer-type door corresponding to the second storage compartment; and the plurality of photographing devices; and the composite image including an image of a door pocket on the right door of the first door, an image of a door pocket on the left door of the first door, an image of the first storage compartment, and an image of the second storage compartment.

13. A program for causing a computer to execute the image generating method according to claim 12.

Images