Image pickup system, image pickup device, and refrigerator

Abstract

From the images captured by the camera, a captured image in which the inside condition of the refrigerator is appropriately captured is selected. The imaging system (7) includes: a narrow-angle camera (302) that captures an image of a refrigerator (2) including a freezing chamber (24) having a drawer (24A) having an opening formed in an upper surface thereof, the narrow-angle camera (302) being disposed on a top surface (20A) of the refrigerator (2); a subject (PS2) that is disposed at the upper end of the drawer (24A) and that changes the imaging state of the narrow-angle camera (302) in accordance with the amount of pull-out of the drawer (24A); a state storage unit (364) that stores a second imaging state (CD2), which is an imaging state of the object (PS2) to be imaged by the narrow-angle camera (302) when the drawer (24A) is pulled out by a predetermined amount; and a generation unit (356) that selects a second captured image (P2), which is a captured image when the drawer (24A) is in a pulled-out state, from the captured image (PB2) captured by the narrow-angle camera (302) on the basis of the second imaging state (CD2).

Description

Technical Field

[0001] The invention relates to a camera system, a camera device and a cold storage. Background Art

[0002] Patent Document 1 discloses a technology in which a camera is installed on the upper part of a refrigerator, an image is taken when an object passing through an opening of the refrigerator is detected, and food stored in the refrigerator is managed based on the imaged result.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-070476 Summary of the invention

[0006] Problems to be solved by the invention

[0007] The present invention provides an imaging system, an imaging device, and a cold storage for selecting an image that appropriately captures the internal conditions of a cold storage from images captured by a camera.

[0008] Technical solutions to solve problems

[0009] This specification includes all the contents of Japanese Patent Application No. 2022-165224 filed on October 14, 2022.

[0010] The present invention provides a camera system, comprising: a camera, which is arranged on the upper surface of the cold storage and is used to shoot a storage room including a drawer with an opening formed on the upper surface; a subject, which is arranged at the upper end of the drawer and the shooting state of which is changed by the camera according to the amount of the drawer being pulled out; a state storage unit, which stores the shooting state of the subject shot by the camera when the drawer is pulled out by a specified amount; and a selection unit, which selects, from the image shot by the camera, a shot image when the drawer is in a pulled-out state based on the shooting state of the subject shot by the camera stored in the state storage unit.

[0011] In addition, the present invention provides a camera device, comprising: a camera, which is configured on the upper surface of the cold storage and is configured to photograph a cold storage including a storage room having a drawer with an opening formed on the upper surface; a state storage unit, which stores a photographing state of a subject photographed by the camera when the drawer is pulled out a specified amount; and a selection unit, which selects a photographing image when the drawer is in a pulled-out state from the image photographed by the camera based on the photographing state of the subject photographed by the camera stored in the state storage unit, wherein the subject is configured at the upper end portion of the drawer, and the photographing state photographed by the camera changes according to the pulling-out amount of the drawer.

[0012] In addition, the present invention provides a cold storage, comprising: a camera, which is arranged on the upper surface of the cold storage for photographing a storage room including a drawer having an opening formed on the upper surface; a subject, which is arranged at the upper end of the drawer, and the photographing state photographed by the camera changes according to the amount of the drawer being pulled out; a state storage unit, which stores the photographing state of the subject photographed by the camera when the drawer is pulled out by a specified amount; and a selection unit, which selects a photographed image when the drawer is in a pulled-out state from the image photographed by the camera based on the photographing state of the subject photographed by the camera stored in the state storage unit.

[0013] Effects of the Invention

[0014] The camera system, camera device, and refrigerator of the present invention can select a captured image of a drawer being pulled out from an image captured by the camera based on the captured state of the subject captured by the camera stored in the state storage unit. Therefore, according to the present invention, an effect is achieved in which a captured image that properly captures the storage state of the drawer of the storage room can be selected from the image captured by the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a diagram showing the structure of the article management system in the first embodiment.

[0016] Figure 2 This is a perspective view of the refrigerator in Embodiment 1.

[0017] Figure 3 This is a side view of the imaging device in Embodiment 1 as viewed from the right.

[0018] Figure 4 This is a perspective view of the restriction member abutting against the front end edge of the main housing in Embodiment 1 as viewed from the left front.

[0019] Figure 5 This is a front view of the refrigerator and the imaging device in Embodiment 1 as seen from the front.

[0020] Figure 6 This is a side view of the refrigerator and the imaging device in Embodiment 1 as seen from the right.

[0021] Figure 7 This is a plan view of the imaging device installed in the refrigerator in Embodiment 1, viewed from above.

[0022] Figure 8 This is a block diagram showing the structure of the imaging device in Embodiment 1.

[0023] Fig. 9 This is a diagram showing the configuration of a first example of a subject in the first embodiment.

[0024] Fig.10 This is a diagram showing the configuration of a second example of the subject in the first embodiment.

[0025] Fig.11 This is a diagram showing the configuration of a second example of the subject in the first embodiment.

[0026] Fig.12 This is a diagram showing a first example of a subject image of a photographed object in the first embodiment.

[0027] Fig.13 This is a diagram showing a second example of a subject image of a photographed object in the first embodiment.

[0028] Fig.14 This is a block diagram showing the configuration of a terminal device and an item management server in the first embodiment.

[0029] Fig.15 This is a flowchart showing the operation of the imaging device in the first embodiment.

[0030] Fig.16 This is a flowchart showing the operations of the imaging device and the item management server in the first embodiment.

[0031] Fig.17 This is a flowchart showing the operations of the terminal device and the item management server in Implementation Example 1.

[0032] Fig.18 This is a diagram showing the structure of the article management system in Implementation Example 2.

[0033] Fig.19 This is a block diagram showing the structure of a cold storage in the second embodiment.

[0034] Fig. 20 This is a flowchart showing the operation of the imaging device. DETAILED DESCRIPTION

[0035] (Foundation, etc. that forms the basis of the present invention)

[0036] When the inventors finally conceived of the present invention, there was a technology for performing imaging when an object passing through an opening of a refrigerator was detected, and managing food stored in the refrigerator based on the imaging result.

[0037] The inventors have discovered a problem of selecting an image that appropriately captures the internal conditions of a refrigerator, and ultimately formed the subject matter of the present invention in order to solve the problem.

[0038] Therefore, the present invention provides an imaging system, an imaging device, and a refrigerator for selecting an image that appropriately captures the internal conditions of a refrigerator from images captured by a camera.

[0039] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. However, sometimes, more detailed descriptions than necessary will be omitted. For example, sometimes, detailed descriptions of already well-known matters or repeated descriptions of substantially the same structures will be omitted.

[0040] In addition, the drawings and the following description are provided for those skilled in the art to fully understand the present invention, and are not intended to limit the subject matter described in the claims.

[0041] (Implementation Method 1)

[0042] [1-1. Structure]

[0043] [1-1-1. Structure of the item management system]

[0044] Figure 1 This is a diagram showing the structure of the article management system 1 in the first embodiment.

[0045] The article management system 1 is a system for managing articles stored in the refrigerator 2. As an example of article management, the article management system 1 of the present embodiment manages images showing articles stored in the refrigerator 2.

[0046] In the present embodiment, the articles stored in the refrigerator 2 include food, containers for storing food, etc. The articles may be marked with a predetermined mark.

[0047] The article management system 1 includes a cold storage 2 .

[0048] The cold storage 2 includes a main body 20 with an opening on the front surface. A refrigerating chamber 21, an ice making chamber 22, a fresh freezing chamber 23, a freezing chamber 24, and a vegetable chamber 25 are formed on the main body 20. These storage chambers are cooled by cold air. A revolving door 21A is provided at the opening 201 on the front surface of the refrigerating chamber 21. The door 21A includes a revolving right door 21B and a revolving left door 21C. Drawers 22A, 23A, 24A, and 25A capable of storing items are provided in the ice making chamber 22, the fresh freezing chamber 23, the freezing chamber 24, and the vegetable chamber 25, respectively. The upper surface of each of the drawers 22A to 25A is open in the setting state. Hereinafter, when the drawers 22A, 23A, 24A, and 25A are collectively referred to without distinguishing them, the reference numeral "26A" is marked and recorded as "drawer 26A". In addition, when the ice making chamber 22 , the fresh freezing chamber 23 , the freezing chamber 24 , and the vegetable chamber 25 are collectively referred to without distinguishing each other, the reference numeral “ 26 ” is given and described as the storage chamber 26 .

[0049] The ice making chamber 22, the fresh freezing chamber 23, the freezing chamber 24, and the vegetable chamber 25 respectively correspond to an example of the "storage chamber" of the present invention.

[0050] In Embodiment 1, a case where the “storage compartment” of the present invention is freezer compartment 24 is described.

[0051] Article management system 1 includes camera 3. Camera 3 is provided on top surface 20A of main body 20 of refrigerator 2. Top surface 20A of main body 20 is made of metal. Camera 3 captures image of refrigerator 2.

[0052] The top surface 20A corresponds to an example of the “upper surface” of the present invention.

[0053] The article management system 1 includes a terminal device 4. The terminal device 4 is a portable computer such as a smartphone. The terminal device 4 has an application program related to the management of articles stored in the refrigerator 2 installed therein, and communicates with the article management server 5 through the function of the application program.

[0054] In the following description, this application is described as an “article management application” and is denoted by the reference numeral “411”.

[0055] exist Figure 1 In FIG. 1 , a user P at home is indicated by a solid line, and a user P outside the home H is indicated by a dotted line. When the terminal device 4 is used by the user P at home, it communicates with the item management server 5 via the communication device 6 or not via the communication device 6. In addition, when the terminal device 4 is used by the user P outside the home H and a communication connection cannot be established with the communication device 6, the terminal device 4 communicates with the item management server 5 not via the communication device 6.

[0056] The communication device 6 is connected to a network NW formed by a public line network or the like, and communicates with the article management server 5 via the network NW. The communication device 6 functions as an interface device for connecting the refrigerator 2, the imaging device 3, and the terminal device 4 to the network NW. The communication device 6 constructs a local area network in the home H.

[0057] The article management system 1 includes an article management server 5. The article management server 5 is a device for managing articles stored in the cold storage 2. As an article management, the article management server 5 stores images of the articles. The article management server 5 is connected to the network NW. In addition, in each figure, the article management server 5 is represented by a single block, but this does not necessarily mean that the article management server 5 is composed of a single device.

[0058] [1-1-2. Structure of cold storage]

[0059] Figure 2 It is a perspective view of the refrigerator 2 in the installed state.

[0060] exist Figure 2 The figure shows an X-axis, a Y-axis, and a Z-axis. The X-axis, the Y-axis, and the Z-axis are orthogonal to each other. The Z-axis represents the up-down direction, which is equivalent to the height direction of the cold storage 2. The X-axis and the Y-axis are parallel to the horizontal direction. The X-axis represents the left-right direction. The left-right direction is equivalent to the width direction of the cold storage 2. The Y-axis represents the front-back direction. The positive direction of the X-axis represents the right direction. The positive direction of the Y-axis represents the front direction. The positive direction of the Z-axis represents the upward direction.

[0061] Refrigerator 2 includes a main body 20, a door 21A, and a drawer 26A.

[0062] The main box body 20 is a box-shaped component that forms the main body of the cold storage 2. The main box body 20 includes: an outer box 202 that is open on the front surface; an inner box 203 that is housed inside the outer box 202 and is open on the front surface; and a heat-insulating material that fills the space between the outer box 202 and the inner box 203. The outer box 202 is formed of, for example, a metal such as iron or a hard resin such as ABS. The upper surface of the outer box 202, that is, the top surface 20A of the main box 20, is formed of metal. The inner box 203 is formed of, for example, a hard resin such as ABS. The heat-insulating material is, for example, hard polyurethane foam. In addition, the heat-insulating material is not limited to this, as long as it is a material that has heat-insulating properties and can be filled in the space between the outer box 202 and the inner box 203.

[0063] The inner space of the inner box 203 is divided into a plurality of spaces by a partition member. In this embodiment, the inner space of the inner box 203 is divided into five spaces, which function as a refrigerating chamber 21, an ice making chamber 22, a fresh freezing chamber 23, a freezing chamber 24, and a vegetable chamber 25, respectively.

[0064] The right door 21B is rotatably supported by a right door hinge member 204. The right door hinge member 204 is a hinge member provided on the top surface 20A of the main housing 20. The left door 21C is rotatably supported by a left door hinge member 205. The left door hinge member 205 is a hinge member provided on the top surface 20A of the main housing 20.

[0065] Second wireless communication module 206 is provided on right door hinge member 204. Second wireless communication module 206 is a module that receives control data for controlling refrigerator 2 from an external device or transmits predetermined data to the external device.

[0066] [1-1-3. Configuration of the imaging device]

[0067] Reference Figure 3-Figure 7 The imaging device 3 will be described.

[0068] exist Figure 3-Figure 7 The diagram and Figure 2 The X-axis, Y-axis, and Z-axis shown are the same as the X-axis, Y-axis, and Z-axis. The X-axis direction corresponds to the width direction of the imaging device 3 and the left-right direction of the imaging device 3. The Y-axis direction corresponds to the front-back direction of the imaging device 3. The Z-axis direction corresponds to the up-down direction of the imaging device 3.

[0069] Figure 3 It is a side view of the imaging device 3 attached to the refrigerator 2 as seen from the right. Figure 4 This is a perspective view of the restriction member 34 in contact with the front end edge 20B of the main housing 20 as viewed from the left front.

[0070] like Figure 3 and Figure 4 As shown in FIG. 1 , the camera device 3 includes a main body 30 disposed on the top surface 20A of the main housing 20 , and a photographing member 31 extending forward from the main body 30 . The photographing member 31 is disposed on the upper front portion of the main body 30 .

[0071] The main body 30 includes a power button 32 on the front surface for turning on and off the power of the imaging device 3. The power button 32 is provided with a first LED (Light emitting diode) 33 for notifying predetermined information.

[0072] The main body 30 includes a limiting component 34. The limiting component 34 limits the position of the shooting component 31 in the front-to-back direction of the cold storage 2 by abutting against the front end edge 20B of the main body 20. The limiting component 34 is a plate-shaped component extending downward from the bottom surface 30A of the main body 30. The limiting component 34 extends along the width direction of the main body 30 (the width direction of the camera device 3) on the front surface of the main body 30, and has a mark 35 at the center of the width direction of the main body 30. The mark 35 indicates the center in the width direction of the camera device 3. The length of the limiting component 34 extending from the bottom surface 30A is formed to be shorter than the length from the top surface 20A of the main body 20 to the door gasket provided on the upper inner surface of the door 21A and substantially parallel to the top surface 20A.

[0073] Figure 5 This is a front view of refrigerator 2 and imaging device 3 attached to refrigerator 2 as seen from the front. Figure 5 Door 21A of refrigerator 2 is shown in an open state. Figure 6 It is a side view which looked at the refrigerator 2 and the imaging device 3 attached to the refrigerator 2 from the right. Figure 6 Drawer 25A of refrigerator 2 is shown in an open state, that is, in a pulled-out state. Figure 7 It is a top view of imaging device 3 attached to refrigerator 2 as seen from above.

[0074] The camera device 3 includes a wide-angle camera 301 and a narrow-angle camera 302. The wide-angle camera 301 and the narrow-angle camera 302 are provided at the front end portion of the bottom surface 31A of the imaging member 31. When the camera device 3 is installed in the cold storage 2 and the door 21A is closed, the wide-angle camera 301 and the narrow-angle camera 302 are located in front of the door 21A.

[0075] Wide-angle camera 301 is a camera having a wider angle than narrow-angle camera 302. Wide-angle camera 301 captures refrigerating room 21 from the upper front of refrigerator 2 in a state where imaging device 3 is attached to refrigerator 2.

[0076] For example, when viewed from the front of the cold storage 2, the shooting range of the wide-angle camera 301 includes Figure 5 The range A1 is a range including the door shelves 21E of the right door 21B and the left door 21C and the opening 201 of the refrigerator 21 when the right door 21B and the left door 21C are opened to the maximum degree. In addition, the shooting range of the wide-angle camera 301 includes Figure 6 Range A3 is a range from the front end of the right door 21B and the left door 21C in the open state to the front end of the shelf 21F in the front-rear direction, and includes the opening 201 of the refrigerating chamber 21 in the up-down direction.

[0077] Narrow-angle camera 302 is a camera having a narrower angle than wide-angle camera 301. Narrow-angle camera 302 captures drawer 26A from the upper front of refrigerator 2 in a state where imaging device 3 is attached to refrigerator 2.

[0078] For example, when viewed from the front of the cold storage 2, the shooting range of the narrow-angle camera 302 includes Figure 5 The range A2 shown. When viewed from the front of the refrigerator 2, the range A2 includes the drawer 26A. In addition, for example, when viewed from the right of the refrigerator 2, the shooting range of the narrow-angle camera 302 includes Figure 6 The range A4 is a range including the drawer 26A in the front-rear direction that is pulled out first. In addition, the imaging range of the narrow-angle camera 302 may also include a range including the drawers provided in the refrigerator compartment 21.

[0079] The narrow-angle camera 302 corresponds to an example of the “camera” of the present invention.

[0080] like Figure 5-Figure 7 As shown, the narrow-angle camera 302 constitutes a part of the imaging system 7 in Embodiment 1. The imaging system 7 in Embodiment 1 includes the narrow-angle camera 302, the subject PS, the state storage unit 364, and the generating unit 356. In other words, the imaging system 7 includes the cold storage 2 including the subject PS, and the imaging device 3 including the narrow-angle camera 302, the state storage unit 364, and the generating unit 356.

[0081] Reference Figure 8 The subject PS, the state storage unit 364 , and the generation unit 356 are described.

[0082] The imaging device 3 includes a first optical sensor 303 and a second optical sensor 304 . The first optical sensor 303 and the second optical sensor 304 are provided on the main body 30 .

[0083] The first optical sensor 303 is a sensor for detecting the opening and closing of the right door 21B. The first optical sensor 303 is a reflective optical sensor. The first optical sensor 303 projects light to a predetermined portion of the right door 21B when the imaging device 3 is mounted on the refrigerator 2. Then, the first optical sensor 303 outputs a signal indicating the amount of light received.

[0084] The second optical sensor 304 is a sensor for detecting the opening and closing of the left door 21C. The second optical sensor 304 is a reflective optical sensor. The second optical sensor 304 projects light to a predetermined portion of the left door 21C when the imaging device 3 is mounted on the refrigerator 2. Then, the second optical sensor 304 outputs a detection value indicating the amount of received light.

[0085] The imaging device 3 includes a thermopile 305. The thermopile 305 is a sensor that detects the opening and closing of the drawer 26A. The thermopile 305 is provided at the front end of the bottom surface 31A of the imaging member 31. The thermopile 305 outputs a detection value indicating the detected temperature.

[0086] The imaging device 3 includes a human sensor 306. The human sensor 306 may be an infrared sensor or an ultrasonic sensor. The human sensor 306 outputs a detection value. The human sensor 306 is provided at the front end of the bottom surface 31A of the imaging member 31.

[0087] The imaging device 3 includes an illuminance sensor 307. The illuminance sensor 307 is a sensor for adjusting the exposure of the wide-angle camera 301 and the narrow-angle camera 302. The illuminance sensor 307 outputs a detection value. The illuminance sensor 307 is provided at the front end of the imaging unit 31.

[0088] The imaging device 3 includes a second LED 308 and a third LED 309. The second LED 308 is provided at the right front end of the bottom surface 31A of the imaging member 31. The third LED 309 is provided at the left front end of the bottom surface 31A of the imaging member 31. The second LED 308 and the third LED 309 illuminate the refrigerator 2 on which the imaging device 3 is mounted from above.

[0089] like Figure 7 As shown, the imaging device 3 includes a first control substrate 311 and a second control substrate 312 .

[0090] The first control substrate 311 is a substrate for controlling various sensors included in the camera device 3 and the power supply of the camera device 3. The first control substrate 311 is provided inside the camera device 3. The power button 32, the first LED 33, the first light sensor 303, the second light sensor 304, the thermocouple 305, the human sensor 306, and the illuminance sensor 307 are connected to the first control substrate 311. And the first control substrate 311 controls these devices. The first control substrate 311 is provided behind the second control substrate 312 and the first wireless communication module 313.

[0091] The second control substrate 312 is a substrate that controls the wide-angle camera 301 and the narrow-angle camera 302, and also controls the first wireless communication module 313. The second control substrate 312 is provided inside the camera device 3. The wide-angle camera 301, the narrow-angle camera 302, and the first wireless communication module 313 are connected to the second control substrate 312. And the second control substrate 312 controls these devices. The second control substrate 312 is provided side by side with the first control substrate 311 in front of the first control substrate 311.

[0092] The first wireless communication module 313 is a device including hardware such as an antenna or a wireless communication circuit. The first wireless communication module 313 is provided inside the imaging component 31, to the left of the second control substrate 312. The first wireless communication module 313 is provided in the imaging device 3 so as to be located in front of the opening 201 of the main body 20 when the imaging device 3 is installed in the cold storage 2. More specifically, the first wireless communication module 313 is provided in front of the limiting component 34 in the imaging device 3.

[0093] like Figure 7 As shown, in a state where the imaging device 3 is mounted in the refrigerator 2, a metal plate 314 is provided between the first wireless communication module 313 and the second wireless communication module 206 in the left-right direction.

[0094] Figure 8 It is a block diagram showing the structure of the imaging device 3.

[0095] like Figure 8 As shown, the first control substrate 311 includes a first processor 320 such as a CPU (Central Processing Unit), a first memory 330, a first communication module 340, and an interface circuit to which other devices and sensors are connected.

[0096] The first memory 330 is a storage device that stores programs and data executed by the first processor 320. The first memory 330 is composed of a non-volatile storage device such as a ROM (Read Only Memory). The first memory 330 may also include a volatile storage device such as a RAM (Random Access Memory) that constitutes a working area of ​​the first processor 320. The first memory 330 stores data processed by the first processor 320 and a control program 331 executed by the first processor 320.

[0097] The first communication module 340 includes hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the second control substrate 312 under the control of the first processor 320 .

[0098] The first processor 320 reads and executes the control program 331 , thereby turning the power of the imaging device 3 on and off in accordance with the operation of the power button 32 .

[0099] Furthermore, the first processor 320 reads out and executes the control program 331 to turn on and off the first LED 33 , the second LED 308 , and the third LED 309 .

[0100] Furthermore, the first processor 320 reads out and executes the control program 331 , thereby transmitting the detection values ​​output by each sensor connected to the first control board 311 to the second control board 312 via the first communication module 340 .

[0101] The second control substrate 312 includes a second processor 350 such as a CPU, a second memory 360, a second communication module 370, and an interface circuit for connecting to other devices or sensors.

[0102] The second memory 360 is a storage device that stores programs and data executed by the second processor 350. The second memory 360 is composed of a non-volatile storage device such as a ROM. In addition, the second memory 360 may also include a volatile storage device such as a RAM that constitutes a working area of ​​the second processor 350. The second memory 360 stores data processed by the second processor 350, a control program 361 executed by the second processor 350, and an imaging device ID (Identification) 362.

[0103] The imaging device ID 362 is information for uniquely identifying the imaging device 3 .

[0104] The second communication module 370 includes hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the first control substrate 311 under the control of the second processor 350 .

[0105] The second processor 350 reads out and executes the control program 361 stored in the second memory 360, thereby functioning as the first communication control unit 351, the second communication control unit 352, the opening / closing determination unit 353, the imaging control unit 354, the recording control unit 355, and the generating unit 356. In addition, the second processor 350 reads out and executes the control program 361 stored in the second memory 360, thereby causing the second memory 360 to function as a state storage unit 364.

[0106] The first communication control unit 351 communicates with the article management server 5 via the first wireless communication module 313 .

[0107] The second communication control unit 352 communicates with the first control board 311 via the second communication module 370. The second communication control unit 352 receives detection values ​​of various sensors connected to the first control board 311 from the first control board 311.

[0108] The opening / closing determination unit 353 determines whether the right door 21B is in the open state or the closed state based on the detection value of the first light sensor 303 received by the second communication control unit 352. When the second communication control unit 352 receives a detection value different from the detection value corresponding to the closed state, the opening / closing determination unit 353 determines that the right door 21B is in the open state. On the other hand, when the second communication control unit 352 receives a detection value corresponding to the closed state, the opening / closing determination unit 353 determines that the right door 21B is in the closed state.

[0109] Similar to the determination regarding the right door 21B, the opening / closing determination unit 353 determines whether the left door 21C is in the open state or the closed state based on the detection value of the second optical sensor 304 received by the second communication control unit 352 .

[0110] The opening and closing judgment unit 353 judges whether the drawer 26A is in an open state or a closed state based on the detection value of the thermopile 305 received by the second communication control unit 352. When the second communication control unit 352 receives the detection value of the thermopile 305 indicating a temperature below the specified temperature, the opening and closing judgment unit 353 judges that the drawer 26A is in an open state. On the other hand, when the second communication control unit 352 receives the detection value of the thermopile 305 indicating a temperature exceeding the specified temperature, the opening and closing judgment unit 353 judges that the drawer 26A is in a closed state.

[0111] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state, the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start imaging. In addition, when the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start imaging, the second communication control unit 352 sends lighting request information to the first control substrate 311. The lighting request information is information requesting the lighting of the second LED 308 and the third LED 309. When the first processor 320 receives the lighting request information, it lights the second LED 308 and the third LED 309, and notifies the user P that imaging has started.

[0112] When the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are in the closed state, the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging. In addition, when the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging, the second communication control unit 352 sends an extinguishing request information to the first control substrate 311. The extinguishing request information is information requesting the extinguishing of the second LED 308 and the third LED 309. When the first processor 320 receives the extinguishing request information, it turns off the second LED 308 and the third LED 309, and notifies the user P that imaging is ended.

[0113] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state, the recording control unit 355 starts recording the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302. That is, the recording control unit 355 starts recording the image capturing data 363 in the second memory 360. The image capturing data 363 includes the first image capturing data DV1 representing the image capturing results of the wide-angle camera 301 and the second image capturing data DV2 representing the image capturing results of the narrow-angle camera 302.

[0114] When the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are in the closed state, the recording control unit 355 ends the recording of the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302 .

[0115] The state storage unit 364 pre-stores a first shooting state CD1 and a second shooting state CD2. The first shooting state CD1 indicates a shooting state of the subject PS1 captured by the wide-angle camera 301 at a specified opening and closing angle of the door 21A. When the opening and closing angle is zero degrees, it indicates that the door 21A is in a closed state. The larger the opening and closing angle, the more the door 21A is opened. The specified opening and closing angle is, for example, within the range of 60 to 100 degrees. The specified opening and closing angle is, for example, 90 degrees. In the following description, the state in which the door 21A is at a specified opening and closing angle (for example, 60 to 100 degrees) is described as the "first reference state".

[0116] The first shooting state CD1 includes, for example, at least one of the position, size, and shape of the image corresponding to the subject PS1 in the shooting image PA1 shot by the wide-angle camera 301 in the first reference state. In the first embodiment, the first shooting state CD1 includes, for example, the size of the image corresponding to the subject PS1 in the shooting image PA1 shot by the wide-angle camera 301 in the first reference state.

[0117] Reference Fig. 9 The subject PS1 will be further described.

[0118] The second shooting state CD2 represents the shooting state of the subject PS2 shot by the narrow-angle camera 302 when the drawer 26A is pulled out by a specified amount. When the pulled-out amount is zero, it indicates that the drawer 26A is in a closed state. The larger the specified pull-out amount is, the more the drawer 26A is opened. A pull-out amount of 100% indicates that the drawer 26A is in a fully open state. The specified pull-out amount is, for example, within the range of 70% to 100%. The specified pull-out amount is, for example, 80%. In the following description, the state in which the drawer 26A is pulled out by a specified amount (for example, 70% to 100%) is described as the "second reference state".

[0119] The second shooting state CD2 includes, for example, at least one of the position, size, and shape of the image corresponding to the subject PS2 in the shot image PA2 shot by the narrow-angle camera 302 in the second reference state. In the first embodiment and the second embodiment, the second shooting state CD2 includes, for example, the size of the image corresponding to the subject PS2 in the shot image PA2 shot by the narrow-angle camera 302 in the second reference state.

[0120] The second imaging state CD2 corresponds to an example of “an imaging state of the subject captured by the camera when the drawer is pulled out by a predetermined amount” of the present invention.

[0121] The photographed object PS2 is arranged at the upper end of the drawer 26A of the storage room 26, and the shooting state of the narrow-angle camera 302 changes according to the pulling-out amount of the drawer 26A. In addition, the pulling-out amount of the drawer 26A is the pulling-out amount of the drawer 26A relative to the storage room 26. In the first embodiment, the storage room 26 is the freezer room 24, and the drawer 26A is the drawer 24A.

[0122] The subject PS2 is disposed, for example, at the upper end of the drawer 24A, and the photographing state of the subject PS2 captured by the narrow-angle camera 302 changes according to the amount of the drawer 24A being pulled out.

[0123] Reference Fig.11 The subject PS2 will be described further.

[0124] Based on the first imaging state CD1, the generator 356 selects the first imaging image P1 from the imaging images PB1 included in the moving image represented by the first recording data DV1 of the recording data 363. The first imaging image P1 is the imaging image PA1 when the door 21A is in the open state.

[0125] Furthermore, based on the second imaging state CD2, the generator 356 selects the second imaging image P2 from the imaging image PB2 included in the moving image represented by the second video data DV2 of the video data 363. The second imaging image P2 is the imaging image PA2 when the drawer 26A is in the open state.

[0126] Furthermore, the generator 356 generates management data D1 based on the first captured image P1 and the second captured image P2. The management data D1 is data uploaded to the article management server 5 and is data for managing articles stored in the cold storage 2. The first communication control unit 351 transmits the management data D1 generated by the generator 356 to the article management server 5.

[0127] The generating unit 356 corresponds to an example of the “selecting unit” of the present invention.

[0128] The captured image PB2 corresponds to an example of the “image captured by a camera” in the present invention.

[0129] The second captured image P2 corresponds to an example of “a captured image when the drawer is in a pulled-out state” according to the present invention.

[0130] In this embodiment, the case where the imaging device 3 includes the generating unit 356 is described, but the present invention is not limited to this. For example, the article management server 5 may include the generating unit 356. In this case, the management data D1 includes the first recorded data DV1 and the second recorded data DV2.

[0131] Furthermore, in the first embodiment, the generation unit 356 selects the first captured image P1 from the captured images PB1 included in the moving image represented by the first recorded data DV1 of the recorded data 363 based on the first captured state CD1, but the present invention is not limited thereto. The generation unit 356 may select the captured image PB1 in which the user P is least reflected as the first captured image P1 from the captured images PB1 included in the moving image represented by the first recorded data DV1 of the recorded data 363 based on the first captured state CD1.

[0132] In this case, the generation unit 356 further selects the first captured image P1 from the first captured images P1 selected based on the first capturing state CD1 as follows: That is, the generation unit 356 recognizes the image of the user P by a method such as pattern matching, and selects the captured image PB1 having the smallest area of ​​the recognized image of the user P as the first captured image P1.

[0133] Similarly, in the first embodiment, the generation unit 356 selects the second captured image P2 from the captured images PB2 included in the dynamic image represented by the second video data DV2 of the video data 363 based on the second capturing state CD2, but the present invention is not limited thereto. The generation unit 356 may select the captured image PB2 in which the user P is least reflected as the second captured image P2 from the captured images PB2 included in the dynamic image represented by the second video data DV2 of the video data 363 based on the second capturing state CD2.

[0134] [1-1-4. Structure of the subject]

[0135] Next, refer to Fig. 9 An example of the structure of the subject PS will be described. Fig. 91 is a diagram showing the structure of the first embodiment of the subject PS in the first embodiment. In the following description, the subject PS of the first embodiment is referred to as subject PS1. Subject PS1 is composed of a connecting member connecting the left end of the right door 21B and the right end of the opening of the refrigerating chamber 21. With respect to subject PS1, the larger the opening and closing angle of the right door 21B is, the larger the length LD in the left-right direction is.

[0136] In this case, as an example of the first shooting state CD1, the state storage unit 364 stores the length LD in the shot image PA1 shot by the wide-angle camera 301 when the opening and closing angle is set to 60 degrees to 100 degrees. In addition, the generation unit 356 selects the first shot image P1 when the right door 21B is in the open state from the shot image PB1 included in the dynamic image represented by the first video data DV1 of the video data 363 based on the length LD in the shot image PA1 stored in the state storage unit 364.

[0137] Furthermore, the generating unit 356 may select the first captured image P1 when the right door 21B is in the open state from the captured image PB1 included in the dynamic image represented by the first video data DV1 of the video data 363, based on the area of ​​the subject PS1 in the captured image PA1 stored in the state storage unit 364. In this case, the state storage unit 364 may store the area of ​​the subject PS1 in the captured image PA1 captured by the wide-angle camera 301 when the opening and closing angle is set to 60 to 100 degrees as an example of the first captured state CD1.

[0138] exist Fig. 9 In the figure, the case where the subject PS1 is composed of a connecting component connecting the left end of the right door 21B and the right end of the opening of the refrigerator room 21 is described, but the subject PS2 can also be composed of a connecting component connecting the right end of the left door 21C and the left end of the opening of the refrigerator room 21.

[0139] In addition, Fig. 9 , Fig.10 and Fig.11 The diagram and Figure 2 The X-axis, Y-axis, and Z-axis shown are the same X-axis, Y-axis, and Z-axis.

[0140] Next, refer to Figure 10-11 The following description will describe a case where the subject PS is disposed at the upper end of the drawer 24A and the photographing state of the subject PS by the narrow-angle camera 302 changes according to the amount of the drawer 24A being drawn out.

[0141] In the following description, a case will be described where the state storage unit 364 stores the second subject image MK as the capture state CD2.

[0142] In addition, refer to Figure 12-13 The subject image MK will be described.

[0143] Figure 10-11 This is a diagram showing the configuration of a second example of the subject PS in Embodiment 1. In the following description, the subject PS of the second example is referred to as subject PS2. Fig.10 24A is a perspective view showing a state where the drawer 24A is 100% pulled out. Fig.10 It is a perspective view showing the drawer 24A in the most pulled-out state, that is, in the fully opened state.

[0144] like Fig.10 As shown in FIG. 1 , a first tray TR1 and a second tray TR2 are arranged in the drawer 24A.

[0145] The second tray TR2 is configured to be slidable in the front-rear direction, i.e., the Y-axis direction, relative to the upper ends of a pair of side members of the drawer 24A, for example. The pair of side members of the drawer 24A are arranged parallel to the YZ plane. In addition, the portion of the second tray TR2 arranged in the freezer compartment 24 is supported by a guide rail arranged on the side of the inner box 203 constituting the freezer compartment 24 so as to be slidable.

[0146] The first pallet TR1 is arranged above the second pallet TR2, and is configured to be slidable in the front-to-rear direction, i.e., the Y-axis direction, relative to the upper end portions of a pair of side members TS1 of the second pallet TR2, for example. The pair of side members TSA of the second pallet TR2 is arranged parallel to the YZ plane. In addition, the upper end portions of the side members TSA extend in the Y-axis direction. In addition, the portion of the first pallet TR1 arranged in the freezer compartment 24 is supported by a guide rail arranged on the side of the inner box 203 constituting the freezer compartment 24 so as to be slidable.

[0147] The first tray TR1 corresponds to an example of a “storage box”.

[0148] like Fig.10 As shown, the photographed object PS2 is arranged at the upper end of a pair of side parts TS1 of the first pallet TR1. The first pallet TR1 is formed into a box shape with an open upper surface by using a pair of side parts TS1, a front surface part TS2, a back surface part TS3, and a bottom surface part TS4. The pair of side parts TS1 respectively constitute the side surfaces of the first pallet TR1. The front surface part TS2 constitutes the front surface of the first pallet TR1. The back surface part TS3 constitutes the back surface of the first pallet TR1. The bottom surface part TS4 constitutes the bottom surface of the first pallet TR1.

[0149] The end of the subject PS2 on the depth side is arranged at a position corresponding to the prescribed pull-out amount in the pull-out direction of the first pallet TR1, that is, in the Y-axis direction. The distance LA is the dimension in the pull-out direction of the first pallet TR1, that is, in the Y-axis direction. The distance LB is the distance between the front surface part TS2 of the first pallet TR1 and the rear end of the subject PS2. The rear end of the subject PS2 is the negative end of the subject PS2 in the Y-axis direction. The distance LB is set to, for example, 80% of the distance LA. That is, when the pull-out amount of the first pallet TR1 is 80% or more, the subject PS2 is arranged at a position where it can be photographed by the narrow-angle camera 302. In addition, the distance LB may also be the distance between the front surface part TS2 of the first pallet TR1 and the center or front end of the subject PS2.

[0150] In addition, the shooting surface PF2 corresponds to the upper surface of the fixing component AT. The fixing component AT is formed into a columnar shape having a right triangle cross section. The fixing component AT is fixed to the side component TS1 of the first pallet TR1. The side component TS1 is composed of a left side component TS11 constituting the left side of the first pallet TR1, and a right side component TS12 constituting the right side of the first pallet TR1. The fixing component AT is fixed to the inner surface side of each of the left side component TS11 and the right side component TS12. In addition, the fixing component AT may also be fixed to the inner surface side of at least one of the left side component TS11 and the right side component TS12.

[0151] like Fig.10 As shown in FIG. 1 , the fixing member AT is arranged so that the long side direction of the upper surface of the fixing member AT is parallel to the side member TS1. Therefore, the length of the fixing member AT protruding into the interior of the first tray TR1 can be suppressed. Therefore, it is possible to suppress the fixing member AT from becoming an obstacle when storing articles in the first tray TR1 or when taking out articles stored in the first tray TR1.

[0152] Fig.11 1 is a perspective view of the fixing member AT fixed to the right side member TS12. Fig.11 As shown, the fixing member AT is composed of a fitting member AT1 and a columnar member AT2.

[0153] The fitting member AT1 is inserted into a hole (not shown) formed in the upper end portion of the right side member TS12 of the first tray TR1, and the fitting member AT1 fits with the right side member TS12 of the first tray TR1.

[0154] The columnar component AT2 is formed into a columnar shape having a right triangle cross section. The columnar component AT2 has an upper surface component AT21, a first side component AT22, and a pair of second side components AT23. The upper surface component AT21 corresponds to the subject PS2. In addition, the upper surface of the upper surface component AT21 corresponds to the shooting surface PF2. The shooting surface PF2 is a surface of the subject PS1 that is photographed by the narrow-angle camera 302. As shown by the dotted line, a shooting object image MK including identification information indicating the type of items stored in the freezer compartment 24 is formed on the shooting surface PF2. For example, a sealing member formed with the shooting object image MK is attached to the shooting surface PF1.

[0155] The state storage unit 364 stores the subject image MK. The subject image MK corresponds to an example of the first imaging state CD1.

[0156] Based on the captured image MK stored in the state storage unit 364 , the generation unit 356 selects the second captured image P2 when the drawer 24A is in the pulled-out state from the captured image PB2 included in the moving image represented by the second recorded data DV2 of the recorded data 363 .

[0157] Reference Figure 12-13 The first captured subject image MK1 will be further described.

[0158] The first side member AT22 is formed to form a right angle with the upper surface member AT21. A pair of interlocking members AT1 are provided upright on the first side member AT22. The pair of interlocking members AT1 are arranged on the first side member AT22 along the front-back direction, that is, the Y-axis direction. The fixing member AT is fixed to the right side member TS12 by interlocking the interlocking members AT1 with a pair of holes formed in the right side member TS12. When the fixing member AT is fixed to the right side member TS12, the first side member AT22 abuts against the right side member TS12.

[0159] A pair of second side members AT23 is arranged at both ends of the columnar member AT2 in the Y-axis direction, namely, the front side end and the depth side end. The pair of second side members AT23 is integrally formed with the first side member AT22 and the first side member AT22. The pair of second side members AT23 fixes the first side member AT22 and the first side member AT22 in a manner that the first side member AT22 and the first side member AT22 form a right angle with each other.

[0160] exist Fig.10In the above description, the subject PS2 is arranged at the upper end of a pair of side members TS1 of the first tray TR1, but the subject PS2 may be arranged at the upper end of a pair of side members TSA of the second tray TR2. In addition, the subject PS2 may be arranged at the upper end of a pair of side members 24S of the drawer 24A.

[0161] For example, when the subject PS2 is arranged in the first tray TR1, the second tray TR2, and the drawer 24A in the freezer 24, the object image MK preferably includes, in addition to identification information indicating the type of items stored in the freezer 24 in the cold storage 2, area information where the subject PS2 is arranged.

[0162] The photographed object image MK of the photographed object PS2 arranged on the first tray TR1 has, for example, area information indicating the first tray TR1. In addition, the photographed object image MK of the photographed object PS2 arranged on the second tray TR2 has, for example, area information indicating the second tray TR2. In addition, the photographed object image MK of the photographed object PS2 arranged on the drawer 24A has, for example, area information indicating the drawer 24A.

[0163] like Fig.10 As shown in the figure, since the object PS2 is arranged at the upper end of the pair of side members TS1 of the first tray TR1, the distance between the narrow-angle camera 302 and the object PS1 can be shortened compared to the case where the object PS2 is arranged at other positions (for example, the bottom member TS4) of the first tray TR1. As a result, the narrow-angle camera 302 can appropriately capture the object image MK formed on the capturing surface PF2 of the object PS2 in the second reference state.

[0164] In addition, if Fig.10 As shown, since the photographed object PS2 is arranged at the upper end portions of the pair of side members TS1 of the first tray TR1, it is possible to suppress the narrow-angle camera 302 and the photographed object PS2 from being blocked by items stored in the cold storage 2. As a result, the narrow-angle camera 302 can appropriately photograph the photographed object image MK formed on the photographing surface PF2 of the photographed object PS2 in the second reference state.

[0165] Next, refer to Figure 12-13 The subject image MK will be described. Fig.12 This is a diagram showing a first example of the first subject image MK1 of the subject PS2 in the first embodiment.

[0166] like Fig.12As shown, the captured object image MK includes a rectangular outer frame image MF and an identification image MD formed inside the outer frame image MF. The identification image MD corresponds to identification information indicating the type of articles stored in the freezer compartment 24 of the refrigerator 2. The outer frame image MF is formed in black, for example.

[0167] The identification image MD is composed of a first rectangular image PD1 of a first color and a second rectangular image PD2 of a second color different from the first color, and the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 represents identification information. The first color is, for example, white, and the second color is, for example, black.

[0168] The size of the first rectangular image PD1 is the same as that of the second rectangular image PD2, and the first rectangular image PD1 and the second rectangular image PD2 are, for example, squares. In other words, the width WP1 of the first rectangular image PD1 and the width WP2 of the second rectangular image PD2 are the same as the height HP of the first rectangular image PD1 and the second rectangular image PD2.

[0169] The thickness HF in the short side direction (up and down direction) of the outer frame image MF and the thickness WF in the long side direction (left and right direction) of the outer frame image MF are respectively the same as, for example, the width WP1 of the first rectangular image PD1, the width WP2 of the second rectangular image PD2, the height HP of the first rectangular image PD1 and the height HP of the second rectangular image PD2.

[0170] The thickness HF in the short-side direction (up-down direction) of the outer frame image MF and the thickness WF in the long-side direction (left-right direction) of the outer frame image MF are determined based on the resolution of the narrow-angle camera 302 and the distance between the narrow-angle camera 302 and the captured object image MK at a predetermined pulling amount (e.g., 90%) of the drawer 24A. In addition, the thickness HF in the short-side direction (up-down direction) of the outer frame image MF and the thickness WF in the long-side direction (left-right direction) of the outer frame image MF are determined based on the capturing range or viewing angle of the narrow-angle camera 302.

[0171] For example, the greater the resolution of the narrow-angle camera 302, the smaller the thickness HF in the long-side direction of the outer frame image MF and the thickness WF in the short-side direction of the outer frame image MF can be. In addition, the longer the distance between the narrow-angle camera 302 and the captured object image MK at a predetermined pulling amount (e.g., 90%) of the drawer 24A, the greater the thickness HF in the long-side direction of the outer frame image MF and the thickness WF in the short-side direction of the outer frame image MF need to be. In addition, the greater the shooting range or viewing angle of the narrow-angle camera 302, the greater the thickness HF in the long-side direction of the outer frame image MF and the thickness WF in the short-side direction of the outer frame image MF need to be.

[0172] exist Fig.12In the illustrated photographic subject image MK, the thickness HF in the long-side direction of the outer-frame image MF and the thickness WF in the short-side direction of the outer-frame image MF are each, for example, 6 mm.

[0173] The first rectangular image PD1 and the second rectangular image PD2 are arranged along the longitudinal direction (here, the left-right direction) of the outer-frame image MF. Fig.12 In FIG. 1 , five first rectangular images PD1 and four second rectangular images PD2 are arranged as identification image MD. The first rectangular image PD1 represents, for example, "0", and the second rectangular image PD2 represents, for example, "1". That is, identification image MD represents a 9-digit binary number.

[0174] The identification image MD is composed of an information image MDA and a symbol image MDB. The information image MDA is composed of five rectangular images PD, and the symbol image MDB is composed of four rectangular images PD. The rectangular image PD is the first rectangular image PD1 or the second rectangular image PD2. The information image MDA corresponds to the identification information indicating the type of articles stored in the cold storage room 21 of the cold storage 2. The symbol image MDB indicates a symbol for error correction of the identification information shown in the information image MDA.

[0175] The information image MDA is composed of five rectangular images PD, so it represents a 5-digit binary number. That is, the information image MDA represents 2 to the fifth power, that is, 32 types of identification information. For example, Fig.12 The information image MDA shown represents a five-digit binary number "01010".

[0176] The size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the resolution of the narrow-angle camera 302 and the distance between the narrow-angle camera 302 and the captured object image MK at a predetermined pulling-out amount (e.g., 90%) of the drawer 24A. In addition, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the capturing range or viewing angle of the narrow-angle camera 302.

[0177] For example, the larger the resolution of the narrow-angle camera 302, the smaller the size of the first rectangular image PD1 and the size of the second rectangular image PD2 can be. In addition, the longer the distance between the narrow-angle camera 302 and the captured object image MK at a predetermined pulling amount (e.g., 90%) of the drawer 24A, the larger the size of the first rectangular image PD1 and the size of the second rectangular image PD2 need to be. In addition, the larger the capturing range or viewing angle of the narrow-angle camera 302, the larger the size of the first rectangular image PD1 and the size of the second rectangular image PD2 need to be.

[0178] exist Fig.12In the captured object image MK shown, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are squares with a side of 6 mm.

[0179] Fig.13 This is a diagram showing a second example of the captured object image MK of the subject PS in the first embodiment. Fig.13 The photographed object image MK shown in the figure has two rows of rectangular images PD in the vertical direction of the identification image MD. Fig.12 The subject image MK shown is different.

[0180] The information image MDA is composed of ten rectangular images PD, and the symbol image MDB is composed of eight rectangular images PD. The four rectangular images PD in the upper layer of the eight rectangular images PD constituting the symbol image MDB represent symbols for error correction of the identification information shown in the five rectangular images PD in the upper layer of the ten rectangular images PD constituting the information image MDA. In addition, the four rectangular images PD in the lower layer of the eight rectangular images PD constituting the symbol image MDB represent symbols for error correction of the identification information shown in the five rectangular images PD in the lower layer of the ten rectangular images PD constituting the information image MDA.

[0181] exist Fig.13 In the above description, the identification image MD has two rows of rectangular images PD in the vertical direction, but the present invention is not limited thereto. The identification image MD may have three or more rows of rectangular images PD in the vertical direction.

[0182] exist Figure 12-13 In the description, the case where the outer frame image MF is formed in black is described, but the present invention is not limited to this. For example, the outer frame image MF may be formed in blue. In addition, the case where the first color is white and the second color is black is described, but the present invention is not limited to this. For example, the first color may be yellow and the second color may be blue.

[0183] Reference Figure 10-11 The photographed object PS2 described above includes the photographed object image MK. Figure 12-13 As shown in FIG. 1 , the captured object image MK includes a rectangular outer-frame image MF and an identification image MD formed inside the outer-frame image MF. Thus, the generating unit 356 executes the following processing.

[0184] That is, the generation unit 356 first determines whether the outer frame image MF can be detected from each of the captured images PB2 included in the dynamic image represented by the second recorded data DV2 of the recorded data 363. Then, if the outer frame image MF can be detected from the captured image PB2, the generation unit 356 determines whether the identification information represented by the identification image MD inside the detected outer frame image MF can be read. Next, if the identification information represented by the identification image MD can be read and the read identification information is consistent with the identification information included in the captured image MK stored in the state storage unit 364, the generation unit 356 selects the captured image PB2 as the second captured image P2.

[0185] In addition, for example, when the outer frame image MF can be detected from each of the captured images PB1 contained in the dynamic image represented by the second video data DV2, the generation unit 356 notifies the user P that the narrow-angle camera 302 has captured the subject PS by lighting, flashing, or making a sound of an LED.

[0186] The case where the outer-frame image MF can be detected from each of the captured images PB2 corresponds to an example of “a case where it is determined that the camera has captured the subject”.

[0187] In order for the generator 356 to easily detect the outer frame image MF from each of the captured images PB2 included in the moving image represented by the second video data DV2, the color of the outer frame image MF is preferably different from the color of the inner surface of the freezing chamber 24 and the color of the items stored in the freezing chamber 24. Therefore, the color of the outer frame image MF is preferably black, blue, or the like, for example.

[0188] [1-1-5. Structure of the item management server]

[0189] Fig.14 It is a block diagram showing the configuration of the terminal device 4 and the item management server 5.

[0190] The article management server 5 includes a server processor 50 which is a processor such as a CPU, a server memory 51, and a server communication module 52. The server processor 50 is connected to the server memory 51 and the server communication module 52.

[0191] The server memory 51 is a storage device that stores programs and data executed by the server processor 50. The server memory 51 is composed of a non-volatile storage device. In addition, the server memory 51 may also include a non-volatile storage device that constitutes a work area of ​​the server processor 50. The server memory 51 stores data processed by the server processor 50, a control program 511 executed by the server processor 50, and an item management DB (database) 512.

[0192] The article management DB 512 has one record for each imaging device 3. Each record includes the imaging device ID 362, the image data of the first captured image P1, and the image data of the second captured image P2.

[0193] Server communication module 52 includes a communication circuit or a connector based on a predetermined communication standard, and communicates with refrigerator 2 , imaging device 3 , and terminal device 4 via network NW.

[0194] The server processor 50 functions as a server communication control unit 501 and a server processing unit 502 by reading and executing a control program 511 stored in the server memory 51 .

[0195] The server communication control unit 501 communicates with the refrigerator 2 , the imaging device 3 , and the terminal device 4 via the server communication module 52 .

[0196] The server processing unit 502 performs various information processing using the camera 3 and the terminal device 4 as clients. The server processing unit 502 may also use the refrigerator 2 as a client.

[0197] [1-1-6. Configuration of terminal device]

[0198] Next, the structure of the terminal device 4 is described.

[0199] The terminal device 4 includes a terminal processor 40 which is a processor such as a CPU, a terminal memory 41 , a terminal communication module 42 , and a touch panel 43 .

[0200] The terminal memory 41 is a storage device that non-volatilely stores programs and data executed by the terminal processor 40. The terminal memory 41 is composed of a non-volatile storage device. In addition, the terminal memory 41 may also include a RAM that constitutes a work area of ​​the terminal processor 40. The terminal memory 41 stores data processed by the terminal processor 40, a control program executed by the terminal processor 40, and an item management application 411.

[0201] The terminal communication module 42 includes a communication circuit or a connector based on a prescribed communication standard, and communicates with the item management server 5 under the control of the terminal processor 40 .

[0202] The touch panel 43 includes a display panel and a touch sensor overlapped with or provided integrally with the display panel.

[0203] The terminal processor 40 functions as an application execution unit 400 .

[0204] The application execution unit 400 communicates with the item management server 5 via the terminal communication module 42 .

[0205] The application execution unit 400 causes the touch panel 43 to display a user interface related to the management of the items stored in the refrigerator 2. In the following description, this user interface is referred to as "application UI." The display control unit 402 of this embodiment displays images of the items stored in the refrigerator 2 and the like by displaying the application UI.

[0206] The application execution unit 400 receives various inputs from the user P via the application UI.

[0207] [1-2. Action]

[0208] Next, the operation of each unit of the article management system 1 will be described.

[0209] [1-2-1. Actions related to video recording]

[0210] First, the operation of the imaging device 3 related to recording will be described.

[0211] Fig.15 FA is a flowchart showing the operation of the imaging device 3 .

[0212] The opening / closing determination unit 353 determines whether the door 21A or the drawer 26A is in the open state (step SA1 ).

[0213] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is not in the open state (step SA1 : NO), the second processor 350 returns the process to step SA1 .

[0214] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state (step SA1 : Yes), the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start imaging (step SA2 ).

[0215] Next, the recording control unit 355 starts recording the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302 (step SA3 ).

[0216] Next, the opening / closing determination unit 353 determines whether the door 21A and the drawer 26A are closed (step SA4). If the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are not closed (step SA4: No), the determination in step SA4 is performed again.

[0217] On the other hand, when it is determined that the door 21A and the drawer 26A are closed (step SA4 : Yes), the recording control unit 355 ends the recording of the photographing result (step SA5 ). Thus, the recording data 363 is generated in the second memory 360 .

[0218] Next, the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging (step SA6 ).

[0219] [1-2-2. Operations related to transmission of management data]

[0220] Next, the operation related to the transmission of the management data D1 will be described.

[0221] Fig.16 This is a flowchart showing the operation of the imaging device 3 and the article management server 5 .

[0222] exist Fig.16 In the flowchart FB, the operation of the imaging device 3 is shown, and the flowchart FC shows the operation of the article management server 5 is shown.

[0223] The generation unit 356 determines whether the recorded data 363 is stored in the second memory 360 (step SB1). When the generation unit 356 determines that the recorded data 363 is not stored (step SB1: No), the determination of step SB1 is performed again.

[0224] On the other hand, when the generation unit 356 determines that the video data 363 is stored (step SB1: yes), based on the first shooting state CD1 stored in the state storage unit 364, the first shot image P1 is selected from the shot image PB1 contained in the dynamic image represented by the first video data DV1 of the video data 363 stored in the second memory 360 (step SB2).

[0225] Step SB2 will be described in detail.

[0226] The first captured image P1 is a captured image when the right door 21B or the left door 21C is opened at a predetermined opening / closing angle (for example, 90 degrees).

[0227] Furthermore, the generating unit 356 selects the first captured image P1 when the right door 21B is in the open state from the captured image PB1 included in the moving image represented by the first recorded data DV1 of the recorded data 363 based on the length LD in the captured image PA1 stored in the state storing unit 364 .

[0228] Returning to the description of the flowchart FB, the generation unit 356 selects a second captured image P2 from each of the captured images PB2 included in the moving image represented by the second recorded data DV2 of the stored recorded data 363 (step SB3 ).

[0229] Step SB3 will be described in detail.

[0230] The second captured image P2 is a captured image when the drawer 26A is in the open state.

[0231] The generation unit 356 first determines whether the outer-frame image MF can be detected from each of the captured images PB2 included in the dynamic image represented by the second recorded data DV2 of the recorded data 363. Then, if the outer-frame image MF can be detected from the captured image PB2, the generation unit 356 determines whether the identification information represented by the identification image MD inside the detected outer-frame image MF can be read. Next, if the identification information represented by the identification image MD can be read and the read identification information is consistent with the identification information included in the captured image MK stored in the state storage unit 364, the generation unit 356 selects the captured image PB2 as the second captured image P2.

[0232] The generation unit 356 determines whether or not at least one of the first captured image P1 and the second captured image P2 can be selected (step SB4 ).

[0233] When the generation unit 356 determines that the first captured image P1 and the second captured image P2 cannot be selected (step SB4 : No), it performs the process of step SB7 .

[0234] On the other hand, when the generating unit 356 determines that at least one of the first captured image P1 and the second captured image P2 can be selected (step SB4 : Yes), it generates the management data D1 (step SB5 ).

[0235] When the first captured image P1 and the second captured image P2 are selectable, the generating unit 356 generates management data D1 including the image data of the first captured image P1 , the image data of the second captured image P2 , and the imaging device ID 362 .

[0236] When the first captured image P1 can be selected but the second captured image P2 cannot be selected, the generating unit 356 generates management data D1 including the image data of the first captured image P1 and the imaging device ID 362 .

[0237] When the second captured image P2 can be selected but the first captured image P1 cannot be selected, the generating unit 356 generates the management data D1 including the image data of the second captured image P2 and the imaging device ID 362 .

[0238] Next, the first communication control unit 351 transmits the management data D1 generated by the generating unit 356 to the article management server 5 (step SB6).

[0239] Next, the generating unit 356 deletes the recorded data 363 to be processed from the second memory 360 after step SB2 (step SB7 ).

[0240] As shown in flowchart FC, the server communication control unit 501 determines whether the management data D1 is received from the camera device 3 (step SC1). When the server communication control unit 501 determines that the management data D1 is received (step SC1: Yes), the server processing unit 502 updates the item management DB 512 based on the management data D1 received by the server communication control unit 501 (step SC2).

[0241] Step SC2 will be described in detail.

[0242] The server processing unit 502 identifies the record including the camera ID 362 included in the management data D1 from the article management DB 512, and updates the image data of the first captured image P1 in the identified record to the image data of the first captured image P1 included in the management data D1.

[0243] The server processing unit 502 identifies the record including the camera ID 362 included in the management data D1 from the article management DB 512, and updates the image data of the second captured image P2 in the identified record to the image data of the second captured image P2 included in the management data D1.

[0244] [1-2-3. Actions related to application UI]

[0245] Next, the operation of the item management system 1 related to the application UI will be described.

[0246] Fig.17 4 is a flowchart showing the operation of the terminal device 4 and the article management server 5. Fig.17 Flowchart FD represents the operation of the terminal device 4, and flowchart FE represents the operation of the item management server 5.

[0247] As shown in flowchart FD, the application execution unit 400 determines whether to request image data stored in the article management DB 512 from the article management server 5 (step SD1). For example, when the article management application 411 is activated, the terminal communication control unit 401 makes an affirmative determination in step SD1.

[0248] When the application execution unit 400 determines that the image data stored in the article management DB 512 is requested (step SD1: Yes), it sends request data for requesting the image data to the article management server 5 (step SD2). The request data includes the camera ID 362 stored in the terminal memory 41. In addition, the terminal device 4 obtains the camera ID 362 at a predetermined time.

[0249] As shown in flowchart FE, the server communication control unit 501 receives the request data (step SE1). Next, the server processing unit 502 obtains the image data corresponding to the camera ID 362 included in the request data received by the server communication control unit 501 from the item management DB 512 (step SE2). Next, the server communication control unit 501 sends the image data obtained by the server processing unit 502 to the terminal device 4 as a response to the request data (step SE3).

[0250] Referring to flowchart FD, the application execution unit 400 receives image data from the item management server 5 (step SD3). Next, the application execution unit 400 causes the touch panel 43 to display an application UI including an image indicated by the image data received in step SD3 (step SD4).

[0251] [1-3. Effects, etc.]

[0252] Above, as reference Figure 1-Figure 17 As described, the camera system 7 in embodiment 1 includes: a narrow-angle camera 302, which is arranged on the top surface 20A of the cold storage 2 and is used to shoot a cold storage 2 including a freezer chamber 24 having a drawer 24A with an opening formed on the upper surface; a subject PS2, which is arranged at the upper end of the drawer 24A and whose shooting state shot by the narrow-angle camera 302 changes according to the amount of the drawer 24A being pulled out; a state storage unit 364, which stores the shooting state of the subject PS2 shot by the narrow-angle camera 302 when the drawer 24A is pulled out by a specified amount, i.e., a second shooting state CD2; and a generation unit 356, which selects, based on the second shooting state CD2, a shooting image when the drawer 24A is in a pulled-out state from the shooting image PB2 shot by the narrow-angle camera 302.

[0253] Accordingly, based on the second photographic state CD2 of the photographic object PS2 photographed by the narrow-angle camera 302 when the drawer 24A is pulled out by a predetermined amount, the second photographic image P2 when the drawer 24A is pulled out is selected from the photographic image PB2 photographed by the narrow-angle camera 302. Therefore, by appropriately setting the predetermined pulling-out amount, it is possible to select a photographic image that appropriately photographs the storage condition of the drawer 24A of the freezer compartment 24 as the photographic image when the drawer 24A is pulled out, namely, the second photographic image P2. The predetermined pulling-out amount is, for example, 80% to 100%.

[0254] In the imaging system 7 , the subject PS2 is arranged on the drawer 24A so that the imaging plane PF2 of the subject PS2 is included in the imaging area of ​​the narrow-angle camera 302 when the drawer 24A is pulled out to a predetermined amount.

[0255] As a result, when the drawer 24A is pulled out to a predetermined amount, the narrow-angle camera 302 can capture the image of the image plane PF2 of the object PS2.

[0256] In the imaging system 7 , the drawer 24A includes a first tray TR1 configured to be drawable, a subject PS2 is placed on the first tray TR1 , and a fixing member AT is provided to fix the subject PS2 to the upper end of the side surface of the first tray TR1 .

[0257] According to this, since the object PS2 is arranged on the first tray TR1, the photographing state by the narrow-angle camera 302 changes according to the pulling amount of the drawer 24A. Therefore, the object PS2 can be arranged at an appropriate position.

[0258] In addition, in the camera system 7, a subject image MK is formed on the shooting surface PF2 of the subject PS2, and the subject image MK includes identification information indicating the type of items stored in the freezer 24 and at least one of the freezer 24 storing the items. The state storage unit 364 stores the subject image MK, and the generation unit 356 selects the second shot image P2 when the drawer 24A is in a pulled-out state from the shot image PB2 taken by the narrow-angle camera 302 based on the subject image MK stored in the state storage unit 364.

[0259] According to this, by selecting, for example, the captured image PB1 in which the captured object image MK is captured appropriately from among the captured images PB2 captured by the narrow-angle camera 302 as the second captured image P2 , the second captured image P2 can be appropriately selected.

[0260] In the camera system 7 , the captured object image MK includes a rectangular outer frame image MF and an identification image MD formed inside the outer frame image MF and corresponding to identification information indicating the type of articles stored in the freezer compartment 24 of the refrigerator 2 .

[0261] According to this, by selecting, from among the captured images PB2 captured by the narrow-angle camera 302 , the captured image PB2 in which the outer-frame image MF and the identification image MD are captured appropriately as the second captured image P2 , the second captured image P2 can be appropriately selected.

[0262] In the imaging system 7 , the identification image MD is composed of a first rectangular image PD1 of a first color and a second rectangular image PD2 of a second color different from the first color, and the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 represents the identification information.

[0263] According to this, by appropriately setting the first color and the second color, the identification information represented by the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 can be accurately detected.

[0264] In the imaging system 7 , the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are the same, and the first rectangular image PD1 and the second rectangular image PD2 are arranged along the long side direction of the outer-frame image MF.

[0265] Accordingly, since the size of the first rectangular image PD1 is the same as the size of the second rectangular image PD2, the identification information can be effectively represented by the pattern of the arrangement of the first rectangular image PD1 and the second rectangular image PD2. In addition, since the first rectangular image PD1 and the second rectangular image PD2 are arranged along the long side direction of the outer frame image MF, the first rectangular image PD1 and the second rectangular image PD2 can be compactly arranged.

[0266] In the imaging system 7, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the resolution of the narrow-angle camera 302 and the distance between the narrow-angle camera 302 and the captured image MK when the drawer 24A is pulled out a predetermined amount.

[0267] Accordingly, for example, the higher the resolution of the narrow-angle camera 302, the smaller the size of the first rectangular image PD1 and the second rectangular image PD2. Also, for example, the longer the distance between the narrow-angle camera 302 and the captured object image MK when the drawer 24A is pulled out by a predetermined amount, the larger the size of the first rectangular image PD1 and the second rectangular image PD2. In this way, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 can be appropriately determined.

[0268] In addition, in the camera system 7, when it is possible to detect the outer frame image MF contained in the captured image PB2 captured by the wide-angle camera 302 and the identification information represented by the identification image MD of the captured object image MK contained in the captured image PB2 captured by the narrow-angle camera 302 is consistent with the identification information represented by the identification image MD of the captured object image MK stored in the status storage unit 364, the generation unit 356 selects the captured image PB1 as the second captured image P2.

[0269] Accordingly, when the outer frame image MF included in the captured image PB2 captured by the narrow-angle camera 302 can be detected, the captured image PB2 is an appropriately captured image. In addition, when the identification information represented by the identification image MD of the captured object image MK included in the captured image PB2 is consistent with the identification information represented by the identification image MD of the captured object image MK stored in the state storage unit 364, the captured image PB2 is an image captured more appropriately. Therefore, the second captured image P2 can be easily and appropriately selected.

[0270] Furthermore, in the imaging system 7 , when it is determined that the narrow-angle camera 302 has captured the subject PS, the generating unit 356 notifies the user that the narrow-angle camera 302 has captured the subject PS.

[0271] According to this, since the user is informed that the narrow-angle camera 302 has captured the subject PS, the user can easily recognize that the narrow-angle camera 302 has captured the subject PS. Therefore, the convenience of the user can be improved.

[0272] In addition, the camera device 3 in embodiment 1 includes: a narrow-angle camera 302, which is configured on the top surface 20A of the cold storage 2 for photographing a cold storage 2 including a freezer chamber 24 having a drawer 24A with an opening formed on the upper surface; a state storage unit 364, which stores a photographing state of the subject PS2 photographed by the narrow-angle camera 302 when the drawer 24A is pulled out by a specified amount, i.e., a second photographing state CD2; and a generation unit 356, which selects, based on the second photographing state CD2, a photographing image when the drawer 24A is in a pulled-out state from the photographing image PB2 photographed by the narrow-angle camera 302, i.e., a second photographing image P2, wherein the subject PS2 is configured at the upper end of the drawer 24A, and the photographing state photographed by the narrow-angle camera 302 changes according to the pulling-out amount of the drawer 24A.

[0273] According to this, the imaging device 3 in the first embodiment achieves the same effect as the imaging system 7 in the first embodiment.

[0274] (Implementation Method 2)

[0275] Next, Embodiment 2 will be described. In the description of Embodiment 2, the same components as those of the respective components of the article management system 1 of Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted as appropriate.

[0276] [2-1. Structure]

[0277] Fig.18 This is a diagram showing the configuration of an article management system 1A in the second embodiment.

[0278] The article management system 1A in the second embodiment includes a cold storage 2A, an article management server 5, and a terminal device 4. The cold storage 2 and the camera 3 in the first embodiment described above are separate bodies, but the cold storage 2A in the second embodiment includes the camera 3A. That is, the cold storage 2A and the camera 3A in the present embodiment are not separate bodies.

[0279] Fig.19 It is a block diagram showing the configuration of refrigerator 2A in the present embodiment.

[0280] Refrigerator 2A includes third control board 220. Third control board 220 includes third processor 230 such as a CPU, third memory 240, third communication module 250, and an interface circuit for connecting to other devices or sensors.

[0281] The third memory 240 is a storage device that stores programs and data executed by the third processor 230. The third memory 240 is composed of a non-volatile storage device. In addition, the third memory 240 may also include a volatile storage device that constitutes a working area of ​​the third processor 230. The third memory 240 stores data processed by the third processor 230 and a control program 241 executed by the third processor 230.

[0282] The third communication module 250 includes communication hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the first control substrate 311A ​​under the control of the third processor 230 .

[0283] The second wireless communication module 206 , the cooling unit 207 , the first opening / closing sensor 208 , the second opening / closing sensor 209 , the third opening / closing sensor 210 , the fourth opening / closing sensor 211 , the fifth opening / closing sensor 212 , and the sixth opening / closing sensor 213 are connected to the third control substrate 220 .

[0284] The cooling unit 207 includes a compressor or a condenser, a capillary tube, a cooler, a cooling fan for conveying cold air generated by the cooler to each storage room, and a damper for diverting the cold air conveyed by the cooling fan, etc., which are mechanisms for cooling each storage room of the cold storage 2A. The cooling unit 207 cools each storage room of the cold storage 2A under the control of the third control board 220.

[0285] The first opening / closing sensor 208 to the sixth opening / closing sensor 213 are constituted by, for example, a Hall IC.

[0286] The first opening / closing sensor 208 is a sensor for detecting the opening / closing of the right door 21B.

[0287] The second opening / closing sensor 209 is a sensor for detecting the opening / closing of the left door 21C.

[0288] The third opening / closing sensor 210 is a sensor for detecting the opening / closing of the drawer 22A.

[0289] The fourth opening / closing sensor 211 is a sensor for detecting the opening / closing of the drawer 23A.

[0290] The fifth opening / closing sensor 212 is a sensor for detecting the opening / closing of the drawer 24A.

[0291] The sixth opening / closing sensor 213 is a sensor for detecting the opening / closing of the drawer 25A.

[0292] Each opening and closing sensor outputs a detection value to the third control substrate 220 .

[0293] The third processor 230 reads out and executes the control program 241 to communicate with the external device via the second wireless communication module 206 .

[0294] Furthermore, third processor 230 reads out and executes control program 241 to cool each storage room of cold storage 2A using cooling unit 207 .

[0295] Furthermore, the third processor 230 reads out and executes the control program 241 , thereby transmitting the detection values ​​outputted from the first opening / closing sensor 208 to the sixth opening / closing sensor 213 to the first control substrate 311A.

[0296] Refrigerator 2A includes imaging device 3A. Compared with imaging device 3, imaging device 3A does not include first optical sensor 303, second optical sensor 304, thermopile 305, human detection sensor 306, second LED 308, and third LED 309.

[0297] In addition, the subject PS is arranged near the opening of the refrigerating chamber 21 of the refrigerator 2A, the right door 21B, or the left door 21C, and the shooting state by the wide-angle camera 301 changes according to the opening and closing angles of the right door 21B and the left door 21C.

[0298] In addition, the subject PS constitutes a part of the refrigerator 2A of the present invention.

[0299] The camera device 3A of the second embodiment includes a first control substrate 311A ​​instead of the first control substrate 311. The power button 32, the first LED 33, and the illuminance sensor 307 are connected to the first control substrate 311A. In addition, the camera device 3A of the second embodiment includes a second control substrate 312A instead of the second control substrate 312. The wide-angle camera 301, the narrow-angle camera 302, and the first wireless communication module 313 are connected to the second control substrate 312A.

[0300] The first control substrate 311A ​​is provided in the imaging device 3A at the same position as the first control substrate 311 in the imaging device 3 .

[0301] In addition, the second control substrate 312A is provided in the imaging device 3A at the same position as the installation position of the second control substrate 312 in the imaging device 3 .

[0302] In this embodiment, the wide-angle camera 301 , the narrow-angle camera 302 , the power button 32 , the first LED 33 , the illuminance sensor 307 , and the first wireless communication module 313 are disposed at the same positions as those in the first embodiment.

[0303] The first control substrate 311A ​​includes a first processor 320A such as a CPU, a first memory 330A, a first communication module 340A, and an interface circuit for connecting to other devices or sensors.

[0304] The first memory 330A is the same memory as the first memory 330. The first memory 330A stores data processed by the first processor 320A and a control program 331A executed by the first processor 320A.

[0305] The first communication module 340A includes communication hardware such as a communication circuit or a connector in compliance with a predetermined wired communication standard, and communicates with the second control board 312A and the third control board 220 via a cable.

[0306] The first processor 320A reads out and executes the control program 331A, thereby turning the power of the imaging device 3A on and off in accordance with the operation of the power button 32 .

[0307] The first processor 320A turns on and off the first LED 33 by reading and executing the control program 331A.

[0308] The first processor 320A reads out and executes the control program 331A, thereby transmitting the detection value output by the illuminance sensor 307 to the second control board 312A via the first communication module 340A.

[0309] The first processor 320A reads out and executes the control program 331A, thereby transmitting the detection values ​​of the first to sixth opening / closing sensors 208 to 213 received from the third control board 220 to the second control board 312A via the first communication module 340A.

[0310] The second control substrate 312A includes a second processor 350A such as a CPU, a second memory 360A, a second communication module 370A, and an interface circuit for connecting to other devices or sensors.

[0311] The second memory 360A is the same memory as the first memory 330. The second memory 360A stores data processed by the second processor 350A and a control program 361A executed by the second processor 350A.

[0312] The second communication module 370A includes communication hardware such as a communication circuit or a connector in compliance with a predetermined communication standard, and communicates with the first control substrate 311A ​​under the control of the second processor 350A.

[0313] The second processor 350A reads and executes the control program 361A stored in the second memory 360A, thereby functioning as the first communication control unit 351, the second communication control unit 352A, the opening / closing determination unit 353A, the imaging control unit 354, the video recording control unit 355, and the generating unit 356. In addition, the second processor 350A reads and executes the control program 361A stored in the second memory 360A, thereby causing the second memory 360A to function as the state storage unit 364.

[0314] Refrigerator 2A in Embodiment 2 includes wide-angle camera 301 , subject PS, state storage unit 364 , and generation unit 356 .

[0315] The second communication control unit 352A communicates with the first control board 311A ​​via the second communication module 370A. The second communication control unit 352A receives detection values ​​of various sensors from the first control board 311A.

[0316] The opening and closing determination unit 353A determines whether the right door 21B is in the open state or the closed state based on the detection value of the first opening and closing sensor 208 received by the second communication control unit 352A. When the second communication control unit 352A receives a detection value different from the detection value corresponding to the closed state, the opening and closing determination unit 353 determines that the right door 21B is in the open state. On the other hand, when the second communication control unit 352 receives a detection value corresponding to the closed state, the opening and closing determination unit 353 determines that the right door 21B is in the closed state.

[0317] Similar to the determination regarding the right door 21B, the opening / closing determination unit 353A determines whether the left door 21C is in the open state or the closed state based on the detection value of the second opening / closing sensor 209 received by the second communication control unit 352 .

[0318] The opening / closing determination unit 353A determines whether the drawer 26A is in the open or closed state based on the detection values ​​of the third opening / closing sensor 210 to the sixth opening / closing sensor 213 received by the second communication control unit 352. The opening / closing determination unit 353A makes the same determination as that of the right door 21B.

[0319] [2-2. Action]

[0320] The operation of each unit of the article management system 1A in the second embodiment is the same as the operation of each unit of the article management system 1 in the first embodiment. That is, the operation of the camera 3A related to recording is the same as the operation of the camera 3 in the first embodiment. In addition, the operation of the camera 3A related to the management data D1 is the same as the operation of the camera 3 in the first embodiment.

[0321] [2-3. Effects, etc.]

[0322] Above, as reference Figure 18-Figure 19 As described, the cold storage 2A of embodiment 2 includes: a narrow-angle camera 302, which is arranged on the top surface 20A of the cold storage 2 for photographing the cold storage 2 including a freezer chamber 24 having a drawer 24A with an opening formed on the upper surface; a subject PS2, which is arranged at the upper end of the drawer 24A, and the photographing state photographed by the narrow-angle camera 302 changes according to the pulling-out amount of the drawer 24A; a state storage unit 364, which stores the photographing state of the subject PS2 photographed by the narrow-angle camera 302 when the drawer 24A is pulled out by a specified amount, i.e., a second photographing state CD2; and a generation unit 356, which selects, based on the second photographing state CD2, a photographing image when the drawer 24A is in a pulled-out state from the photographing image PB2 photographed by the narrow-angle camera 302.

[0323] According to this, refrigerator 2A in the second embodiment achieves the same effects as imaging system 7 in the first embodiment.

[0324] (Other Embodiments)

[0325] As described above, the above-mentioned embodiments 1 and 2 are described as examples disclosed in this application. However, the technology in the present invention is not limited to this, and can also be applied to embodiments that have been changed, replaced, added, omitted, etc. In addition, the various components described in the above-mentioned embodiments 1 and 2 can also be combined to form a new embodiment. Therefore, other embodiments are exemplified below.

[0326] In the above-mentioned first and second embodiments, the case where the subject PS2 is arranged in the freezing chamber 24 is described, but the subject PS2 may be arranged in the ice making chamber 22 , the fresh freezing chamber 23 , or the vegetable chamber 25 .

[0327] In the above-mentioned first and second embodiments, the case where the subject PS2 is arranged on the first tray TR1 of the freezing chamber 24 is described, but the subject PS2 may be arranged on the second tray TR2 of the freezing chamber 24 or on the drawer 24A.

[0328] In the above-mentioned first and second embodiments, the case where the photographed object PS2 is fixed to the pair of side members TS1 of the first tray TR1 by the interlocking member AT1 of the fixing member AT is described, but the photographed object PS2 can be arranged at the upper end portions of the pair of side members TS1 of the first tray TR1. For example, the columnar member AT2 of the fixing member AT may be attached to the upper end portions of the pair of side members TS1 of the first tray TR1 by using an adhesive or the like.

[0329] In the above-mentioned first and second embodiments, the identification image MD is composed of the information image MDA and the symbol image MDB, but the identification image MD may be composed of only the information image MDA. In other words, the identification image MD may not include the symbol image MDB.

[0330] In the above-mentioned first and second embodiments, the case where the first rectangular image PD1 and the second rectangular image PD2 are squares has been described, but the first rectangular image PD1 and the second rectangular image PD2 may be rectangular.

[0331] In the above-mentioned embodiments 1 and 2, the case where the outer frame image MF is formed in black is described, but the present invention is not limited to this. For example, the outer frame image MF may be formed in blue. In addition, the case where the first color is white and the second color is black is described, but the present invention is not limited to this. For example, the first color may be yellow and the second color may be blue.

[0332] In order to enable the generator 356 to easily detect the outer frame image MF from each of the captured images PB1 included in the moving image represented by the first video data DV1, the color of the outer frame image MF is preferably different from the color of the inner side surface of the refrigerating chamber 21 and the color of the items stored in the refrigerating chamber 21. Therefore, the color of the outer frame image MF is preferably black, blue, etc., for example.

[0333] In the above-mentioned second embodiment, the first optical sensor 303, the second optical sensor 304, the thermopile 305, the human sensor 306, the second LED 308, and the third LED 309 are not included. In other embodiments of the second embodiment, any of these devices may be included.

[0334] In the above-mentioned Embodiments 1 and 2, the right door 21B and the left door 21C have different widths. In other embodiments, the right door 21B and the left door 21C may have the same width, or the left door 21C may have a width greater than the right door 21B.

[0335] In the above-mentioned Embodiments 1 and 2, the imaging device 3 and 3A is configured to include the second LED 308 and the third LED 309. In other embodiments, the second LED 308 and the third LED 309 may be configured by a single LED.

[0336] In the above-mentioned embodiment 1, the case where the sensor for detecting the opening and closing of the drawer 26A is composed of the thermopile 305 is exemplified. However, the sensor for detecting the opening and closing of the drawer 26A may also be a distance sensor. In this case, the opening and closing judgment unit 353 judges whether the drawer 26A is in an open state or a closed state based on the detection value of the distance sensor. When the drawer 26A is pulled out, it is detected as having an action. In addition, the distance sensor may be an infrared method or an ultrasonic method.

[0337] In the above-mentioned second embodiment, the case where the first opening and closing sensor 208 to the sixth opening and closing sensor 213 are constituted by a Hall IC is exemplified. However, the first opening and closing sensor 208 and the second opening and closing sensor 209 may also be other sensors such as an angle sensor, a camera, and a temperature sensor. In addition, the third opening and closing sensor 210 to the sixth opening and closing sensor 213 may also be other sensors such as an acceleration sensor, a mechanical switch, an angle sensor, a camera, and a temperature sensor.

[0338] In other embodiments, the generator 356 may compare the image captured by the camera with a pre-stored image of the drawer being pulled out, and select the image of the drawer being pulled out.

[0339] In other embodiments, the camera device 3 may also include a subject detection unit. In this case, the video control unit 355 may also control the start and end of the video recording based on the fact that the subject detection unit can detect the subject. This can reduce the capacity of the video data. In addition, the processing speed can be increased.

[0340] The subject detection unit corresponds to an example of a “selection unit”.

[0341] The operation of the imaging device 3 related to recording of a subject will be described. Fig. 20 It is a flowchart FF showing the operation of the imaging device 3.

[0342] The opening / closing determination unit 353 determines whether the door 21A or the drawer 26A is in the open state (step SF1 ).

[0343] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is not in the open state (step SF1 : NO), the second processor 350 returns the process to step SF1 .

[0344] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state (step SF1 : Yes), the imaging control unit 354 causes the wide-angle camera and the narrow-angle camera 302 to start imaging (step SF2 ).

[0345] Next, the subject detection unit determines whether the subject can be detected (step SF3 ).

[0346] When determining that the subject detection unit cannot detect the subject (step SF3 : NO), the second processor 350 advances the process to step SF7 .

[0347] When it is determined that the subject detection unit can detect the subject (step SF3: Yes), the recording control unit 355 starts recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 (step SF4). In addition, based on the subject corresponding to the wide-angle camera 301, only the recording of the shooting results of the wide-angle camera 301 may be started. Similarly, based on the subject corresponding to the narrow-angle camera 302, only the recording of the shooting results of the narrow-angle camera 302 may be started. In addition, when it is determined that the subject detection unit can detect the subject, the subject detection unit may also notify the user that the subject can be detected (the recording of the shooting results has been started). Accordingly, since the user is notified that the subject is detected (the recording of the shooting results has been started), the user can easily recognize that the recording of the shooting results has been started. Therefore, the noise of shooting is reduced, and the convenience of the user can be improved.

[0348] After the recording control unit 355 starts recording the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302 , it determines whether the imaged object detection unit has detected the object (step SF5 ).

[0349] When it is determined that the subject detection unit can detect the subject (step SF5 : Yes), the determination in step SF5 is performed again.

[0350] On the other hand, when it is determined that the subject detection unit cannot detect the subject (step SF5: No), the video control unit 355 ends the video recording of the shooting result (step SF6). As a result, video data 363 is generated in the second memory 360. In addition, the video data 363 may also include a plurality of video data. For example, a plurality of video data may be generated in the second memory 360 through a single shooting. In this case, the generation unit 356 may also perform the processing of steps SB1 to SB7 on the plurality of video data respectively. In addition, when it is determined that the subject detection unit cannot detect the subject, the user may be notified that the subject detection unit cannot detect the subject (the video recording of the shooting result has been ended). Accordingly, since the user is notified that the subject cannot be detected (the video recording of the shooting result has been ended), the user can easily recognize that the video recording of the shooting result has been ended. Therefore, the convenience of the user can be improved.

[0351] Next, the opening / closing determination unit 353 determines whether the door 21A and the drawer 26A are closed (step SF7). If the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are not closed (step SF7: No), the second processor 350 returns the process to step SF3.

[0352] On the other hand, when the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are closed (step SF7 : Yes), the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging (step SF8 ).

[0353] In other embodiments, the generator 356 may compare the image captured by the camera with pre-stored images of the drawer in the open state, and select the image of the drawer in the open state.

[0354] In another embodiment, the recording control unit 355 may end the recording of the image capture result after a predetermined time (for example, 1 minute) has passed since the start of the recording of the image capture result. This can also cope with a malfunction of the image capture device 3 .

[0355] In other embodiments, the camera device 3 may also include a camera device confirmation unit. When the user sets the camera device 3 in the cold storage 2, the camera device confirmation unit is used to confirm that the camera device 3 can be set in a specified position of the cold storage 2. In this case, the camera device confirmation unit may also notify the user that the camera device 3 can be set in a specified position of the cold storage 2 (for example, the center position in the left-right direction of the cold storage 2) by lighting up, flashing, or making a sound or notifying the terminal device 4 when the subject can be detected in a pre-stored area. Accordingly, the user can easily recognize that the camera device 3 can be set in the specified position of the cold storage 2 and can move to the next setting operation. Therefore, the convenience of the user can be improved.

[0356] In addition, the camera device confirmation unit may also notify the user that the camera device 3 cannot be set at the specified position of the cold storage 2 by lighting, flashing, or sound of an LED or notifying the terminal device 4 when the subject cannot be detected in the pre-stored area. Accordingly, the user can easily recognize that the camera device 3 cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved. In addition, the camera device confirmation unit may also make the notification method to the user different when the camera device 3 can be set at the specified position of the cold storage 2 and when the camera device 3 cannot be set at the specified position of the cold storage 2. For example, the LED may be lit when the camera device 3 can be set at the specified position of the cold storage 2, and the LED may be flashed when the camera device 3 cannot be set at the specified position of the cold storage 2. Accordingly, the user can easily recognize that the camera device 3 can be set at the specified position of the cold storage 2 and that the camera device 3 cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved.

[0357] In other embodiments, the camera device 3 may also include a subject confirmation unit. When the user sets the subject in the cold storage 2, the subject confirmation unit is used to confirm that the subject can be set in the specified position of the cold storage 2. In this case, the subject confirmation unit may also notify the user that the subject can be set in the specified position of the cold storage 2 by lighting, flashing, or sound of an LED or notifying the terminal device 4 when the information of the subject (information obtained from the photographed subject) is consistent with the information of the pre-stored storage room. Accordingly, for example, the user can easily recognize that the subject for the freezer is not set in the vegetable room or the subject for the lower box is not set in the upper box. That is, the user can easily recognize that the subject can be set in the specified position of the cold storage 2 and can move to the next setting operation. Therefore, the convenience of the user can be improved.

[0358] In addition, the subject confirmation unit may also notify the user that the subject cannot be set at the specified position of the cold storage 2 by lighting, flashing, or sound of an LED or notification to the terminal device 4 (for example, displaying the information of the subject) when the information of the subject (the information obtained from the photographed subject) is inconsistent with the information of the pre-stored storage room. Accordingly, the user can easily recognize that the subject cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved. In addition, the subject confirmation unit may also make the notification method to the user different when the subject can be set at the specified position of the cold storage 2 and when the subject cannot be set at the specified position of the cold storage 2. For example, the LED may be lit when the subject can be set at the specified position of the cold storage 2, and the LED may be flashed when the subject cannot be set at the specified position of the cold storage 2. Accordingly, the user can easily recognize that the subject can be set at the specified position of the cold storage 2 and that the subject cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved.

[0359] In addition, when the subject is set only on one side of the storage box, the subject confirmation unit may notify the user that the subject can be set at the specified position of the cold storage 2 by lighting, flashing, or making a sound of an LED, or by notifying the terminal device 4, when the pre-registered information (which side is set) is consistent with the subject information (the information of the side obtained from the photographed subject). Thus, the user can easily recognize that the subject can be set at the specified position of the cold storage 2 and can move to the next setting operation. Therefore, the convenience of the user can be improved.

[0360] In addition, the subject confirmation unit may also notify the user that the subject cannot be set at the specified position of the cold storage 2 by lighting, flashing, or sound of an LED, or by notifying the terminal device 4 (for example, displaying the information of the subject) when the pre-registered information is inconsistent with the information of the subject. Accordingly, the user can easily recognize that the subject cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved. In addition, the subject confirmation unit may also make the notification method to the user different when the subject can be set at the specified position of the cold storage 2 and when the subject cannot be set at the specified position of the cold storage 2. For example, the LED may be lit when the subject can be set at the specified position of the cold storage 2, and the LED may be flashed when the subject cannot be set at the specified position of the cold storage 2. Accordingly, the user can easily recognize that the subject can be set at the specified position of the cold storage 2 and that the subject cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved.

[0361] The photographed object may also include characters or illustrations that provide information about the storage room or storage box to be set (e.g., characters such as the upper box of the vegetable compartment, illustrations of the upper box of the vegetable compartment, or marks of fruits). Accordingly, when the user sets the photographed object in the cold storage 2, the storage room or storage box to be set can be easily identified. This can prevent the user from making setting errors and improve the convenience of the user.

[0362] In other embodiments, in addition to the functional units of the second processors 350 and 350A in Embodiments 1 and 2, the second processors 350 and 350A may also function as a detection unit for detecting an object. The detection unit may also cut out images of objects entering and leaving the refrigerating chamber 21 from the dynamic image during the period from the start to the end of the recording of the wide-angle camera 301 and the narrow-angle camera 302 by the recording control unit 355. In addition, when a mark is attached to the object, the detection unit may detect the marked object.

[0363] In other embodiments, in addition to the functional units of the second processor 350 in Embodiment 1, the second processor 350 may also function as a person presence determination unit. The person presence determination unit determines whether a person exists around the cold storage 2 based on the detection value of the human sensing sensor 306. More specifically, the person presence determination unit determines that a person exists around the cold storage 2 when the detection value of the human sensing sensor 306 is a detection value corresponding to the presence of a person, and determines that no person exists around the cold storage 2 when the detection value of the human sensing sensor 306 is a detection value corresponding to the presence of a person. Otherwise, the person presence determination unit determines that no person exists around the cold storage 2.

[0364] When the second processor 350 functions as a person presence determination unit, if the person presence determination unit determines that a person exists, the shooting control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start shooting, and if the person presence determination unit determines that a person does not exist, the shooting control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to stop shooting. In addition, when the second processor 350 functions as a person presence determination unit, if the person presence determination unit determines that a person exists, the recording control unit 355 starts recording, and if the person presence determination unit determines that a person does not exist, the recording control unit 355 stops recording. Thus, even if the opening and closing determination unit 353 makes an erroneous determination, the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 can be properly recorded.

[0365] In other embodiments, in addition to the functional units of the second processors 350 and 350A in embodiments 1 and 2, the second processors 350 and 350A may also function as an entry and exit judgment unit. In this case, the entry and exit judgment unit changes the boundary line used for the entry and exit judgment according to the opening and closing angles of the left door 21C and the right door 21B, and judges the entry and exit of the article. In addition, the entry and exit judgment unit changes the boundary line used for the entry and exit judgment according to the pulling degree of the drawer 26A, and judges the entry and exit of the article.

[0366] In other embodiments, the first captured image P1 may be subjected to image correction and uploaded to the article management server 5. As an example of image correction, correction of viewpoint conversion is exemplified.

[0367] In the above-described embodiment, the first captured image P1 and the second captured image P2 are used for display purposes of the terminal device 4. However, the first captured image P1 and the second captured image P2 may be used for other purposes such as detecting items stored in the cold storage 2 or 2A.

[0368] In other embodiments, the compartment formed in the main body 20 of the cold storage 2, 2A may also have storage rooms other than the above five types, or may be less. In addition, in other embodiments, the drawer-type storage room formed in the main body 20 of the cold storage 2, 2A may also be one.

[0369] The first processor 320, 320A, the second processor 350, 350A, the server processor 50, and the terminal processor 40 may be composed of a single processor or a plurality of processors. These processors may also be hardware programmed in a manner to implement corresponding functional units. That is, these processors may also be composed of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0370] Figure 8 , Fig.14 ,and Fig.19 The structures of the camera device 3, 3A, the cold storage 2, 2A, the terminal device 4, and the article management server 5 shown are examples, and the specific installation form is not particularly limited. That is, it is not necessary to install the corresponding hardware for each part separately, and it is also possible to set the structure in which the functions of each part are realized by executing a program by a processor. In addition, part of the functions realized by software in the above-mentioned embodiment can also be set as hardware, or part of the functions realized by hardware can also be realized by software.

[0371] Fig.15 , Fig.16 , Fig.17 and Fig. 20 The step units of the actions shown are divided according to the main processing content for easy understanding of the actions, and the actions are not limited by the method or name of the division of the processing units. It can also be divided into more step units according to the processing content. In addition, a step unit can also be divided to include more processing. In addition, the order of the steps can also be appropriately replaced within the scope of not hindering the purpose of the present invention.

[0372] Furthermore, since the above-described embodiments are provided to illustrate the technology of the present invention, various changes, substitutions, additions, omissions, etc. may be made within the scope of the claims or their equivalents.

[0373] (Note)

[0374] The following technology is disclosed through the description of the above embodiments.

[0375] (Technology 1) A camera system, comprising: a camera, which is arranged on the upper surface of the cold storage and is configured to capture a storage room including a drawer having an opening formed on the upper surface; a subject, which is arranged at the upper end of the drawer, and the capturing state captured by the camera changes according to the amount of the drawer being pulled out; a state storage unit, which stores the capturing state of the subject captured by the camera when the drawer is pulled out by a specified amount; and a selection unit, which selects, from the image captured by the camera, an image captured when the drawer is in a pulled-out state based on the capturing state of the subject captured by the camera stored in the state storage unit.

[0376] According to this structure, based on the photographic state of the subject photographed by the camera when the drawer is pulled out by a specified amount, the photographic image when the drawer is pulled out is selected from the image photographed by the camera. Therefore, by appropriately setting the specified pulling amount, it is possible to select a photographic image that appropriately photographs the internal conditions of the cold storage as the photographic image when the drawer is pulled out.

[0377] (Technique 2) The imaging system according to Technique 1, wherein when the drawer is pulled out by a predetermined amount, the subject is arranged in the drawer so that an imaging surface of the subject is included in an imaging area of ​​the camera.

[0378] According to this configuration, when the drawer is pulled out to a predetermined amount, the camera can capture an image of the subject.

[0379] (Technique 3) According to the camera system described in Technique 2, wherein the drawer is configured with a storage box that can be pulled out, the subject is configured in the storage box, and the camera system includes a fixing component that fixes the subject to the upper end of the side surface of the storage box.

[0380] According to this structure, since the subject is fixed to the upper end of the side surface of the storage box, the photographing state of the subject by the camera changes according to the pulling amount of the drawer. Therefore, the subject can be arranged at an appropriate position.

[0381] (Technique 4) An imaging system according to any one of Techniques 1 to 3, wherein a subject image is formed on a photographing surface of the object, the subject image includes identification information indicating the types of items stored in the storage room or the storage room where the items are stored, the state storage unit stores the subject image, and the selection unit selects, from the images captured by the camera, an image captured when the drawer is in a pulled-out state based on the subject image stored in the state storage unit.

[0382] According to this structure, by selecting, for example, an image that appropriately captures the image of the subject from the images captured by the camera as the image captured when the drawer is in the pulled-out state, it is possible to appropriately select the image captured when the drawer is in the pulled-out state.

[0383] (Technology 5) An imaging system according to Technology 4, wherein the captured object image includes a rectangular outer frame image, and an identification image formed inside the outer frame image and corresponding to identification information indicating the type of items stored in the storage room in the cold storage, or the storage room that stores the items.

[0384] According to this structure, by selecting, for example, an image in which the outer frame image and the identification image are appropriately captured from the images captured by the camera as the image captured when the drawer is in a pulled-out state, it is possible to appropriately select an image captured when the drawer is in a pulled-out state.

[0385] (Technology 6) An imaging system according to Technology 5, wherein the identification image is composed of a first rectangular image of a first color and a second rectangular image of a second color different from the first color, and a pattern of the configuration of the first rectangular image and the second rectangular image represents the identification information.

[0386] According to this configuration, by appropriately setting each of the first color and the second color, the identification information represented by the pattern of arrangement of the first rectangular image and the second rectangular image can be accurately detected.

[0387] (Technique 7) An imaging system according to Technique 6, wherein the size of the first rectangular image is the same as the size of the second rectangular image, and the first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image.

[0388] According to this configuration, since the size of the first rectangular image is the same as the size of the second rectangular image, identification information can be effectively represented by the pattern of the arrangement of the first rectangular image and the second rectangular image. In addition, since the first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image, the first rectangular image and the second rectangular image can be compactly arranged.

[0389] (Technique 8) A camera system according to Technique 6 or Technique 7, wherein the size of the first rectangular image and the size of the second rectangular image are determined based on the resolution of the camera and the distance between the camera and the image of the object when the drawer is pulled out a specified amount.

[0390] According to this configuration, for example, the higher the resolution of the camera, the smaller the size of the first rectangular image and the second rectangular image. Also, for example, the longer the distance between the camera and the captured object image when the drawer is pulled out by a predetermined amount, the larger the size of the first rectangular image and the second rectangular image. In this way, the size of the first rectangular image and the size of the second rectangular image can be appropriately determined.

[0391] (Technique 9) A camera system according to any one of Techniques 5 to 8, wherein, when the outer frame image contained in the image taken by the camera can be detected and the identification information represented by the identification image of the photographed object contained in the image taken by the camera is consistent with the identification information represented by the identification image of the photographed object stored in the state storage unit, the selection unit selects the image taken by the camera as the image taken when the drawer is in the pulled-out state.

[0392] According to this structure, when the outer frame image included in the image captured by the camera can be detected, the image captured by the camera is an image that is captured appropriately. In addition, when the identification information represented by the identification image of the captured object image included in the image captured by the camera is consistent with the identification information represented by the identification image of the captured object image stored in the state storage unit, the image captured by the camera is an image that is captured more appropriately. Therefore, the image captured when the drawer is in the pulled-out state can be easily and appropriately selected.

[0393] (Technique 10) In the imaging system according to any one of Techniques 1 to 9, when the selection unit determines that the camera has captured the subject, the selection unit notifies a user that the camera has captured the subject.

[0394] According to this configuration, when it is determined that the camera has captured the subject, the user is notified that the camera has captured the subject, so the user can easily recognize that the camera has captured the subject. Therefore, user convenience can be improved.

[0395] (Technology 11) A camera device, comprising: a camera, which is arranged on the upper surface of the cold storage and is used to photograph a cold storage including a storage room having a drawer with an opening formed on the upper surface; a state storage unit, which stores a photographing state of a subject photographed by the camera when the drawer is pulled out a specified amount; and a selection unit, which selects a photographing image when the drawer is in a pulled-out state from the image photographed by the camera based on the photographing state of the subject photographed by the camera stored in the state storage unit, wherein the subject is arranged at the upper end of the storage room, and the photographing state photographed by the camera changes according to the pulling-out amount of the drawer.

[0396] According to this structure, the same effect as the imaging system described in the technique 1 is achieved.

[0397] (Technology 12) A cold storage, comprising: a camera, which is arranged on the upper surface of the cold storage and is configured to shoot a storage room including a drawer having an opening formed on the upper surface; a subject, which is arranged at the upper end of the drawer, and the shooting state of the subject shot by the camera changes according to the amount of the drawer being pulled out; a state storage unit, which stores the shooting state of the subject shot by the camera when the drawer is pulled out by a specified amount; and a selection unit, which selects, from the image shot by the camera, an image shot when the drawer is in a pulled-out state based on the shooting state of the subject shot by the camera stored in the state storage unit.

[0398] According to this structure, the same effect as the imaging system described in the technique 1 is achieved.

[0399] Industrial Applicability

[0400] As described above, the imaging system, imaging device, and refrigerator of the present invention can be used for selecting an image that appropriately captures the internal conditions of the refrigerator from images captured by a camera.

[0401] Description of Reference Numerals

[0402] 1. 1A Item Management System

[0403] 2. 2A cold storage

[0404] 3. 3A Camera Device

[0405] 4 Terminal Devices

[0406] 5 Item Management Server

[0407] 7 Camera System

[0408] 20A top surface (upper surface)

[0409] 22Ice making room (storage room)

[0410] 23Fresh freezing room (storage room)

[0411] 24 Freezer (Storage Room)

[0412] 25 Vegetable room (storage room)

[0413] 26Storage Room

[0414] 22A, 23A, 24A, 25A, 26A drawers

[0415] 301 wide angle camera

[0416] 302 narrow angle camera (camera)

[0417] 350, 350A second processor

[0418] 351 1st Communication Control Unit

[0419] 352, 352A Second communication control unit

[0420] 353, 353A Open / Close Judgment Unit

[0421] 354 Shooting Control Unit

[0422] 355 Video Control Department

[0423] 356 Generation Department (Selected Department)

[0424] 360, 360A 2nd Memory

[0425] 361, 361A control program

[0426] 362 Camera Device ID

[0427] 363 video data

[0428] 364 State storage unit

[0429] AT fixing parts

[0430] AT1 mating parts

[0431] AT2 Columnar Parts

[0432] CD1 1st shooting status

[0433] CD2 Second shooting state (the subject is photographed by the camera when the drawer is pulled out a predetermined amount)

[0434] Shooting status of the camera)

[0435] D1 management data

[0436] DV1 1st video data

[0437] DV2 2nd video data

[0438] MK Subject Image

[0439] MD recognition image

[0440] MDA infographic

[0441] MDB symbol image

[0442] PD1 1st rectangular image

[0443] PD2 2nd rectangular image

[0444] MF frame image

[0445] P1 1st captured image

[0446] P2 Second captured image (image captured when the drawer is pulled out)

[0447] PB2 captured images (images captured by the camera)

[0448] PF2 shooting surface

[0449] PS1, PS2, PS subject

[0450] TR1 No.1 Tray (Storage Box)

[0451] TR2 2nd tray.

Claims

1. A camera system, characterized in that: include: a camera for photographing a refrigerator including a storage room having a drawer with an opening formed on an upper surface, and arranged on an upper surface of the refrigerator; A photographed object, which is arranged at the upper end of the drawer, and the photographing state of the object photographed by the camera changes according to the amount of the drawer being pulled out; a state storage unit for storing a state of the subject photographed by the camera when the drawer is pulled out by a predetermined amount; and A selection unit selects, from the images captured by the camera, an image captured when the drawer is in a pulled-out state.

2. The camera system according to claim 1, wherein: The subject is arranged in the drawer so that an imaging surface of the subject is included in an imaging area of ​​the camera when the drawer is pulled out by a predetermined amount.

3. The camera system according to claim 2, wherein: The drawer is provided with a storage box which can be pulled out. The photographed object is arranged in the storage box, The imaging system includes a fixing member that fixes the object to be photographed to an upper end portion of a side surface of the storage box.

4. The camera system according to claim 1, wherein: forming a photographic subject image on a photographing surface of the object, the photographic subject image including identification information indicating the type of articles stored in the storage room or the storage room storing the articles, The state storage unit stores the subject image. The selecting unit selects, from among the images captured by the camera, an image captured when the drawer is in a pulled-out state.

5. The camera system according to claim 4, characterized in that: The photographed object image comprises: A rectangular frame image; and An identification image is formed inside the outer frame image and corresponds to identification information indicating the type of articles stored in the storage room or the storage room where the articles are stored.

6. The camera system according to claim 5, characterized in that: The identification image is composed of a first rectangular image of a first color and a second rectangular image of a second color different from the first color. The arrangement pattern of the first rectangular image and the second rectangular image corresponds to the identification information.

7. The camera system according to claim 6, characterized in that: The size of the first rectangular image is the same as the size of the second rectangular image. The first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image.

8. The camera system according to claim 7, characterized in that: The size of the first rectangular image and the size of the second rectangular image are determined based on the resolution of the camera and the distance between the camera and the captured image when the storage room is pulled out by a predetermined amount.

9. The imaging system according to any one of claims 5 to 8, characterized in that: When the outer frame image contained in the image taken by the camera can be detected and the identification information represented by the identification image of the photographed object contained in the image taken by the camera is consistent with the identification information represented by the identification image of the photographed object stored in the state storage unit, the selection unit selects the image taken by the camera as the image taken when the drawer is in the pulled-out state.

10. The camera system according to claim 1, wherein: When the selection unit determines that the camera has captured the subject, the selection unit notifies a user that the camera has captured the subject.

11. A camera device, characterized in that: include: a camera for photographing a refrigerator including a storage room having a drawer with an opening formed on an upper surface, and arranged on an upper surface of the refrigerator; a state storage unit for storing a photographing state of the subject photographed by the camera when the drawer is pulled out by a predetermined amount; and a selecting unit that selects, from the images captured by the camera, an image captured when the drawer is in a pulled-out state, based on the captured state of the subject captured by the camera stored in the state storage unit; The subject is arranged at an upper end portion of the drawer, and a photographing state captured by the camera changes according to a pulling amount of the drawer.

12. A cold storage, characterized in that: include: a camera for photographing a refrigerator including a storage room having a drawer with an opening formed on an upper surface, and arranged on an upper surface of the refrigerator; A photographed object, which is arranged at the upper end of the drawer, and the photographing state of the object photographed by the camera changes according to the pulling amount of the drawer; a state storage unit for storing a state of the subject photographed by the camera when the drawer is pulled out by a predetermined amount; and A selection unit selects, from the images captured by the camera, an image captured when the drawer is in a pulled-out state.

Images