Image pickup system, image pickup device, and refrigerator

Abstract

From the images captured by the camera, a captured image in which the inside condition of the refrigerator is appropriately captured is selected. An image capturing system (7) is provided with: a wide-angle camera (301) which captures an image of a refrigerator (2) having a refrigerating chamber (21) and a door (21A) for opening and closing an opening (201) of the refrigerating chamber (21), and which is disposed on the top surface (20A) of the refrigerator (2); a subject (PS) which is disposed in the vicinity of the opening (201) of the refrigerating chamber (21) or in the door (21A), and which changes the imaging state of the wide-angle camera (301) according to the opening / closing angle of the door (21A); a state storage unit (364) that stores a first imaging state (CD1), which is an imaging state of the subject (PS) at a predetermined opening / closing angle of the door (21A), said imaging state being an imaging state of the subject (PS) by the wide-angle camera (301); and a generation unit (356) that selects a first captured image (P1), which is a captured image when the door (21A) is in an open state, from a captured image (PB1) captured by the wide-angle camera (301) on the basis of the first imaging state (CD1).

Description

Technical Field

[0001] The invention relates to a camera system, a camera device and a cold storage. Background Art

[0002] Patent Document 1 discloses a technology in which a camera is installed on the upper part of a refrigerator, an image is taken when an object passing through an opening of the refrigerator is detected, and food stored in the refrigerator is managed based on the imaged result.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-070476 Summary of the invention

[0006] Problems to be solved by the invention

[0007] The present invention provides an imaging system, an imaging device, and a cold storage for selecting an image that appropriately captures the internal conditions of a cold storage from images captured by a camera.

[0008] Technical solutions to solve problems

[0009] This specification includes all the contents of Japanese Patent Application No. 2022-165223 filed on October 14, 2022.

[0010] The present invention provides a camera system, comprising: a camera, which is arranged on the upper surface of the cold storage and is configured to photograph a cold storage having a storage room and an opening and closing body capable of opening and closing the opening of the storage room; a photographed object, which is arranged near the opening of the storage room or the opening and closing body, and the photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body; a state storage unit, which stores the photographing state of the photographed object photographed by the camera at a specified opening and closing angle of the opening and closing body; and a selection unit, which selects a photographed image when the opening and closing body is in an open state from the image photographed by the camera based on the photographing state of the photographed object photographed by the camera stored in the state storage unit.

[0011] In addition, the present invention provides a camera device, comprising: a camera, which photographs a cold storage having a storage room and an opening and closing body capable of opening and closing an opening of the storage room, and is arranged on the upper surface of the cold storage; a state storage unit, which stores a photographing state of a subject photographed by the camera at a specified opening and closing angle of the opening and closing body; and a selection unit, which selects a photographing image when the opening and closing body is in an open state from an image photographed by the camera based on the photographing state of the subject photographed by the camera stored in the state storage unit, wherein the subject is arranged near the opening of the storage room or the opening and closing body, and the photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body.

[0012] In addition, the present invention provides a cold storage, comprising: a camera, which photographs the cold storage having a storage room and an opening and closing body capable of opening and closing the opening of the storage room, and is arranged on the upper surface of the cold storage; a subject, which is arranged near the opening of the storage room or the opening and closing body, and a photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body; a state storage unit, which stores the photographing state of the subject photographed by the camera at a specified opening and closing angle of the opening and closing body; and a selection unit, which selects a photographed image when the opening and closing body is in an open state from the image photographed by the camera based on the photographing state of the subject photographed by the camera stored in the state storage unit.

[0013] Effects of the Invention

[0014] The camera system, camera device, and cold storage of the present invention can select a captured image when the opening and closing body is in the open state from the image captured by the camera based on the captured state of the subject captured by the camera stored in the state storage unit. Therefore, according to the present invention, an effect is achieved that a captured image that properly captures the internal conditions of the cold storage can be selected from the image captured by the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a diagram showing the structure of the article management system in the first embodiment.

[0016] Figure 2 This is a perspective view of the refrigerator in Embodiment 1.

[0017] Figure 3 This is a side view of the imaging device in Embodiment 1 as viewed from the right.

[0018] Figure 4 This is a perspective view of the restriction member abutting against the front end edge of the main housing in Embodiment 1 as viewed from the left front.

[0019] Figure 5This is a front view of the refrigerator and the imaging device in Embodiment 1 as seen from the front.

[0020] Figure 6 This is a side view of the refrigerator and the imaging device in Embodiment 1 as seen from the right.

[0021] Figure 7 This is a plan view of the imaging device installed in the refrigerator in Embodiment 1, viewed from above.

[0022] Figure 8 This is a block diagram showing the structure of the imaging device in Embodiment 1.

[0023] Fig. 9 This is a diagram showing the structure of a fourth example of the subject in the first embodiment.

[0024] Fig.10 This is a diagram showing the configuration of a first example of a subject in the first embodiment.

[0025] Fig.11 This is a diagram showing the configuration of a first example of a subject in the first embodiment.

[0026] Fig.12 This is a diagram showing the configuration of a second example of the subject in the first embodiment.

[0027] Fig.13 This is a diagram showing the configuration of a second example of the subject in the first embodiment.

[0028] Fig.14 This is a diagram showing the configuration of a third example of the subject in the first embodiment.

[0029] Fig.15 This is a diagram showing a first example of a subject image of a photographed object in the first embodiment.

[0030] Fig.16 This is a diagram showing a second example of a subject image of a photographed object in the first embodiment.

[0031] Fig.17 This is a block diagram showing the configuration of a terminal device and an item management server in the first embodiment.

[0032] Fig.18 This is a flowchart showing the operation of the imaging device in the first embodiment.

[0033] Fig.19 This is a flowchart showing the operations of the imaging device and the item management server in the first embodiment.

[0034] Fig. 20This is a flowchart showing the operations of the terminal device and the item management server in Implementation Example 1.

[0035] Fig.21 This is a diagram showing the structure of the article management system in Implementation Example 2.

[0036] Fig. 22 This is a block diagram showing the structure of a cold storage in the second embodiment.

[0037] Fig.23 This is a flowchart showing the operation of the imaging device. DETAILED DESCRIPTION

[0038] (Foundation, etc. that forms the basis of the present invention)

[0039] When the inventors finally conceived of the present invention, there was a technology for performing imaging when an object passing through an opening of a refrigerator was detected, and managing food stored in the refrigerator based on the imaging result.

[0040] Therefore, the inventors discovered a problem of selecting an image that appropriately captured the internal conditions of a refrigerator, and ultimately formed the subject matter of the present invention in order to solve the problem.

[0041] Therefore, the present invention provides an imaging system, an imaging device, and a refrigerator for selecting an image that appropriately captures the internal conditions of a refrigerator from images captured by a camera.

[0042] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. However, sometimes, more detailed descriptions than necessary will be omitted. For example, sometimes, detailed descriptions of already well-known matters or repeated descriptions of substantially the same structures will be omitted.

[0043] In addition, the drawings and the following description are provided for those skilled in the art to fully understand the present invention, and are not intended to limit the subject matter described in the claims.

[0044] (Implementation Method 1)

[0045] [1-1. Structure]

[0046] [1-1-1. Structure of the item management system]

[0047] Figure 1 This is a diagram showing the structure of the article management system 1 in the first embodiment.

[0048] The article management system 1 is a system for managing articles stored in the refrigerator 2. As an example of article management, the article management system 1 of the present embodiment manages images showing articles stored in the refrigerator 2.

[0049] In the present embodiment, the articles stored in the refrigerator 2 include food, containers for storing food, etc. The articles may be marked with a predetermined mark.

[0050] The article management system 1 includes a cold storage 2 .

[0051] The cold storage 2 includes a main body 20 with an opening on the front surface. A refrigerating chamber 21, an ice making chamber 22, a fresh freezing chamber 23, a freezing chamber 24, and a vegetable chamber 25 are formed on the main body 20. These storage chambers are cooled by cold air. A revolving door 21A is provided at the opening 201 on the front surface of the refrigerating chamber 21. The door 21A includes a revolving right door 21B and a revolving left door 21C. Drawers 22A, 23A, 24A, and 25A capable of storing items are provided in the ice making chamber 22, the fresh freezing chamber 23, the freezing chamber 24, and the vegetable chamber 25, respectively. The upper surface of each of the drawers 22A to 25A is open in the setting state. Hereinafter, when the drawers 22A, 23A, 24A, and 25A are collectively referred to without distinguishing them, the reference numeral "26A" is marked and recorded as "drawer 26A".

[0052] Refrigerating room 21 corresponds to an example of the "storage room" of the present invention. Moreover, door 21A corresponds to an example of the "opening and closing body" of the present invention.

[0053] Article management system 1 includes camera 3. Camera 3 is provided on top surface 20A of main body 20 of refrigerator 2. Top surface 20A of main body 20 is made of metal. Camera 3 captures image of refrigerator 2.

[0054] The top surface 20A corresponds to an example of the “upper surface” of the present invention.

[0055] The article management system 1 includes a terminal device 4. The terminal device 4 is a portable computer such as a smartphone. The terminal device 4 has an application program related to the management of articles stored in the refrigerator 2 installed therein, and communicates with the article management server 5 through the function of the application program.

[0056] In the following description, this application is described as an “article management application” and is denoted by the reference numeral “411”.

[0057] exist Figure 1 In FIG. 1 , a user P at home is indicated by a solid line, and a user P outside the home H is indicated by a dotted line. When the terminal device 4 is used by the user P at home, it communicates with the item management server 5 via the communication device 6 or not via the communication device 6. In addition, when the terminal device 4 is used by the user P outside the home H and a communication connection cannot be established with the communication device 6, the terminal device 4 communicates with the item management server 5 not via the communication device 6.

[0058] The communication device 6 is connected to a network NW formed by a public line network or the like, and communicates with the article management server 5 via the network NW. The communication device 6 functions as an interface device for connecting the refrigerator 2, the imaging device 3, and the terminal device 4 to the network NW. The communication device 6 constructs a local area network in the home H.

[0059] The article management system 1 includes an article management server 5. The article management server 5 is a device for managing articles stored in the cold storage 2. As an article management, the article management server 5 stores images of the articles. The article management server 5 is connected to the network NW. In addition, in each figure, the article management server 5 is represented by a single block, but this does not necessarily mean that the article management server 5 is composed of a single device.

[0060] [1-1-2. Structure of cold storage]

[0061] Figure 2 It is a perspective view of the refrigerator 2 in the installed state.

[0062] exist Figure 2 The figure shows an X-axis, a Y-axis, and a Z-axis. The X-axis, the Y-axis, and the Z-axis are orthogonal to each other. The Z-axis represents the up-down direction, which is equivalent to the height direction of the cold storage 2. The X-axis and the Y-axis are parallel to the horizontal direction. The X-axis represents the left-right direction. The left-right direction is equivalent to the width direction of the cold storage 2. The Y-axis represents the front-back direction. The positive direction of the X-axis represents the right direction. The positive direction of the Y-axis represents the front direction. The positive direction of the Z-axis represents the upward direction.

[0063] Refrigerator 2 includes a main body 20, a door 21A, and a drawer 26A.

[0064] The main box body 20 is a box-shaped component that forms the main body of the cold storage 2. The main box body 20 includes: an outer box 202 that is open on the front surface; an inner box 203 that is housed inside the outer box 202 and is open on the front surface; and a heat-insulating material that fills the space between the outer box 202 and the inner box 203. The outer box 202 is formed of, for example, a metal such as iron or a hard resin such as ABS. The upper surface of the outer box 202, that is, the top surface 20A of the main box 20, is formed of metal. The inner box 203 is formed of, for example, a hard resin such as ABS. The heat-insulating material is, for example, hard polyurethane foam. In addition, the heat-insulating material is not limited to this, as long as it is a material that has heat-insulating properties and can be filled in the space between the outer box 202 and the inner box 203.

[0065] The inner space of the inner box 203 is divided into a plurality of spaces by a partition member. In this embodiment, the inner space of the inner box 203 is divided into five spaces, which function as a refrigerating chamber 21, an ice making chamber 22, a fresh freezing chamber 23, a freezing chamber 24, and a vegetable chamber 25, respectively.

[0066] The right door 21B is rotatably supported by a right door hinge member 204. The right door hinge member 204 is a hinge member provided between the top surface 20A of the main housing 20 and the right door 21B and the drawer 23A. The right door hinge member 204 disposed between the right door 21B and the drawer 23A is Fig.12 The left door 21C is rotatably supported by a left door hinge member 205. The left door hinge member 205 is a hinge member provided between the top surface 20A of the main housing 20, the right door 21B, and the drawer 22A.

[0067] Second wireless communication module 206 is provided on right door hinge member 204. Second wireless communication module 206 is a module that receives control data for controlling refrigerator 2 from an external device or transmits predetermined data to the external device.

[0068] [1-1-3. Configuration of the imaging device]

[0069] Reference Figure 3-Figure 7 The imaging device 3 will be described.

[0070] exist Figure 3-Figure 7 The diagram and Figure 2 The X-axis, Y-axis, and Z-axis shown are the same as the X-axis, Y-axis, and Z-axis. The X-axis direction corresponds to the width direction of the imaging device 3 and the left-right direction of the imaging device 3. The Y-axis direction corresponds to the front-back direction of the imaging device 3. The Z-axis direction corresponds to the up-down direction of the imaging device 3.

[0071] Figure 3 It is a side view of the imaging device 3 attached to the refrigerator 2 as seen from the right. Figure 4 This is a perspective view of the restriction member 34 in contact with the front end edge 20B of the main housing 20 as viewed from the left front.

[0072] like Figure 3 and Figure 4 As shown in FIG. 1 , the camera device 3 includes a main body 30 disposed on the top surface 20A of the main housing 20 , and a photographing member 31 extending forward from the main body 30 . The photographing member 31 is disposed on the upper front portion of the main body 30 .

[0073] The main body 30 includes a power button 32 on the front surface for turning on and off the power of the imaging device 3. The power button 32 is provided with a first LED (Light emitting diode) 33 for notifying predetermined information.

[0074] The main body 30 includes a limiting component 34. The limiting component 34 limits the position of the shooting component 31 in the front-to-back direction of the cold storage 2 by abutting against the front end edge 20B of the main body 20. The limiting component 34 is a plate-shaped component extending downward from the bottom surface 30A of the main body 30. The limiting component 34 extends along the width direction of the main body 30 (the width direction of the camera device 3) on the front surface of the main body 30, and has a mark 35 at the center of the width direction of the main body 30. The mark 35 indicates the center in the width direction of the camera device 3. The length of the limiting component 34 extending from the bottom surface 30A is formed to be shorter than the length from the top surface 20A of the main body 20 to the door gasket provided on the upper inner surface of the door 21A and substantially parallel to the top surface 20A.

[0075] Figure 5 This is a front view of refrigerator 2 and imaging device 3 attached to refrigerator 2 as seen from the front. Figure 5 Door 21A of refrigerator 2 is shown in an open state. Figure 6 It is a side view which looked at the refrigerator 2 and the imaging device 3 attached to the refrigerator 2 from the right. Figure 6 Drawer 25A of refrigerator 2 is shown in an open state, that is, in a pulled-out state. Figure 7 It is a top view of imaging device 3 attached to refrigerator 2 as seen from above.

[0076] The camera device 3 includes a wide-angle camera 301 and a narrow-angle camera 302. The wide-angle camera 301 and the narrow-angle camera 302 are provided at the front end portion of the bottom surface 31A of the imaging member 31. When the camera device 3 is installed in the cold storage 2 and the door 21A is closed, the wide-angle camera 301 and the narrow-angle camera 302 are located in front of the door 21A.

[0077] Wide-angle camera 301 is a camera having a wider angle than narrow-angle camera 302. Wide-angle camera 301 captures refrigerating room 21 from the upper front of refrigerator 2 in a state where imaging device 3 is attached to refrigerator 2.

[0078] For example, when viewed from the front of the cold storage 2, the shooting range of the wide-angle camera 301 includes Figure 5 The range A1 is a range including the door shelves 21E of the right door 21B and the left door 21C and the opening 201 of the refrigerator 21 when the right door 21B and the left door 21C are opened to the maximum degree. In addition, the shooting range of the wide-angle camera 301 includes Figure 6 Range A3 is a range from the front end of the right door 21B and the left door 21C in the open state to the front end of the shelf 21F in the front-rear direction, and includes the opening 201 of the refrigerating chamber 21 in the up-down direction.

[0079] The wide-angle camera 301 corresponds to an example of the “camera” of the present invention.

[0080] like Figure 5-Figure 7 As shown, the wide-angle camera 301 constitutes a part of the camera system 7 in Embodiment 1. The camera system 7 in Embodiment 1 includes the wide-angle camera 301, the subject PS, the state storage unit 364, and the generation unit 356. In other words, the camera system 7 includes the cold storage 2 including the subject PS, and the camera device 3 including the wide-angle camera 301, the state storage unit 364, and the generation unit 356.

[0081] Reference Figure 8 The subject PS, the state storage unit 364 , and the generation unit 356 are described.

[0082] Narrow-angle camera 302 is a camera having a narrower angle than wide-angle camera 301. Narrow-angle camera 302 captures an image of drawer 26A from the upper front of refrigerator 2 in a state where imaging device 3 is attached to refrigerator 2.

[0083] For example, when viewed from the front of the cold storage 2, the shooting range of the narrow-angle camera 302 includes Figure 5 The range A2 shown. When viewed from the front of the refrigerator 2, the range A2 includes the drawer 26A. In addition, for example, when viewed from the right of the refrigerator 2, the shooting range of the narrow-angle camera 302 includes Figure 6 The range A4 is a range including the drawer 26A in the front-rear direction that is pulled out first. In addition, the imaging range of the narrow-angle camera 302 may also include a range including the drawers provided in the refrigerator compartment 21.

[0084] The imaging device 3 includes a first optical sensor 303 and a second optical sensor 304 . The first optical sensor 303 and the second optical sensor 304 are provided on the main body 30 .

[0085] The first optical sensor 303 is a sensor for detecting the opening and closing of the right door 21B. The first optical sensor 303 is a reflective optical sensor. The first optical sensor 303 projects light to a predetermined portion of the right door 21B when the imaging device 3 is mounted on the refrigerator 2. Then, the first optical sensor 303 outputs a signal indicating the amount of light received.

[0086] The second optical sensor 304 is a sensor for detecting the opening and closing of the left door 21C. The second optical sensor 304 is a reflective optical sensor. The second optical sensor 304 projects light to a predetermined portion of the left door 21C when the imaging device 3 is mounted on the refrigerator 2. Then, the second optical sensor 304 outputs a detection value indicating the amount of received light.

[0087] The imaging device 3 includes a thermopile 305. The thermopile 305 is a sensor that detects the opening and closing of the drawer 26A. The thermopile 305 is provided at the front end of the bottom surface 31A of the imaging member 31. The thermopile 305 outputs a detection value indicating the detected temperature.

[0088] The imaging device 3 includes a human sensor 306. The human sensor 306 may be an infrared sensor or an ultrasonic sensor. The human sensor 306 outputs a detection value. The human sensor 306 is provided at the front end of the bottom surface 31A of the imaging member 31.

[0089] The imaging device 3 includes an illuminance sensor 307. The illuminance sensor 307 is a sensor for adjusting the exposure of the wide-angle camera 301 and the narrow-angle camera 302. The illuminance sensor 307 outputs a detection value. The illuminance sensor 307 is provided at the front end of the imaging unit 31.

[0090] The imaging device 3 includes a second LED 308 and a third LED 309. The second LED 308 is provided at the right front end of the bottom surface 31A of the imaging member 31. The third LED 309 is provided at the left front end of the bottom surface 31A of the imaging member 31. The second LED 308 and the third LED 309 illuminate the refrigerator 2 on which the imaging device 3 is mounted from above.

[0091] like Figure 7 As shown, the imaging device 3 includes a first control substrate 311 and a second control substrate 312 .

[0092] The first control substrate 311 is a substrate for controlling various sensors included in the camera device 3 and the power supply of the camera device 3. The first control substrate 311 is provided inside the camera device 3. The power button 32, the first LED 33, the first light sensor 303, the second light sensor 304, the thermocouple 305, the human sensor 306, and the illuminance sensor 307 are connected to the first control substrate 311. And the first control substrate 311 controls these devices. The first control substrate 311 is provided behind the second control substrate 312 and the first wireless communication module 313.

[0093] The second control substrate 312 is a substrate that controls the wide-angle camera 301 and the narrow-angle camera 302, and also controls the first wireless communication module 313. The second control substrate 312 is provided inside the camera device 3. The wide-angle camera 301, the narrow-angle camera 302, and the first wireless communication module 313 are connected to the second control substrate 312. And the second control substrate 312 controls these devices. The second control substrate 312 is provided side by side with the first control substrate 311 in front of the first control substrate 311.

[0094] The first wireless communication module 313 is a device including hardware such as an antenna or a wireless communication circuit. The first wireless communication module 313 is provided inside the imaging component 31, to the left of the second control substrate 312. The first wireless communication module 313 is provided in the imaging device 3 so as to be located in front of the opening 201 of the main body 20 when the imaging device 3 is installed in the cold storage 2. More specifically, the first wireless communication module 313 is provided in front of the limiting component 34 in the imaging device 3.

[0095] like Figure 7 As shown, in a state where the imaging device 3 is mounted in the refrigerator 2, a metal plate 314 is provided between the first wireless communication module 313 and the second wireless communication module 206 in the left-right direction.

[0096] Figure 8 It is a block diagram showing the structure of the imaging device 3.

[0097] like Figure 8 As shown, the first control substrate 311 includes a first processor 320 such as a CPU (Central Processing Unit), a first memory 330, a first communication module 340, and an interface circuit to which other devices and sensors are connected.

[0098] The first memory 330 is a storage device that stores programs and data executed by the first processor 320. The first memory 330 is composed of a non-volatile storage device such as a ROM (Read Only Memory). The first memory 330 may also include a volatile storage device such as a RAM (Random Access Memory) that constitutes a working area of ​​the first processor 320. The first memory 330 stores data processed by the first processor 320 and a control program 331 executed by the first processor 320.

[0099] The first communication module 340 includes hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the second control substrate 312 under the control of the first processor 320 .

[0100] The first processor 320 reads and executes the control program 331 , thereby turning the power of the imaging device 3 on and off in accordance with the operation of the power button 32 .

[0101] Furthermore, the first processor 320 reads out and executes the control program 331 to turn on and off the first LED 33 , the second LED 308 , and the third LED 309 .

[0102] Furthermore, the first processor 320 reads out and executes the control program 331 , thereby transmitting the detection values ​​output by each sensor connected to the first control board 311 to the second control board 312 via the first communication module 340 .

[0103] The second control substrate 312 includes a second processor 350 such as a CPU, a second memory 360, a second communication module 370, and an interface circuit for connecting to other devices or sensors.

[0104] The second memory 360 is a storage device that stores programs and data executed by the second processor 350. The second memory 360 is composed of a non-volatile storage device such as a ROM. In addition, the second memory 360 may also include a volatile storage device such as a RAM that constitutes a working area of ​​the second processor 350. The second memory 360 stores data processed by the second processor 350, a control program 361 executed by the second processor 350, and an imaging device ID (Identification) 362.

[0105] The imaging device ID 362 is information for uniquely identifying the imaging device 3 .

[0106] The second communication module 370 includes hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the first control substrate 311 under the control of the second processor 350 .

[0107] The second processor 350 reads out and executes the control program 361 stored in the second memory 360, thereby functioning as the first communication control unit 351, the second communication control unit 352, the opening / closing determination unit 353, the imaging control unit 354, the recording control unit 355, and the generating unit 356. In addition, the second processor 350 reads out and executes the control program 361 stored in the second memory 360, thereby causing the second memory 360 to function as a state storage unit 364.

[0108] The first communication control unit 351 communicates with the article management server 5 via the first wireless communication module 313 .

[0109] The second communication control unit 352 communicates with the first control board 311 via the second communication module 370. The second communication control unit 352 receives detection values ​​of various sensors connected to the first control board 311 from the first control board 311.

[0110] The opening / closing determination unit 353 determines whether the right door 21B is in the open state or the closed state based on the detection value of the first light sensor 303 received by the second communication control unit 352. When the second communication control unit 352 receives a detection value different from the detection value corresponding to the closed state, the opening / closing determination unit 353 determines that the right door 21B is in the open state. On the other hand, when the second communication control unit 352 receives a detection value corresponding to the closed state, the opening / closing determination unit 353 determines that the right door 21B is in the closed state.

[0111] Similar to the determination regarding the right door 21B, the opening / closing determination unit 353 determines whether the left door 21C is in the open state or the closed state based on the detection value of the second optical sensor 304 received by the second communication control unit 352 .

[0112] The opening and closing judgment unit 353 judges whether the drawer 26A is in an open state or a closed state based on the detection value of the thermopile 305 received by the second communication control unit 352. When the second communication control unit 352 receives the detection value of the thermopile 305 indicating a temperature below the specified temperature, the opening and closing judgment unit 353 judges that the drawer 26A is in an open state. On the other hand, when the second communication control unit 352 receives the detection value of the thermopile 305 indicating a temperature exceeding the specified temperature, the opening and closing judgment unit 353 judges that the drawer 26A is in a closed state.

[0113] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state, the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start imaging. In addition, when the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start imaging, the second communication control unit 352 sends lighting request information to the first control substrate 311. The lighting request information is information requesting the lighting of the second LED 308 and the third LED 309. When the first processor 320 receives the lighting request information, it lights the second LED 308 and the third LED 309, and notifies the user P that imaging has started.

[0114] When the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are in the closed state, the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging. In addition, when the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging, the second communication control unit 352 sends an extinguishing request information to the first control substrate 311. The extinguishing request information is information requesting the extinguishing of the second LED 308 and the third LED 309. When the first processor 320 receives the extinguishing request information, it turns off the second LED 308 and the third LED 309, and notifies the user P that imaging is ended.

[0115] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state, the recording control unit 355 starts recording the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302. That is, the recording control unit 355 starts recording the image capturing data 363 in the second memory 360. The image capturing data 363 includes the first image capturing data DV1 representing the image capturing results of the wide-angle camera 301 and the second image capturing data DV2 representing the image capturing results of the narrow-angle camera 302.

[0116] When the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are in the closed state, the recording control unit 355 ends the recording of the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302 .

[0117] The state storage unit 364 pre-stores a first shooting state CD1 and a second shooting state CD2. The first shooting state CD1 indicates a shooting state of the subject PS captured by the wide-angle camera 301 at a specified opening and closing angle of the door 21A. When the opening and closing angle is zero degrees, it indicates that the door 21A is in a closed state. The larger the opening and closing angle, the more the door 21A is opened. The specified opening and closing angle is, for example, within the range of 60 to 100 degrees. The specified opening and closing angle is, for example, 90 degrees. In the following description, the state in which the door 21A is at a specified opening and closing angle (for example, 60 to 100 degrees) is described as the "first reference state".

[0118] The first shooting state CD1 includes, for example, at least one of the position, size, and shape of the image corresponding to the subject PS in the shot image PA1 shot by the wide-angle camera 301 in the first reference state. In the first embodiment, the first shooting state CD1 includes, for example, the size of the image corresponding to the subject PS in the shot image PA1 shot by the wide-angle camera 301 in the first reference state.

[0119] The first imaging state CD1 corresponds to an example of “an imaging state of an object captured by a camera at a predetermined opening and closing angle of the opening and closing body” of the present invention.

[0120] The subject PS is disposed near the opening of the refrigerating room 21, namely the door 21A, and the image captured by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A.

[0121] Reference Figures 9 to 14 The subject PS will be further described.

[0122] The second shooting state CD2 represents the shooting state of the subject at the specified withdrawal amount of the drawer 26A photographed by the narrow-angle camera 302. When the withdrawal amount is zero, it indicates that the drawer 26A is in a closed state. The larger the specified withdrawal amount is, the more the drawer 26A is opened. A withdrawal amount of 100% indicates that the drawer 26A is in a fully open state. The specified withdrawal amount is, for example, within the range of 70% to 100%. The specified withdrawal amount is, for example, 80%. In the following description, the state in which the drawer 26A is at the specified withdrawal amount (for example, 70% to 100%) is recorded as the "second reference state".

[0123] The second shooting state CD2 includes, for example, at least one of the position, size, and shape of the image corresponding to the subject in the shot image PA2 shot by the narrow-angle camera 302 in the second reference state. In the first embodiment and the second embodiment, the second shooting state CD2 includes, for example, the size of the image corresponding to the subject in the shot image PA2 shot by the narrow-angle camera 302 in the second reference state.

[0124] The subject is disposed, for example, at the upper end of the drawer 26A, and the photographing state of the subject by the narrow-angle camera 302 changes according to the amount of withdrawal of the drawer 26A.

[0125] Based on the first imaging state CD1, the generator 356 selects the first imaging image P1 from the imaging images PB1 included in the moving image represented by the first recording data DV1 of the recording data 363. The first imaging image P1 is an imaging image when the door 21A is in the open state.

[0126] The captured image PB1 corresponds to an example of “an image captured by a camera” in the present invention.

[0127] The first captured image P1 corresponds to an example of “a captured image when the opening and closing body is in the open state” of the present invention.

[0128] Furthermore, based on the second imaging state CD2, the generator 356 selects the second imaging image P2 from the imaging image PB2 included in the moving image represented by the second video data DV2 of the video data 363. The second imaging image P2 is an imaging image when the drawer 26A is in the open state.

[0129] Furthermore, the generator 356 generates management data D1 based on the first captured image P1 and the second captured image P2. The management data D1 is data uploaded to the article management server 5 and is data for managing articles stored in the cold storage 2. The first communication control unit 351 transmits the management data D1 generated by the generator 356 to the article management server 5.

[0130] The generating unit 356 corresponds to an example of the “selecting unit” of the present invention.

[0131] In this embodiment, the case where the imaging device 3 includes the generating unit 356 is described, but the present invention is not limited to this. For example, the article management server 5 may include the generating unit 356. In this case, the management data D1 includes the first recorded data DV1 and the second recorded data DV2.

[0132] Furthermore, in the first embodiment, the generation unit 356 selects the first captured image P1 from the captured images PB1 included in the moving image represented by the first recorded data DV1 of the recorded data 363 based on the first captured state CD1, but the present invention is not limited thereto. The generation unit 356 may select the captured image PB1 in which the user P is least reflected as the first captured image P1 from the captured images PB1 included in the moving image represented by the first recorded data DV1 of the recorded data 363 based on the first captured state CD1.

[0133] In this case, the generation unit 356 further selects the first captured image P1 from the first captured images P1 selected based on the first capturing state CD1 as follows: That is, the generation unit 356 recognizes the image of the user P by a method such as pattern matching, and selects the captured image PB1 having the smallest area of ​​the recognized image of the user P as the first captured image P1.

[0134] Similarly, in the first embodiment, the generation unit 356 selects the second captured image P2 from the captured images PB2 included in the dynamic image represented by the second video data DV2 of the video data 363 based on the second capturing state CD2, but the present invention is not limited thereto. The generation unit 356 may select the captured image PB2 in which the user P is least reflected as the second captured image P2 from the captured images PB2 included in the dynamic image represented by the second video data DV2 of the video data 363 based on the second capturing state CD2.

[0135] [1-1-4. Structure of the subject]

[0136] Next, refer to Fig. 9 An example of the structure of the subject PS will be described. Fig. 9 1 is a diagram showing the structure of a fourth embodiment of the subject PS in the first embodiment. In the following description, the subject PS of the fourth embodiment is referred to as subject PS4. Subject PS4 is composed of a connecting member connecting the left end of the right door 21B and the right end of the opening of the refrigerating chamber 21. With respect to subject PS4, the greater the opening and closing angle of the right door 21B, the greater the length LD in the left-right direction.

[0137] In this case, as an example of the first shooting state CD1, the state storage unit 364 stores the length LD in the shot image PA1 shot by the wide-angle camera 301 when the opening and closing angle is set to 60 degrees to 100 degrees. In addition, the generation unit 356 selects the first shot image P1 when the right door 21B is in the open state from the shot image PB1 included in the dynamic image represented by the first video data DV1 of the video data 363 based on the length LD in the shot image PA1 stored in the state storage unit 364.

[0138] Furthermore, the generating unit 356 may select the first captured image P1 when the right door 21B is in the open state from the captured image PB1 included in the dynamic image represented by the first video data DV1 of the video data 363, based on the area of ​​the subject PS4 in the captured image PA1 stored in the state storage unit 364. In this case, the state storage unit 364 may store the area of ​​the subject PS4 in the captured image PA1 captured by the wide-angle camera 301 when the opening and closing angle is set to 60 to 100 degrees as an example of the first captured state CD1.

[0139] exist Fig. 9 In the figure, the case where the subject PS4 is composed of a connecting component connecting the left end of the right door 21B and the right end of the opening of the refrigerator room 21 is described, but the subject PS4 can also be composed of a connecting component connecting the right end of the left door 21C and the left end of the opening of the refrigerator room 21.

[0140] In addition, Fig. 9 , Fig.10 , Fig.12 ,and Fig.14 The diagram and Figure 2 The X-axis, Y-axis, and Z-axis shown are the same X-axis, Y-axis, and Z-axis.

[0141] Next, refer to Figure 10-Figure 16 The structure of the subject PS will be described. In the following description, a case where the subject PS is disposed near the opening of the refrigerating chamber 21 or the right door 21B and is photographed by the wide-angle camera 301 will be described.

[0142] In the following description, a case will be described in which the state storage unit 364 stores the first captured object image MK1 as the first captured state CD1.

[0143] In addition, refer to Figure 15-16 The first captured subject image MK1 will be described.

[0144] The first subject image MK1 corresponds to an example of the “subject image” of the present invention.

[0145] Figure 10-11This is a diagram showing the configuration of a first example of the subject PS in Embodiment 1. In the following description, the subject PS of the first example is referred to as subject PS1. Fig.10 The left figure is a front view of the cold storage 2 and the camera device 3 as viewed from the front. Figure 5 The upper part corresponds to .

[0146] Fig.10 The right picture is Fig.10 The enlarged view of the area AR1 in the left figure. Fig.10 As shown in the right figure of FIG. 1 , the photographed object PS1 is arranged on the door shelf 21E, and the door shelf 21E is provided at the uppermost part of the right door 21B. Figure 10-11 In the description, the case where the subject PS1 is arranged on the door shelf 21E of the right door 21B is described, but the subject PS1 may be arranged on the door shelf 21E of the left door 21C.

[0147] The photographed object PS1 is constituted as a rectangular flat plate member AT3, for example. The flat plate member AT3 is fixed to the upper end of the side member 21S of the door shelf 21E via the connecting member AT2 and the mounting member AT1. The side member 21S is the side of the door shelf 21E that is close to the opening of the refrigerator compartment 21 (at the Fig.10 The flat plate member AT3, the connecting member AT2, and the mounting member AT1 are integrally formed as the photographed member AT. The photographed member AT is formed of, for example, resin.

[0148] The connecting member AT2 and the mounting member AT1 correspond to an example of the “fixing member” of the present invention.

[0149] The imaging plane PF1 is a plane of the subject PS1 that is imaged by the wide-angle camera 301. The imaging plane PF1 is fixed to the side member 21S of the door frame 21E in such a manner that the right door 21B is substantially orthogonal to the imaging direction DR of the wide-angle camera 301 at a predetermined opening and closing angle (e.g., 90 degrees). In other words, the normal line of the wide-angle camera 301 at the center of the imaging plane PF1, i.e., the normal line of the wide-angle camera 301 at the center of the imaging plane PF1, is Fig.10 The subject PS1 is arranged on the right door 21B so as to be on the one-dot chain line in the shooting direction DR.

[0150] like Fig.10 As shown in the figure, since the subject PS1 is arranged on the door shelf 21E provided at the top of the right door 21B, the distance between the wide-angle camera 301 and the subject PS1 can be shortened compared to the case where the subject PS1 is arranged on another door shelf 21E. As a result, the wide-angle camera 301 can appropriately capture the first captured object image MK1 formed on the capturing surface PF1 of the subject PS1 in the first reference state.

[0151] In addition, if Fig.10 As shown in FIG. 1 , since the subject PS1 is disposed on the side member 21S of the door shelf 21E of the right door 21B, the distance between the wide-angle camera 301 and the subject PS1 can be shortened, for example, compared with the case where the subject PS1 is disposed at other positions of the door shelf 21E. For example, the other side of the door shelf 21E (at the Fig.10 As a result, the wide-angle camera 301 can appropriately capture the first captured object image MK1 formed on the capturing plane PF1 of the subject PS1 in the first reference state.

[0152] In addition, if Fig.10 As shown, since the photographed object PS1 is arranged at the upper end of the side member 21S of the door shelf 21E of the right door 21B, it is possible to suppress the wide-angle camera 301 and the photographed object PS1 from being blocked by items stored in the cold storage 2. As a result, the wide-angle camera 301 can appropriately capture the first photographed object image MK1 formed on the photographing surface PF1 of the photographed object PS1 in the first reference state.

[0153] Fig.11 is a stereogram of the subject PS1. Fig.11 As shown by the middle dotted line, a first photographic target image MK1 including identification information indicating the type of articles stored in the refrigerating chamber 21 is formed on the photographing plane PF1 of the subject PS1. For example, a sealing material on which the first photographic target image MK1 is formed is attached to the photographing plane PF1.

[0154] The state storage unit 364 stores the first object image MK1. The first object image MK1 corresponds to an example of the first imaging state CD1.

[0155] The generation unit 356 selects the first captured image P1 when the right door 21B is in the open state from the captured image PB1 included in the dynamic image represented by the first video data DV1 of the video data 363, based on the first captured image MK1 stored in the state storage unit 364. In addition, the generation unit 356 may select the first captured image P1 when the right door 21B is in the open state from the captured image PB2 included in the dynamic image represented by the second video data DV2 of the video data 363, based on the first captured image MK1 stored in the state storage unit 364.

[0156] Reference Figure 15-16 The first captured subject image MK1 will be further described.

[0157] In addition, if Fig.11As shown, the mounting part AT1 includes a first part AT11, a second part AT12, and an opening and closing part AT13. The opening and closing part AT13 is composed of a spring and the like, and opens and closes the first part AT11 and the second part AT12. That is, the first part AT11 and the second part AT12 are set to an open state, inserted into the side part 21S of the door shelf 21E, and the first part AT11 and the second part AT12 are set to a closed state, thereby fixing the subject PS1 to the side part 21S of the door shelf 21E.

[0158] Figure 12-13 This is a diagram showing the configuration of a second example of the subject PS in Embodiment 1. In the following description, the subject PS of the second example is referred to as subject PS2.

[0159] like Fig.12 As shown in FIG. 1 , the photographed object PS2 is constructed as a hinged cover AS. Fig.12 The left figure shows the state where the hinge cover AS is not installed. Fig.12 The right figure shows the state where the hinge cover AS is installed. Fig.12 In the left figure, since the hinge cover AS is not installed, the right door hinge member 204 is exposed. The right door hinge member 204 rotatably supports the right door 21B.

[0160] The right door hinge member 204 corresponds to an example of a “hinge member”.

[0161] The hinge cover AS is formed in an L shape when viewed from above, and a shooting surface PF2 is formed on the upper surface. A first shooting target image MK1 including identification information indicating the type of articles stored in the refrigerating chamber 21 is formed on the shooting surface PF2. For example, a sealing member formed with the first shooting target image MK1 is attached to the shooting surface PF2.

[0162] Reference Figure 15-16 The first captured subject image MK1 will be further described.

[0163] Fig.13 is a three-dimensional diagram of the hinged cover AS. Fig.13 As shown, a notch AS1 is formed on the hinge cover AS. The notch AS1 is fitted into the upper part of the frame of the drawer 23A. The hinge cover AS is formed of, for example, resin. In addition, the hinge cover AS may be the same color as the main body 20 or the hinge, or may be a different color.

[0164] In addition, the state storage unit 364 and the generation unit 356 perform and refer to Figure 10-11 The processing is the same as described.

[0165] Fig.141 is a diagram showing the structure of the third embodiment of the subject PS in the first embodiment. In the following description, the subject PS of the third embodiment is described as subject PS31, subject PS32, and subject PS33. Subject PS31, subject PS32, and subject PS33 correspond to an example of the subject PS of the third embodiment, respectively. Fig.14 As shown, subject PS31 , subject PS32 , and subject PS33 are respectively arranged near the opening of refrigerating chamber 21 .

[0166] The photographed object PS31 is arranged on the lower right side surface of the inner surface of the refrigerating chamber 21. In other words, the photographed object PS31 is constituted by a part of the lower right side surface of the inner surface of the refrigerating chamber 21. The photographing surface PF31 is formed integrally with the photographed object PS31. The first photographing object image MK1 including identification information indicating the type of the article stored in the refrigerating chamber 21 is formed on the photographing surface PF31. For example, a sealing member formed with the first photographing object image MK1 is attached to the photographing surface PF31.

[0167] In addition, the photographed object PS32 is arranged at the right end of the lower surface of the inner surface of the refrigerating chamber 21. In other words, the photographed object PS32 is formed by a part of the right end of the lower surface of the inner surface of the refrigerating chamber 21. The photographing surface PF32 is formed integrally with the photographed object PS32. The first photographed object image MK1 including identification information indicating the type of the article stored in the refrigerating chamber 21 is formed on the photographing surface PF32. For example, a sealing member formed with the first photographed object image MK1 is attached to the photographing surface PF32.

[0168] In addition, the photographed object PS33 is arranged at the right end of the upper surface of the front surface member in the drawer 23A of the fresh freezing chamber 23 arranged below the refrigerator chamber 21. In other words, the photographed object PS33 is composed of a part of the right end of the upper surface of the front surface member in the drawer 23A of the fresh freezing chamber 23 arranged below the refrigerator chamber 21. The photographing surface PF33 is integrally formed with the photographed object PS33. The first photographing object image MK1 including identification information indicating the type of the article stored in the refrigerator chamber 21 is formed on the photographing surface PF33. For example, a sealing member on which the first photographing object image MK1 is formed is attached to the photographing surface PF33.

[0169] In addition, the state storage unit 364 and the generation unit 356 perform and refer to Figure 10-11 The processing is the same as described.

[0170] exist Fig.14In the description, the subject PS31 is arranged on the lower right side of the inner surface of the refrigerating chamber 21, but the subject PS31 may be arranged on the lower left side of the inner surface of the refrigerating chamber 21. Fig.14 In the description, the case where the subject PS32 is arranged at the right end of the lower surface of the inner surface of the refrigerating chamber 21 is described, but the subject PS32 may be arranged at the left end of the lower surface of the inner surface of the refrigerating chamber 21. Fig.14 In the above description, the subject PS33 is described as being disposed at the right end of the upper surface of the front surface member in the drawer 23A of the fresh freezing chamber 23 disposed below the refrigerator chamber 21, but the present invention is not limited thereto. The subject PS33 may also be disposed at the left end of the upper surface of the front surface member in the drawer 22A of the ice making chamber 22 disposed below the refrigerator chamber 21.

[0171] Next, refer to Figure 15-16 The first captured subject image MK1 will be described. Fig.15 This is a diagram showing a first example of the first target image MK1 of the subject PS in the first embodiment.

[0172] like Fig.15 As shown, the first photographic subject image MK1 includes a rectangular outer frame image MF and an identification image MD formed inside the outer frame image MF. The identification image MD corresponds to identification information indicating the type of articles stored in the refrigerating chamber 21 of the refrigerator 2. The outer frame image MF is formed in black, for example.

[0173] The identification image MD is composed of a first rectangular image PD1 of a first color and a second rectangular image PD2 of a second color different from the first color, and the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 represents identification information. The first color is, for example, white, and the second color is, for example, black.

[0174] The size of the first rectangular image PD1 is the same as that of the second rectangular image PD2, and the first rectangular image PD1 and the second rectangular image PD2 are, for example, squares. In other words, the width WP1 of the first rectangular image PD1 and the width WP2 of the second rectangular image PD2 are the same as the height HP of the first rectangular image PD1 and the second rectangular image PD2.

[0175] The thickness HF in the short side direction (up and down direction) of the outer frame image MF and the thickness WF in the long side direction (left and right direction) of the outer frame image MF are respectively the same as, for example, the width WP1 of the first rectangular image PD1, the width WP2 of the second rectangular image PD2, the height HP of the first rectangular image PD1 and the height HP of the second rectangular image PD2.

[0176] The thickness HF in the short-side direction (vertical direction) of the outer-frame image MF and the thickness WF in the long-side direction (horizontal direction) of the outer-frame image MF are determined based on the resolution of the wide-angle camera 301 and the distance between the wide-angle camera 301 and the first captured object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B. In addition, the thickness HF in the short-side direction of the outer-frame image MF and the thickness WF in the long-side direction of the outer-frame image MF are determined based on the capturing range or viewing angle of the wide-angle camera 301.

[0177] For example, the greater the resolution of the wide-angle camera 301, the smaller the thickness HF in the long-side direction of the outer-frame image MF and the thickness WF in the short-side direction of the outer-frame image MF can be. In addition, the longer the distance between the wide-angle camera 301 and the first captured object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B, the greater the thickness HF in the long-side direction of the outer-frame image MF and the thickness WF in the short-side direction of the outer-frame image MF need to be. In addition, the greater the shooting range or viewing angle of the wide-angle camera 301, the greater the thickness HF in the long-side direction of the outer-frame image MF and the thickness WF in the short-side direction of the outer-frame image MF need to be.

[0178] exist Fig.15 In the first captured object image MK1 shown, the thickness HF in the long-side direction of the outer-frame image MF and the thickness WF in the short-side direction of the outer-frame image MF are each 6 mm, for example.

[0179] The first rectangular image PD1 and the second rectangular image PD2 are arranged along the longitudinal direction (here, the left-right direction) of the outer-frame image MF. Fig.15 In FIG. 1 , five first rectangular images PD1 and four second rectangular images PD2 are arranged as identification image MD. The first rectangular image PD1 represents, for example, "0", and the second rectangular image PD2 represents, for example, "1". That is, identification image MD represents a 9-digit binary number.

[0180] The identification image MD is composed of an information image MDA and a symbol image MDB. The information image MDA is composed of five rectangular images PD, and the symbol image MDB is composed of four rectangular images PD. The rectangular image PD is the first rectangular image PD1 or the second rectangular image PD2. The information image MDA corresponds to the identification information indicating the type of articles stored in the cold storage room 21 of the cold storage 2. The symbol image MDB indicates a symbol for error correction of the identification information shown in the information image MDA.

[0181] The information image MDA is composed of five rectangular images PD, so it represents a 5-digit binary number. That is, the information image MDA represents 2 to the fifth power, that is, 32 types of identification information. For example, Fig.15The information image MDA shown represents a five-digit binary number "01010".

[0182] The size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the resolution of the wide-angle camera 301 and the distance between the wide-angle camera 301 and the first captured object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B. In addition, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the capturing range or viewing angle of the wide-angle camera 301.

[0183] For example, the larger the resolution of the wide-angle camera 301, the smaller the size of the first rectangular image PD1 and the size of the second rectangular image PD2. In addition, the longer the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle (e.g., 90 degrees) of the right door 21B, the larger the size of the first rectangular image PD1 and the size of the second rectangular image PD2 need to be. In addition, the larger the photographing range or viewing angle of the wide-angle camera 301, the larger the size of the first rectangular image PD1 and the size of the second rectangular image PD2 need to be.

[0184] exist Fig.15 In the first captured object image MK1 shown, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are squares with a side of 6 mm.

[0185] Fig.16 This is a diagram showing a second example of the first subject image MK1 of the subject PS in the first embodiment. Fig.16 The first photographed object image MK1 shown in FIG. 1 has two rows of rectangular images PD in the vertical direction of the identification image MD, which is different from the Fig.15 The first object image MK1 shown is different.

[0186] The information image MDA is composed of ten rectangular images PD, and the symbol image MDB is composed of eight rectangular images PD. The four rectangular images PD in the upper layer of the eight rectangular images PD constituting the symbol image MDB represent symbols for error correction of the identification information shown in the five rectangular images PD in the upper layer of the ten rectangular images PD constituting the information image MDA. In addition, the four rectangular images PD in the lower layer of the eight rectangular images PD constituting the symbol image MDB represent symbols for error correction of the identification information shown in the five rectangular images PD in the lower layer of the ten rectangular images PD constituting the information image MDA.

[0187] exist Fig.16In the above description, the identification image MD has two rows of rectangular images PD in the vertical direction, but the present invention is not limited thereto. The identification image MD may have three or more rows of rectangular images PD in the vertical direction.

[0188] exist Figure 15-16 In the description, the case where the outer frame image MF is formed in black is described, but the present invention is not limited to this. For example, the outer frame image MF may be formed in blue. In addition, the case where the first color is white and the second color is black is described, but the present invention is not limited to this. For example, the first color may be yellow and the second color may be blue.

[0189] Reference Figure 10-Figure 14 The subject PS1, subject PS2, subject PS31, subject PS32, and subject PS33 described above each include the first photographic object image MK1. Figure 15-16 As shown, the first photographic subject image MK1 includes a rectangular outer-frame image MF and an identification image MD formed inside the outer-frame image MF. Thus, the generating unit 356 executes the following processing.

[0190] That is, the generation unit 356 first determines whether the outer frame image MF can be detected from each of the captured images PB1 included in the dynamic image represented by the first recorded data DV1 of the recorded data 363. Then, when the outer frame image MF can be detected from the captured image PB1, the generation unit 356 determines whether the identification information represented by the identification image MD inside the detected outer frame image MF can be read. Next, when the identification information represented by the identification image MD can be read and the read identification information is consistent with the identification information included in the first captured object image MK1 stored in the state storage unit 364, the generation unit 356 selects the captured image as the first captured image P1.

[0191] In addition, for example, when the outer-frame image MF can be detected from each of the captured images PB1 included in the moving image represented by the first video data DV1, the generation unit 356 notifies the user P that the wide-angle camera 301 has captured the subject PS by lighting, blinking, or making a sound. In addition, for example, when the outer-frame image MF can be detected from each of the captured images PB2 included in the moving image represented by the second video data DV2, the generation unit 356 may also notify the user P that the narrow-angle camera 302 has captured the subject PS by lighting, blinking, or making a sound.

[0192] The case where the outer-frame image MF can be detected from each of the captured images PB1 corresponds to an example of “a case where it is determined that the camera has captured the subject”.

[0193] In order for the generator 356 to easily detect the outer frame image MF from each of the captured images PB1 included in the moving image represented by the first video data DV1, the color of the outer frame image MF is preferably different from the color of the inner side surface of the refrigerating chamber 21 and the color of the items stored in the refrigerating chamber 21. Therefore, the color of the outer frame image MF is preferably black, blue, or the like, for example.

[0194] In addition, a subject (not shown) is also arranged at the upper end of the drawer 26A. The subject includes a second subject image MK2 having the same shape as the reference image MK2. Figure 15-16 The structure is the same as the first captured object image MK1 described above.

[0195] Therefore, the state storage unit 364 stores the second object image MK2 as the second shooting state CD2. In addition, the generation unit 356 executes the following processing.

[0196] That is, the generation unit 356 first determines whether the outer frame image MF can be detected from each of the captured images PB2 contained in the dynamic image represented by the second video data DV2 of the video data 363. Then, when the outer frame image MF can be detected from the captured image PB2, the generation unit 356 determines whether the identification information represented by the identification image MD inside the detected outer frame image MF can be read. Next, when the identification information represented by the identification image MD can be read and the read identification information is consistent with the identification information contained in the second captured object image MK2 stored in the state storage unit 364, the generation unit 356 selects the captured image PB2 as the second captured image P2.

[0197] In addition, for example, when the outer frame image MF can be detected from each of the captured images PB2 contained in the dynamic image represented by the second video data DV2, the generation unit 356 notifies the user P that the narrow-angle camera 302 has captured the subject PS by lighting, flashing, or making a sound of an LED.

[0198] The case where the outer-frame image MF can be detected from each of the captured images PB2 corresponds to an example of “a case where it is determined that the camera has captured the subject”.

[0199] [1-1-5. Structure of the item management server]

[0200] Fig.17 It is a block diagram showing the configuration of the terminal device 4 and the item management server 5.

[0201] The article management server 5 includes a server processor 50 which is a processor such as a CPU, a server memory 51, and a server communication module 52. The server processor 50 is connected to the server memory 51 and the server communication module 52.

[0202] The server memory 51 is a storage device that stores programs and data executed by the server processor 50. The server memory 51 is composed of a non-volatile storage device. In addition, the server memory 51 may also include a non-volatile storage device that constitutes a work area of ​​the server processor 50. The server memory 51 stores data processed by the server processor 50, a control program 511 executed by the server processor 50, and an item management DB (database) 512.

[0203] The article management DB 512 has one record for each imaging device 3. Each record includes the imaging device ID 362, the image data of the first captured image P1, and the image data of the second captured image P2.

[0204] Server communication module 52 includes a communication circuit or a connector based on a predetermined communication standard, and communicates with refrigerator 2 , imaging device 3 , and terminal device 4 via network NW.

[0205] The server processor 50 functions as a server communication control unit 501 and a server processing unit 502 by reading and executing a control program 511 stored in the server memory 51 .

[0206] The server communication control unit 501 communicates with the refrigerator 2 , the imaging device 3 , and the terminal device 4 via the server communication module 52 .

[0207] The server processing unit 502 performs various information processing using the camera 3 and the terminal device 4 as clients. The server processing unit 502 may also use the refrigerator 2 as a client.

[0208] [1-1-6. Configuration of terminal device]

[0209] Next, the structure of the terminal device 4 is described.

[0210] The terminal device 4 includes a terminal processor 40 which is a processor such as a CPU, a terminal memory 41 , a terminal communication module 42 , and a touch panel 43 .

[0211] The terminal memory 41 is a storage device that non-volatilely stores programs and data executed by the terminal processor 40. The terminal memory 41 is composed of a non-volatile storage device. In addition, the terminal memory 41 may also include a RAM that constitutes a work area of ​​the terminal processor 40. The terminal memory 41 stores data processed by the terminal processor 40, a control program executed by the terminal processor 40, and an item management application 411.

[0212] The terminal communication module 42 includes a communication circuit or a connector based on a prescribed communication standard, and communicates with the item management server 5 under the control of the terminal processor 40 .

[0213] The touch panel 43 includes a display panel and a touch sensor overlapped with or provided integrally with the display panel.

[0214] The terminal processor 40 functions as an application execution unit 400 .

[0215] The application execution unit 400 communicates with the item management server 5 via the terminal communication module 42 .

[0216] The application execution unit 400 causes the touch panel 43 to display a user interface related to the management of the items stored in the refrigerator 2. In the following description, this user interface is referred to as "application UI." The display control unit 402 of this embodiment displays images of the items stored in the refrigerator 2 and the like by displaying the application UI.

[0217] The application execution unit 400 receives various inputs from the user P via the application UI.

[0218] [1-2. Action]

[0219] Next, the operation of each unit of the article management system 1 will be described.

[0220] [1-2-1. Actions related to video recording]

[0221] First, the operation of the imaging device 3 related to recording will be described.

[0222] Fig.18 FA is a flowchart showing the operation of the imaging device 3 .

[0223] The opening / closing determination unit 353 determines whether the door 21A or the drawer 26A is in the open state (step SA1 ).

[0224] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is not in the open state (step SA1 : NO), the second processor 350 returns the process to step SA1 .

[0225] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state (step SA1 : Yes), the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start imaging (step SA2 ).

[0226] Next, the recording control unit 355 starts recording the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302 (step SA3 ).

[0227] Next, the opening / closing determination unit 353 determines whether the door 21A and the drawer 26A are closed (step SA4). If the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are not closed (step SA4: No), the determination in step SA4 is performed again.

[0228] On the other hand, when it is determined that the door 21A and the drawer 26A are closed (step SA4 : Yes), the recording control unit 355 ends the recording of the photographing result (step SA5 ). Thus, the recording data 363 is generated in the second memory 360 .

[0229] Next, the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging (step SA6 ).

[0230] [1-2-2. Operations related to transmission of management data]

[0231] Next, the operation related to the transmission of the management data D1 will be described.

[0232] Fig.19 This is a flowchart showing the operation of the imaging device 3 and the article management server 5 .

[0233] exist Fig.19 In the flowchart FB, the operation of the imaging device 3 is shown, and the flowchart FC shows the operation of the article management server 5 is shown.

[0234] The generation unit 356 determines whether the recorded data 363 is stored in the second memory 360 (step SB1). When the generation unit 356 determines that the recorded data 363 is not stored (step SB1: No), the determination of step SB1 is performed again.

[0235] On the other hand, when the generation unit 356 determines that the video data 363 is stored (step SB1: yes), based on the first shooting state CD1 stored in the state storage unit 364, the first shot image P1 is selected from the shot image PB1 contained in the dynamic image represented by the first video data DV1 of the video data 363 stored in the second memory 360 (step SB2).

[0236] Step SB2 will be described in detail.

[0237] The first captured image P1 is a captured image when the right door 21B or the left door 21C is opened at a predetermined opening / closing angle (for example, 90 degrees).

[0238] The generation unit 356 first determines whether the outer-frame image MF can be detected from each of the captured images PB1 included in the dynamic image represented by the first recorded data DV1 of the recorded data 363. Then, if the outer-frame image MF can be detected from the captured image PB1, the generation unit 356 determines whether the identification information represented by the identification image MD inside the detected outer-frame image MF can be read. Next, if the identification information represented by the identification image MD can be read and the read identification information is consistent with the identification information included in the first captured object image MK1 stored in the state storage unit 364, the generation unit 356 selects the captured image PB1 as the first captured image P1.

[0239] Returning to the description of the flowchart FB, the generation unit 356 selects a second captured image P2 from each of the captured images PB2 included in the moving image represented by the second recorded data DV2 of the stored recorded data 363 (step SB3 ).

[0240] Step SB3 will be described in detail.

[0241] The second captured image P2 is a captured image when the drawer 26A is in the open state.

[0242] The generation unit 356 first determines whether the outer frame image MF can be detected from each of the captured images PB2 included in the dynamic image represented by the second video data DV2 of the video data 363. Then, when the outer frame image MF can be detected from the captured image PB2, the generation unit 356 determines whether the identification information represented by the identification image MD inside the detected outer frame image MF can be read. Next, when the identification information represented by the identification image MD can be read and the read identification information is consistent with the identification information included in the second captured object image MK2 stored in the state storage unit 364, the generation unit 356 selects the captured image PB2 as the second captured image P2.

[0243] The generation unit 356 determines whether or not at least one of the first captured image P1 and the second captured image P2 can be selected (step SB4 ).

[0244] When the generation unit 356 determines that the first captured image P1 and the second captured image P2 cannot be selected (step SB4 : No), it performs the process of step SB7 .

[0245] On the other hand, when the generating unit 356 determines that at least one of the first captured image P1 and the second captured image P2 can be selected (step SB4 : Yes), it generates the management data D1 (step SB5 ).

[0246] When the first captured image P1 and the second captured image P2 are selectable, the generating unit 356 generates management data D1 including the image data of the first captured image P1 , the image data of the second captured image P2 , and the imaging device ID 362 .

[0247] When the first captured image P1 can be selected but the second captured image P2 cannot be selected, the generating unit 356 generates management data D1 including the image data of the first captured image P1 and the imaging device ID 362 .

[0248] When the second captured image P2 can be selected but the first captured image P1 cannot be selected, the generating unit 356 generates the management data D1 including the image data of the second captured image P2 and the imaging device ID 362 .

[0249] Next, the first communication control unit 351 transmits the management data D1 generated by the generating unit 356 to the article management server 5 (step SB6).

[0250] Next, the generating unit 356 deletes the recorded data 363 to be processed from the second memory 360 after step SB2 (step SB7 ).

[0251] As shown in flowchart FC, the server communication control unit 501 determines whether the management data D1 is received from the camera device 3 (step SC1). When the server communication control unit 501 determines that the management data D1 is received (step SC1: Yes), the server processing unit 502 updates the item management DB 512 based on the management data D1 received by the server communication control unit 501 (step SC2).

[0252] Step SC2 will be described in detail.

[0253] The server processing unit 502 identifies the record including the camera ID 362 included in the management data D1 from the article management DB 512, and updates the image data of the first captured image P1 in the identified record to the image data of the first captured image P1 included in the management data D1.

[0254] The server processing unit 502 identifies the record including the camera ID 362 included in the management data D1 from the article management DB 512, and updates the image data of the second captured image P2 in the identified record to the image data of the second captured image P2 included in the management data D1.

[0255] [1-2-3. Actions related to application UI]

[0256] Next, the operation of the item management system 1 related to the application UI will be described.

[0257] Fig. 20 4 is a flowchart showing the operation of the terminal device 4 and the article management server 5. Fig. 20 Flowchart FD represents the operation of the terminal device 4, and flowchart FE represents the operation of the item management server 5.

[0258] As shown in flowchart FD, the application execution unit 400 determines whether to request image data stored in the article management DB 512 from the article management server 5 (step SD1). For example, when the article management application 411 is activated, the terminal communication control unit 401 makes an affirmative determination in step SD1.

[0259] When the application execution unit 400 determines that the image data stored in the article management DB 512 is requested (step SD1: Yes), it sends request data for requesting the image data to the article management server 5 (step SD2). The request data includes the camera ID 362 stored in the terminal memory 41. In addition, the terminal device 4 obtains the camera ID 362 at a predetermined time.

[0260] As shown in flowchart FE, the server communication control unit 501 receives the request data (step SE1). Next, the server processing unit 502 obtains the image data corresponding to the camera ID 362 included in the request data received by the server communication control unit 501 from the item management DB 512 (step SE2). Next, the server communication control unit 501 sends the image data obtained by the server processing unit 502 to the terminal device 4 as a response to the request data (step SE3).

[0261] Referring to flowchart FD, the application execution unit 400 receives image data from the item management server 5 (step SD3). Next, the application execution unit 400 causes the touch panel 43 to display an application UI including an image indicated by the image data received in step SD3 (step SD4).

[0262] [1-3. Effects, etc.]

[0263] Above, as reference Figure 1-Figure 20As described, the camera system 7 in embodiment 1 includes: a wide-angle camera 301, which captures a cold storage 2 having a cold storage room 21 and a door 21A for opening and closing the opening 201 of the cold storage room 21, and is arranged on the top surface 20A of the cold storage 2; a subject PS, which is arranged near the opening 201 of the cold storage room 21 or the door 21A, and the shooting state captured by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A; a state storage unit 364, which stores the shooting state of the subject PS captured by the wide-angle camera 301 at a specified opening and closing angle of the door 21A, namely, a first shooting state CD1; and a generation unit 356, which selects, based on the first shooting state CD1, a shooting image when the door 21A is in an open state from the shooting image PB1 shot by the wide-angle camera 301.

[0264] Accordingly, based on the first photographing state CD1 of the photographed object PS photographed by the wide-angle camera 301 at the predetermined opening and closing angle of the door 21A, the first photographing image P1 when the door 21A is in the open state is selected from the photographing image PB1 photographed by the wide-angle camera 301. Therefore, by appropriately setting the predetermined opening and closing angle, it is possible to select a photographing image that appropriately photographs the internal conditions of the cold storage 2 as the first photographing image P1 when the door 21A is in the open state. The predetermined opening and closing angle is, for example, 60 to 100 degrees.

[0265] In the imaging system 7 , the subject PS1 is arranged on the door 21A so that the imaging plane PF1 of the subject PS1 is substantially orthogonal to the imaging direction of the wide-angle camera 301 when the door 21A is opened or closed at the predetermined angle.

[0266] Accordingly, when the door 21A is opened or closed at the specified angle, the imaging plane PF1 of the subject PS1 is substantially orthogonal to the imaging direction of the wide-angle camera 301 . Therefore, when the door 21A is opened or closed at the specified angle, the subject PS1 can be appropriately imaged by the wide-angle camera 301 .

[0267] In addition, in the imaging system 7 , the subject PS1 is disposed on the door frame 21E of the door 21A, and the imaging system 7 includes a connecting member AT2 and an attaching member AT1 for fixing the subject PS1 to the door frame 21E.

[0268] According to this, since the object PS1 is fixed to the door frame 21E, the shooting state of the wide-angle camera 301 changes according to the opening and closing angle of the door 21A. Therefore, the object PS1 can be arranged at an appropriate position.

[0269] In addition, in the imaging system 7 , a right door hinge member 204 that rotatably supports the right door 21B is provided, and the subject PS2 is configured as a hinge cover AS of the right door hinge member 204 .

[0270] According to this, since the object PS2 is formed as the hinge cover AS of the right door hinge member 204, the photographing state by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A. Therefore, the object PS2 can be arranged at an appropriate position.

[0271] In the imaging system 7, a notch AS1 is formed in the hinge cover AS, and the notch AS1 is fitted into the upper portion of the frame of the drawer 23A at the lower portion of the right door.

[0272] Thus, since the notch AS1 formed in the hinge cover AS is fitted into the upper portion of the frame of the drawer 23A below the right door, the hinge cover AS can be fixed to the right door hinge member 204. Therefore, the subject PS2 can be arranged at an appropriate position.

[0273] In addition, in the camera system 7, a first object image MK1 including identification information indicating the type of items stored in the cold storage room 21 and at least one of the cold storage room 21 storing the items is formed on the shooting surface PF1 of the subject PS1, the state storage unit 364 stores the first object image MK1, and the generation unit 356 selects the first image P1 when the door 21A is in the open state from the shooting image PB1 shot by the wide-angle camera 301 based on the first object image MK1 stored in the state storage unit 364.

[0274] According to this, by selecting, for example, the captured image PB1 in which the first captured subject image MK1 is appropriately captured as the first captured image P1 from among the captured images PB1 captured by the wide-angle camera 301 , it is possible to appropriately select the first captured image P1 .

[0275] In the camera system 7 , the first captured object image MK1 includes a rectangular outer frame image MF and an identification image MD formed inside the outer frame image MF and corresponding to identification information indicating the type of articles stored in the refrigerating chamber 21 of the cold storage 2 .

[0276] According to this, by selecting, from among the captured images PB1 captured by the wide-angle camera 301 , the captured image PB1 in which the outer-frame image MF and the identification image MD are captured appropriately as the first captured image P1 , the first captured image P1 can be appropriately selected.

[0277] In the imaging system 7 , the identification image MD is composed of a first rectangular image PD1 of a first color and a second rectangular image PD2 of a second color different from the first color, and the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 represents the identification information.

[0278] According to this, by appropriately setting the first color and the second color, the identification information represented by the arrangement pattern of the first rectangular image PD1 and the second rectangular image PD2 can be accurately detected.

[0279] In the imaging system 7 , the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are the same, and the first rectangular image PD1 and the second rectangular image PD2 are arranged along the long side direction of the outer-frame image MF.

[0280] Accordingly, since the size of the first rectangular image PD1 is the same as the size of the second rectangular image PD2, the identification information can be effectively represented by the pattern of the arrangement of the first rectangular image PD1 and the second rectangular image PD2. In addition, since the first rectangular image PD1 and the second rectangular image PD2 are arranged along the long side direction of the outer frame image MF, the first rectangular image PD1 and the second rectangular image PD2 can be compactly arranged.

[0281] In the camera system 7, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 are determined based on the resolution of the wide-angle camera 301 and the distance between the wide-angle camera 301 and the first captured object image MK1 at a specified opening and closing angle of the door 21A.

[0282] Accordingly, for example, the higher the resolution of the wide-angle camera 301, the smaller the size of the first rectangular image PD1 and the second rectangular image PD2. Also, for example, the longer the distance between the wide-angle camera 301 and the first photographic object image MK1 at a predetermined opening and closing angle of the door 21A, the larger the size of the first rectangular image PD1 and the second rectangular image PD2. In this way, the size of the first rectangular image PD1 and the size of the second rectangular image PD2 can be appropriately determined.

[0283] In addition, in the camera system 7, when it is possible to detect the outer frame image MF contained in the captured image PB1 captured by the wide-angle camera 301 and the identification information represented by the identification image MD of the first captured object image MK1 contained in the captured image PB1 captured by the wide-angle camera 301 is consistent with the identification information represented by the identification image MD of the first captured object image MK1 stored in the status storage unit 364, the generation unit 356 selects the captured image PB1 as the first captured image P1.

[0284] Accordingly, when the outer frame image MF included in the captured image PB1 captured by the wide-angle camera 301 can be detected, the captured image PB1 is an appropriately captured image. In addition, when the identification information represented by the identification image MD of the first captured object image MK1 included in the captured image PB1 is consistent with the identification information represented by the identification image MD of the first captured object image MK1 stored in the state storage unit 364, the captured image PB1 is an image captured more appropriately. Therefore, the first captured image P1 can be easily and appropriately selected.

[0285] In the imaging system 7 , when it is determined that the wide-angle camera 301 or the narrow-angle camera 302 has captured the subject PS, the generating unit 356 notifies the user P that the wide-angle camera 301 or the narrow-angle camera 302 has captured the subject PS.

[0286] Thus, since the user P is informed that the wide-angle camera 301 or the narrow-angle camera 302 has captured the subject PS, the user P can easily recognize that the wide-angle camera 301 or the narrow-angle camera 302 has captured the subject PS. Therefore, the convenience of the user P can be improved.

[0287] In addition, the camera device 3 in embodiment 1 includes: a wide-angle camera 301, which captures a cold storage 2 having a cold storage room 21 and a door 21A for opening and closing the opening 201 of the cold storage room 21, and is arranged on the top surface 20A of the cold storage 2; a state storage unit 364, which stores a shooting state of the subject PS1 captured by the wide-angle camera 301 at a specified opening and closing angle of the door 21A, namely, a first shooting state CD1; and a generation unit 356, which selects a shooting image when the door 21A is in an open state, namely, a first shooting image P1, from a shooting image PB1 captured by the wide-angle camera 301 based on the first shooting state CD1, wherein the subject PS1 is arranged near the opening 201 of the cold storage room 21 or the door 21A, and the shooting state captured by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A.

[0288] According to this, the imaging device 3 in the first embodiment achieves the same effect as the imaging system 7 in the first embodiment.

[0289] (Implementation Method 2)

[0290] Next, Embodiment 2 will be described. In the description of Embodiment 2, the same components as those of the respective components of the article management system 1 of Embodiment 1 are denoted by the same reference numerals, and detailed descriptions thereof will be omitted as appropriate.

[0291] [2-1. Structure]

[0292] Fig.21This is a diagram showing the configuration of an article management system 1A in the second embodiment.

[0293] The article management system 1A in the second embodiment includes a cold storage 2A, an article management server 5, and a terminal device 4. The cold storage 2 and the camera 3 in the first embodiment described above are separate bodies, but the cold storage 2A in the second embodiment includes the camera 3A. That is, the cold storage 2A and the camera 3A in the present embodiment are not separate bodies.

[0294] Fig. 22 It is a block diagram showing the configuration of refrigerator 2A in the present embodiment.

[0295] Refrigerator 2A includes third control board 220. Third control board 220 includes third processor 230 such as a CPU, third memory 240, third communication module 250, and an interface circuit for connecting to other devices or sensors.

[0296] The third memory 240 is a storage device that stores programs and data executed by the third processor 230. The third memory 240 is composed of a non-volatile storage device. In addition, the third memory 240 may also include a volatile storage device that constitutes a working area of ​​the third processor 230. The third memory 240 stores data processed by the third processor 230 and a control program 241 executed by the third processor 230.

[0297] The third communication module 250 includes communication hardware such as a communication circuit or a connector in accordance with a predetermined wired communication standard, and communicates with the first control substrate 311A ​​under the control of the third processor 230 .

[0298] The second wireless communication module 206 , the cooling unit 207 , the first opening / closing sensor 208 , the second opening / closing sensor 209 , the third opening / closing sensor 210 , the fourth opening / closing sensor 211 , the fifth opening / closing sensor 212 , and the sixth opening / closing sensor 213 are connected to the third control substrate 220 .

[0299] The cooling unit 207 includes a compressor or a condenser, a capillary tube, a cooler, a cooling fan for conveying cold air generated by the cooler to each storage room, and a damper for diverting the cold air conveyed by the cooling fan, etc., which are mechanisms for cooling each storage room of the cold storage 2A. The cooling unit 207 cools each storage room of the cold storage 2A under the control of the third control board 220.

[0300] The first opening / closing sensor 208 to the sixth opening / closing sensor 213 are constituted by, for example, a Hall IC.

[0301] The first opening / closing sensor 208 is a sensor for detecting the opening / closing of the right door 21B.

[0302] The second opening / closing sensor 209 is a sensor for detecting the opening / closing of the left door 21C.

[0303] The third opening / closing sensor 210 is a sensor for detecting the opening / closing of the drawer 22A.

[0304] The fourth opening / closing sensor 211 is a sensor for detecting the opening / closing of the drawer 23A.

[0305] The fifth opening / closing sensor 212 is a sensor for detecting the opening / closing of the drawer 24A.

[0306] The sixth opening / closing sensor 213 is a sensor for detecting the opening / closing of the drawer 25A.

[0307] Each opening and closing sensor outputs a detection value to the third control substrate 220 .

[0308] The third processor 230 reads out and executes the control program 241 to communicate with the external device via the second wireless communication module 206 .

[0309] Furthermore, third processor 230 reads out and executes control program 241 to cool each storage room of cold storage 2A using cooling unit 207 .

[0310] Furthermore, the third processor 230 reads out and executes the control program 241 , thereby transmitting the detection values ​​outputted from the first opening / closing sensor 208 to the sixth opening / closing sensor 213 to the first control substrate 311A.

[0311] Refrigerator 2A includes imaging device 3A. Compared with imaging device 3, imaging device 3A does not include first optical sensor 303, second optical sensor 304, thermopile 305, human detection sensor 306, second LED 308, and third LED 309.

[0312] In addition, the subject PS is arranged near the opening of the refrigerating chamber 21 of the refrigerator 2A, the right door 21B, or the left door 21C, and the shooting state by the wide-angle camera 301 changes according to the opening and closing angles of the right door 21B and the left door 21C.

[0313] In addition, the subject PS constitutes a part of the refrigerator 2A of the present invention.

[0314] The camera device 3A of the second embodiment includes a first control substrate 311A ​​instead of the first control substrate 311. The power button 32, the first LED 33, and the illuminance sensor 307 are connected to the first control substrate 311A. In addition, the camera device 3A of the second embodiment includes a second control substrate 312A instead of the second control substrate 312. The wide-angle camera 301, the narrow-angle camera 302, and the first wireless communication module 313 are connected to the second control substrate 312A.

[0315] The first control substrate 311A ​​is provided in the imaging device 3A at the same position as the first control substrate 311 in the imaging device 3 .

[0316] In addition, the second control substrate 312A is provided in the imaging device 3A at the same position as the installation position of the second control substrate 312 in the imaging device 3 .

[0317] In this embodiment, the wide-angle camera 301 , the narrow-angle camera 302 , the power button 32 , the first LED 33 , the illuminance sensor 307 , and the first wireless communication module 313 are disposed at the same positions as those in the first embodiment.

[0318] The first control substrate 311A ​​includes a first processor 320A such as a CPU, a first memory 330A, a first communication module 340A, and an interface circuit for connecting to other devices or sensors.

[0319] The first memory 330A is the same memory as the first memory 330. The first memory 330A stores data processed by the first processor 320A and a control program 331A executed by the first processor 320A.

[0320] The first communication module 340A includes communication hardware such as a communication circuit or a connector in compliance with a predetermined wired communication standard, and communicates with the second control board 312A and the third control board 220 via a cable.

[0321] The first processor 320A reads out and executes the control program 331A, thereby turning the power of the imaging device 3A on and off in accordance with the operation of the power button 32 .

[0322] The first processor 320A turns on and off the first LED 33 by reading and executing the control program 331A.

[0323] The first processor 320A reads out and executes the control program 331A, thereby transmitting the detection value output by the illuminance sensor 307 to the second control board 312A via the first communication module 340A.

[0324] The first processor 320A reads out and executes the control program 331A, thereby transmitting the detection values ​​of the first to sixth opening / closing sensors 208 to 213 received from the third control board 220 to the second control board 312A via the first communication module 340A.

[0325] The second control substrate 312A includes a second processor 350A such as a CPU, a second memory 360A, a second communication module 370A, and an interface circuit for connecting to other devices or sensors.

[0326] The second memory 360A is the same memory as the first memory 330. The second memory 360A stores data processed by the second processor 350A and a control program 361A executed by the second processor 350A.

[0327] The second communication module 370A includes communication hardware such as a communication circuit or a connector in compliance with a predetermined communication standard, and communicates with the first control substrate 311A ​​under the control of the second processor 350A.

[0328] The second processor 350A reads and executes the control program 361A stored in the second memory 360A, thereby functioning as the first communication control unit 351, the second communication control unit 352A, the opening / closing determination unit 353A, the imaging control unit 354, the video recording control unit 355, and the generating unit 356. In addition, the second processor 350A reads and executes the control program 361A stored in the second memory 360A, thereby causing the second memory 360A to function as the state storage unit 364.

[0329] Refrigerator 2A in Embodiment 2 includes wide-angle camera 301 , subject PS, state storage unit 364 , and generation unit 356 .

[0330] The second communication control unit 352A communicates with the first control board 311A ​​via the second communication module 370A. The second communication control unit 352A receives detection values ​​of various sensors from the first control board 311A.

[0331] The opening and closing determination unit 353A determines whether the right door 21B is in the open state or the closed state based on the detection value of the first opening and closing sensor 208 received by the second communication control unit 352A. When the second communication control unit 352A receives a detection value different from the detection value corresponding to the closed state, the opening and closing determination unit 353 determines that the right door 21B is in the open state. On the other hand, when the second communication control unit 352 receives a detection value corresponding to the closed state, the opening and closing determination unit 353 determines that the right door 21B is in the closed state.

[0332] Similar to the determination regarding the right door 21B, the opening / closing determination unit 353A determines whether the left door 21C is in the open state or the closed state based on the detection value of the second opening / closing sensor 209 received by the second communication control unit 352 .

[0333] The opening / closing determination unit 353A determines whether the drawer 26A is in the open or closed state based on the detection values ​​of the third opening / closing sensor 210 to the sixth opening / closing sensor 213 received by the second communication control unit 352. The opening / closing determination unit 353A makes the same determination as that of the right door 21B.

[0334] [2-2. Action]

[0335] The operation of each unit of the article management system 1A in the second embodiment is the same as the operation of each unit of the article management system 1 in the first embodiment. That is, the operation of the camera 3A related to recording is the same as the operation of the camera 3 in the first embodiment. In addition, the operation of the camera 3A related to the management data D1 is the same as the operation of the camera 3 in the first embodiment.

[0336] [2-3. Effects, etc.]

[0337] Above, as reference Figure 21-22 As described, the cold storage 2A in the second embodiment includes: a wide-angle camera 301, which shoots the cold storage 2 having a cold storage room 21 and a door 21A for opening and closing the opening 201 of the cold storage room 21, and is arranged on the top surface 20A of the cold storage 2; a subject PS, which is arranged near the opening 201 of the cold storage room 21 or the door 21A, and the shooting state shot by the wide-angle camera 301 changes according to the opening and closing angle of the door 21A; a state storage unit 364, which stores the shooting state of the subject PS shot by the wide-angle camera 301 at a specified opening and closing angle of the door 21A, i.e., a first shooting state CD1; and a generation unit 356, which selects, based on the first shooting state CD1, a shooting image when the door 21A is in an open state from the shooting image PB1 shot by the wide-angle camera 301.

[0338] According to this, refrigerator 2A in the second embodiment achieves the same effects as imaging system 7 in the first embodiment.

[0339] (Other Embodiments)

[0340] As described above, the above-mentioned embodiments 1 and 2 are described as examples disclosed in this application. However, the technology in the present invention is not limited to this, and can also be applied to embodiments that have been changed, replaced, added, omitted, etc. In addition, the various components described in the above-mentioned embodiments 1 and 2 can also be combined to form a new embodiment. Therefore, other embodiments are exemplified below.

[0341] In the above-mentioned first and second embodiments, the case where the subject PS1 is arranged on the door shelf 21E of the right door 21B is described, but the subject PS1 may be arranged on the door shelf 21E of the left door 21C.

[0342] In the above-mentioned first and second embodiments, the case where the subject PS1 is arranged on the door shelf 21E provided at the top of the door 21A is described, but the subject PS1 may be arranged on the door shelf 21E of the door 21A. For example, the subject PS1 may be arranged on the second or third door shelf 21E provided from the top of the door 21A.

[0343] In the above-mentioned embodiments 1 and 2, the case where the subject PS1 is fixed to the side member 21S of the door shelf 21E of the door 21A is described, but the subject PS1 can be arranged on the door shelf 21E of the door 21A. For example, the subject PS1 may be fixed to the front surface member. Figure 5 In the embodiment, the front surface member is a member on the positive side of the Y axis of the door frame 21E, that is, on the front surface side. In addition, for example, the subject PS1 may be fixed to the bottom surface member. Figure 5 In the figure, the bottom surface member is a member on the negative side of the Z axis of the door shelf 21E, that is, on the lower side.

[0344] In the above-mentioned first and second embodiments, the identification image MD is composed of the information image MDA and the symbol image MDB, but the identification image MD may be composed of only the information image MDA. In other words, the identification image MD may not include the symbol image MDB.

[0345] In the above-mentioned first and second embodiments, the case where the first rectangular image PD1 and the second rectangular image PD2 are squares has been described, but the first rectangular image PD1 and the second rectangular image PD2 may be rectangular.

[0346] In the above-mentioned embodiments 1 and 2, the case where the outer frame image MF is formed in black is described, but the present invention is not limited to this. For example, the outer frame image MF may be formed in blue. In addition, the case where the first color is white and the second color is black is described, but the present invention is not limited to this. For example, the first color may be yellow and the second color may be blue.

[0347] In order to enable the generator 356 to easily detect the outer frame image MF from each of the captured images PB1 included in the moving image represented by the first video data DV1, the color of the outer frame image MF is preferably different from the color of the inner side surface of the refrigerating room 21 and the color of the items stored in the refrigerating room 21. Therefore, the color of the outer frame image MF is preferably black, blue, etc., for example.

[0348] In the above-mentioned second embodiment, the first optical sensor 303, the second optical sensor 304, the thermopile 305, the human sensor 306, the second LED 308, and the third LED 309 are not included. In other embodiments of the second embodiment, any of these devices may be included.

[0349] In the above-mentioned Embodiments 1 and 2, the right door 21B and the left door 21C have different widths. In other embodiments, the right door 21B and the left door 21C may have the same width, or the left door 21C may have a width greater than the right door 21B.

[0350] In the above-mentioned Embodiments 1 and 2, the imaging device 3 and 3A is configured to include the second LED 308 and the third LED 309. In other embodiments, the second LED 308 and the third LED 309 may be configured by a single LED.

[0351] In the above-mentioned second embodiment, the case where the first opening and closing sensor 208 to the sixth opening and closing sensor 213 are constituted by a Hall IC is exemplified. However, the first opening and closing sensor 208 and the second opening and closing sensor 209 may also be other sensors such as an angle sensor, a camera, and a temperature sensor. In addition, the third opening and closing sensor 210 to the sixth opening and closing sensor 213 may also be other sensors such as an acceleration sensor, a mechanical switch, an angle sensor, a camera, and a temperature sensor.

[0352] In other embodiments, the camera device 3 may also include a subject detection unit. In this case, the video control unit 355 may also control the start and end of the video recording based on the fact that the subject detection unit can detect the subject. This can reduce the capacity of the video data. In addition, the processing speed can be increased.

[0353] The subject detection unit corresponds to an example of a “selection unit”.

[0354] The operation of the imaging device 3 related to recording of a subject will be described. Fig.23 It is a flowchart FF showing the operation of the imaging device 3.

[0355] The opening / closing determination unit 353 determines whether the door 21A or the drawer 26A is in the open state (step SF1 ).

[0356] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is not in the open state (step SF1 : NO), the second processor 350 returns the process to step SF1 .

[0357] When the opening / closing determination unit 353 determines that the door 21A or the drawer 26A is in the open state (step SF1 : Yes), the imaging control unit 354 causes the wide-angle camera and the narrow-angle camera 302 to start imaging (step SF2 ).

[0358] Next, the subject detection unit determines whether the subject can be detected (step SF3 ).

[0359] When determining that the subject detection unit cannot detect the subject (step SF3 : NO), the second processor 350 advances the process to step SF7 .

[0360] When it is determined that the subject detection unit can detect the subject (step SF3: Yes), the recording control unit 355 starts recording the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 (step SF4). In addition, based on the subject corresponding to the wide-angle camera 301, only the recording of the shooting results of the wide-angle camera 301 may be started. Similarly, based on the subject corresponding to the narrow-angle camera 302, only the recording of the shooting results of the narrow-angle camera 302 may be started. In addition, when it is determined that the subject detection unit can detect the subject, the subject detection unit may also notify the user that the subject can be detected (the recording of the shooting results has been started). Accordingly, since the user is notified that the subject is detected (the recording of the shooting results has been started), the user can easily recognize that the recording of the shooting results has been started. Therefore, the noise of shooting is reduced, and the convenience of the user can be improved.

[0361] After the recording control unit 355 starts recording the image capturing results of the wide-angle camera 301 and the narrow-angle camera 302 , it determines whether the imaged object detection unit has detected the object (step SF5 ).

[0362] When it is determined that the subject detection unit can detect the subject (step SF5 : Yes), the determination in step SF5 is performed again.

[0363] On the other hand, when it is determined that the subject detection unit cannot detect the subject (step SF5: No), the video control unit 355 ends the video recording of the shooting result (step SF6). As a result, the video data 363 is generated in the second memory 360. In addition, the video data 363 may also include a plurality of video data. For example, a plurality of video data may be generated in the second memory 360 through a single shooting. In this case, the generation unit 356 may also perform the processing of steps SB1 to SB7 on the plurality of video data respectively. In addition, when it is determined that the subject detection unit cannot detect the subject, the user P may be notified that the subject detection unit cannot detect the subject (the video recording of the shooting result has been ended). According to this, since the user P is notified that the subject cannot be detected (the video recording of the shooting result has been ended), the user P can easily recognize that the video recording of the shooting result has been ended. Therefore, the convenience of the user P can be improved.

[0364] Next, the opening / closing determination unit 353 determines whether the door 21A and the drawer 26A are closed (step SF7). If the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are not closed (step SF7: No), the second processor 350 returns the process to step SF3.

[0365] On the other hand, when the opening / closing determination unit 353 determines that the door 21A and the drawer 26A are closed (step SF7 : Yes), the imaging control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to end imaging (step SF8 ).

[0366] In another embodiment, the recording control unit 355 may end the recording of the image capture result after a predetermined time (for example, 1 minute) has passed since the start of the recording of the image capture result. This can also cope with a malfunction of the image capture device 3 .

[0367] In other embodiments, the generator 356 may compare the image captured by the camera with pre-stored images of the door being in an open state, and select the image of the door being in an open state.

[0368] In other embodiments, the camera device 3 may also include a camera device confirmation unit. When the user sets the camera device 3 in the cold storage 2, the camera device confirmation unit is used to confirm that the camera device 3 can be set in a specified position of the cold storage 2. In this case, the camera device confirmation unit may also notify the user that the camera device 3 can be set in a specified position of the cold storage 2 (for example, the center position in the left-right direction of the cold storage 2) by lighting up, flashing, or making a sound or notifying the terminal device 4 when the subject can be detected in a pre-stored area. Accordingly, the user can easily recognize that the camera device 3 can be set in the specified position of the cold storage 2 and can move to the next setting operation. Therefore, the convenience of the user can be improved.

[0369] In addition, the camera device confirmation unit may also notify the user that the camera device 3 cannot be set at the specified position of the cold storage 2 by lighting, flashing, or sound of an LED or notifying the terminal device 4 when the subject cannot be detected in the pre-stored area. Accordingly, the user can easily recognize that the camera device 3 cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved. In addition, the camera device confirmation unit may also make the notification method to the user different when the camera device 3 can be set at the specified position of the cold storage 2 and when the camera device 3 cannot be set at the specified position of the cold storage 2. For example, the LED may be lit when the camera device 3 can be set at the specified position of the cold storage 2, and the LED may be flashed when the camera device 3 cannot be set at the specified position of the cold storage 2. Accordingly, the user can easily recognize that the camera device 3 can be set at the specified position of the cold storage 2 and that the camera device 3 cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved.

[0370] In other embodiments, the camera device 3 may also include a subject confirmation unit. When the user sets the subject in the cold storage 2, the subject confirmation unit is used to confirm that the subject can be set in the specified position of the cold storage 2. In this case, the subject confirmation unit may also notify the user that the subject can be set in the specified position of the cold storage 2 by lighting up, flashing, or making a sound or notifying the terminal device 4 when the information of the subject (information obtained from the photographed subject) is consistent with the information of the pre-stored storage room. Accordingly, the user can easily recognize that the subject can be set in the specified position of the cold storage 2 and can move to the next setting operation. Therefore, the convenience of the user can be improved.

[0371] In addition, the subject confirmation unit may also notify the user that the subject cannot be set at the specified position of the cold storage 2 by lighting, flashing, or sound of an LED or notification to the terminal device 4 (for example, displaying the information of the subject) when the information of the subject (the information obtained from the photographed subject) is inconsistent with the information of the pre-stored storage room. Accordingly, the user can easily recognize that the subject cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved. In addition, the subject confirmation unit may also make the notification method to the user different when the subject can be set at the specified position of the cold storage 2 and when the subject cannot be set at the specified position of the cold storage 2. For example, the LED may be lit when the subject can be set at the specified position of the cold storage 2, and the LED may be flashed when the subject cannot be set at the specified position of the cold storage 2. Accordingly, the user can easily recognize that the subject can be set at the specified position of the cold storage 2 and that the subject cannot be set at the specified position of the cold storage 2. Therefore, the convenience of the user can be improved.

[0372] In other embodiments, the subject may also include characters or illustrations that provide information about the storage room to be installed (e.g., characters such as a refrigerator room or illustrations of a refrigerator room). Accordingly, when the user sets the subject in the refrigerator 2, the user can easily identify the storage room to be installed. This can prevent the user from making setting errors and improve the convenience of the user.

[0373] In other embodiments, in addition to the functional units of the second processors 350 and 350A in Embodiments 1 and 2, the second processors 350 and 350A may also function as a detection unit for detecting an object. The detection unit may also cut out images of objects entering and leaving the refrigerating chamber 21 from the dynamic image during the period from the start to the end of the recording of the wide-angle camera 301 and the narrow-angle camera 302 by the recording control unit 355. In addition, when a mark is attached to the object, the detection unit may detect the marked object.

[0374] In other embodiments, in addition to the functional units of the second processor 350 in Embodiment 1, the second processor 350 may also function as a person presence determination unit. The person presence determination unit determines whether a person exists around the cold storage 2 based on the detection value of the human sensing sensor 306. More specifically, the person presence determination unit determines that a person exists around the cold storage 2 when the detection value of the human sensing sensor 306 is a detection value corresponding to the presence of a person, and determines that no person exists around the cold storage 2 when the detection value of the human sensing sensor 306 is a detection value corresponding to the presence of a person. Otherwise, the person presence determination unit determines that no person exists around the cold storage 2.

[0375] When the second processor 350 functions as a person presence determination unit, if the person presence determination unit determines that a person exists, the shooting control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to start shooting, and if the person presence determination unit determines that a person does not exist, the shooting control unit 354 causes the wide-angle camera 301 and the narrow-angle camera 302 to stop shooting. In addition, when the second processor 350 functions as a person presence determination unit, if the person presence determination unit determines that a person exists, the recording control unit 355 starts recording, and if the person presence determination unit determines that a person does not exist, the recording control unit 355 stops recording. Thus, even if the opening and closing determination unit 353 makes an erroneous determination, the shooting results of the wide-angle camera 301 and the narrow-angle camera 302 can be properly recorded.

[0376] In other embodiments, in addition to the functional units of the second processors 350 and 350A in embodiments 1 and 2, the second processors 350 and 350A may also function as an entry and exit judgment unit. In this case, the entry and exit judgment unit changes the boundary line used for the entry and exit judgment according to the opening and closing angles of the left door 21C and the right door 21B, and judges the entry and exit of the article. In addition, the entry and exit judgment unit changes the boundary line used for the entry and exit judgment according to the degree of withdrawal of the drawer 26A, and judges the entry and exit of the article.

[0377] In other embodiments, the first captured image P1 may be subjected to image correction and uploaded to the article management server 5. As an example of image correction, correction of viewpoint conversion is exemplified.

[0378] In the above-described embodiment, the first captured image P1 and the second captured image P2 are used for display purposes of the terminal device 4. However, the first captured image P1 and the second captured image P2 may be used for other purposes such as detecting items stored in the cold storage 2 or 2A.

[0379] In other embodiments, the compartments formed in the main body 20 of the refrigerator 2, 2A may have storage rooms other than the above five types, or may be fewer. In addition, in other embodiments, the number of doors provided in the opening 201 of the refrigerating chamber 21 may be one.

[0380] The first processor 320, 320A, the second processor 350, 350A, the server processor 50, and the terminal processor 40 may be composed of a single processor or a plurality of processors. These processors may also be hardware programmed in a manner to implement corresponding functional units. That is, these processors may also be composed of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0381] Figure 8 , Fig.17 ,and Fig. 22 The structures of the camera device 3, 3A, the cold storage 2, 2A, the terminal device 4, and the article management server 5 shown are examples, and the specific installation form is not particularly limited. That is, it is not necessary to install the corresponding hardware for each part separately, and it is also possible to set the structure in which the functions of each part are realized by executing a program by a processor. In addition, part of the functions realized by software in the above-mentioned embodiment can also be set as hardware, or part of the functions realized by hardware can also be realized by software.

[0382] Fig.18 , Fig.19 , Fig. 20 ,and Fig.23 The step units of the actions shown are divided according to the main processing content for easy understanding of the actions, and the actions are not limited by the method or name of the division of the processing units. It can also be divided into more step units according to the processing content. In addition, a step unit can also be divided to include more processing. In addition, the order of the steps can also be appropriately replaced within the scope of not hindering the purpose of the present invention.

[0383] Furthermore, since the above-described embodiments are provided to illustrate the technology of the present invention, various changes, substitutions, additions, omissions, etc. may be made within the scope of the claims or their equivalents.

[0384] (Note)

[0385] The following technology is disclosed through the description of the above embodiments.

[0386] (Technology 1) A camera system, comprising: a camera, which captures a cold storage having a storage room and an opening and closing body for opening and closing the opening of the storage room, and is arranged on the upper surface of the cold storage; a subject, which is arranged near the opening of the storage room or the opening and closing body, and the capturing state captured by the camera changes according to the opening and closing angle of the opening and closing body; a state storage unit, which stores the capturing state of the subject captured by the camera at a specified opening and closing angle of the opening and closing body; and a selection unit, which selects, from the image captured by the camera, a captured image when the opening and closing body is in an open state, based on the capturing state of the subject captured by the camera stored in the state storage unit.

[0387] According to this structure, based on the photographic state of the photographed object photographed by the camera at the specified opening and closing angle of the opening and closing body, the photographic image when the opening and closing body is in the open state is selected from the image photographed by the camera. Therefore, by appropriately setting the specified opening and closing angle, it is possible to select a photographic image that appropriately photographs the internal conditions of the cold storage as the photographic image when the opening and closing body is in the open state.

[0388] (Technique 2) According to the camera system described in Technique 1, the above-mentioned object is configured on the above-mentioned opening and closing body so that when the above-mentioned opening and closing body is at the above-mentioned specified opening and closing angle, the shooting surface of the above-mentioned object is approximately orthogonal to the shooting direction of the above-mentioned camera.

[0389] According to this structure, since the photographing plane of the object is approximately orthogonal to the photographing direction of the camera when the opening and closing body is at the specified opening and closing angle, the object can be properly photographed by the camera when the opening and closing body is at the specified opening and closing angle.

[0390] (Technique 3) The imaging system according to Technique 2, wherein the subject is disposed on a door frame of the opening and closing body, and includes a fixing member that fixes the subject to the door frame.

[0391] According to this structure, since the photographed object is fixed to the door frame of the opening and closing body, the photographing state of the photographed object by the camera changes according to the opening and closing angle of the opening and closing body. Therefore, the photographed object can be arranged at an appropriate position.

[0392] (Technique 4) The imaging system according to Technique 1, wherein a hinge member is provided to rotatably support the opening and closing body, and the photographed object is configured as a hinge cover of the hinge member.

[0393] According to this configuration, since the photographed object is formed as a hinge cover of the hinge member, the photographed state of the photographed object by the camera changes according to the opening and closing angle of the opening and closing body, so that the photographed object can be arranged at an appropriate position.

[0394] (Technique 5) The camera system according to Technique 4, wherein a notch is formed in the hinge cover, and the notch is fitted into an upper portion of a frame of a drawer below the storage chamber.

[0395] According to this configuration, since the notch formed in the hinge cover is fitted into the upper portion of the frame of the drawer below the storage chamber, the hinge cover can be fixed to the hinge member. Therefore, the subject can be arranged at an appropriate position.

[0396] (Technique 6) An imaging system according to any one of Techniques 1 to 5, wherein a subject image is formed on a photographing surface of the photographed object, the subject image corresponds to identification information indicating the type of items stored in the storage room and at least one of the storage rooms storing the items, the state storage unit stores the subject image, and the selection unit selects an image captured when the opening and closing body is in an open state from images captured by the camera based on the subject image stored in the state storage unit.

[0397] According to this configuration, a captured image in which the captured object image is appropriately captured is selected from the images captured by the camera as a captured image when the opening and closing body is in the open state, thereby appropriately selecting a captured image when the opening and closing body is in the open state.

[0398] (Technique 7) An imaging system according to Technique 6, wherein the captured object image includes a rectangular outer frame image and an identification image, wherein the identification image is formed inside the outer frame image and corresponds to identification information indicating the type of items stored in the storage room in the cold storage.

[0399] According to this structure, by selecting, for example, an image that appropriately captures the outer frame image and the identification image from the images captured by the camera as a captured image when the opening and closing body is in an open state, it is possible to appropriately select an image captured when the opening and closing body is in an open state.

[0400] (Technology 8) An imaging system according to Technology 7, wherein the identification image is composed of a first rectangular image of a first color and a second rectangular image of a second color different from the first color, and a pattern of the configuration of the first rectangular image and the second rectangular image represents the identification information.

[0401] According to this configuration, by appropriately setting each of the first color and the second color, the identification information represented by the pattern of arrangement of the first rectangular image and the second rectangular image can be accurately detected.

[0402] (Technique 9) An imaging system according to Technique 8, wherein the size of the first rectangular image is the same as the size of the second rectangular image, and the first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image.

[0403] According to this configuration, since the size of the first rectangular image is the same as the size of the second rectangular image, identification information can be effectively represented by the pattern of the arrangement of the first rectangular image and the second rectangular image. In addition, since the first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image, the first rectangular image and the second rectangular image can be compactly arranged.

[0404] (Technique 10) A camera system according to Technique 8 or Technique 9, wherein the size of the first rectangular image and the size of the second rectangular image are determined based on the resolution of the camera and the distance between the camera and the image of the object at a specified opening and closing angle of the opening and closing body.

[0405] According to this configuration, for example, the higher the resolution of the camera, the smaller the size of the first rectangular image and the second rectangular image. Also, for example, the longer the distance between the camera and the photographed object image at a predetermined opening and closing angle of the opening and closing body, the larger the size of the first rectangular image and the second rectangular image. In this way, the size of the first rectangular image and the size of the second rectangular image can be appropriately determined.

[0406] (Technique 11) A camera system according to any one of Techniques 7 to 10, wherein, when the outer frame image contained in the image taken by the camera can be detected and the identification information represented by the identification image of the photographed object contained in the image taken by the camera is consistent with the identification information represented by the identification image of the photographed object stored in the state storage unit, the selection unit selects the image taken by the camera as the image taken when the opening and closing body is in the open state.

[0407] According to this structure, when the outer frame image contained in the image captured by the camera can be detected, the image captured by the camera is an image that is captured appropriately. In addition, when the identification information represented by the identification image of the captured object image contained in the image captured by the camera is consistent with the identification information represented by the identification image of the captured object image stored in the state storage unit, the image captured by the camera is an image that is captured more appropriately. Therefore, the captured image when the opening and closing body is in the open state can be easily and appropriately selected.

[0408] (Technique 12) In the imaging system according to any one of Techniques 1 to 11, when the selection unit determines that the camera has captured the subject, the selection unit notifies a user that the camera has captured the subject.

[0409] According to this configuration, when it is determined that the camera has captured the subject, the user is notified that the camera has captured the subject, so the user can easily recognize that the camera has captured the subject. Therefore, user convenience can be improved.

[0410] (Technology 13) A camera device, comprising: a camera that captures a cold storage having a storage room and an opening and closing body that opens and closes an opening of the storage room, and is arranged on an upper surface of the cold storage; a state storage unit that stores a photographic state of a subject photographed by the camera at a specified opening and closing angle of the opening and closing body; and a selection unit that selects a photographic image when the opening and closing body is in an open state from an image photographed by the camera based on the photographic state of the subject photographed by the camera stored in the state storage unit, wherein the subject is arranged near the opening of the storage room or the opening and closing body, and the photographic state photographed by the camera changes according to the opening and closing angle of the opening and closing body.

[0411] According to this structure, the same effect as the imaging system described in the technique 1 is achieved.

[0412] (Technology 14) A cold storage, comprising: a camera, which captures the cold storage having a storage room and an opening and closing body for opening and closing the opening of the storage room, and is arranged on the upper surface of the cold storage; a photographed object, which is arranged near the opening of the storage room or the opening and closing body, and the photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body; a state storage unit, which stores the photographing state of the photographed object photographed by the camera at a specified opening and closing angle of the opening and closing body; and a selection unit, which selects a photographed image when the opening and closing body is in an open state from the image photographed by the camera based on the photographing state of the photographed object photographed by the camera stored in the state storage unit.

[0413] According to this structure, the same effect as the imaging system described in the technique 1 is achieved.

[0414] Industrial Applicability

[0415] As described above, the imaging system, imaging device, and refrigerator of the present invention can be used for selecting an image that appropriately captures the internal conditions of the refrigerator from images captured by a camera.

[0416] Description of Reference Numerals

[0417] 1. 1A Item Management System

[0418] 2. 2A cold storage

[0419] 3. 3A camera device

[0420] 4 Terminal Devices

[0421] 5 Item Management Server

[0422] 7 Camera System

[0423] 21 Cold room (storage room)

[0424] 21A Door (opening and closing body)

[0425] 21B Right Door

[0426] 21C Left Door

[0427] 21E Door Shelf

[0428] 201 Opening

[0429] 20A Top surface (upper surface)

[0430] 204 right door hinge part (hinge part)

[0431] 301 wide angle camera (camera)

[0432] 302 narrow angle camera

[0433] 350, 350A second processor

[0434] 351 1st Communication Control Unit

[0435] 352, 352A Second communication control unit

[0436] 353, 353A Open / Close Judgment Unit

[0437] 354 Shooting Control Unit

[0438] 355 Video Control Department

[0439] 356 Generation Department (Selected Department)

[0440] 360, 360A 2nd Memory

[0441] 361, 361A control program

[0442] 362 Camera Device ID

[0443] 363 video data

[0444] 364 State storage unit

[0445] AS hinged cover

[0446] AS1 gap

[0447] AT Photographed Parts

[0448] AT1 Tablet Components

[0449] AT2 connecting part (part of the fixing part)

[0450] AT3 mounting parts (part of the fixing parts)

[0451] CD1 1st shooting state (the object is shot at a predetermined opening and closing angle of the opening and closing body)

[0452] The shooting status of the camera)

[0453] CD2 2nd shooting status

[0454] D1 management data

[0455] DV1 1st video data

[0456] DV2 2nd video data

[0457] MK1 1st subject image (subject image)

[0458] MK2 2nd subject image

[0459] MD recognition image

[0460] MDA infographic

[0461] MDB symbol image

[0462] PD1 1st rectangular image

[0463] PD2 2nd rectangular image

[0464] MF frame image

[0465] P1 1st captured image (image captured when the opening and closing body is in the open state)

[0466] P2 2nd captured image

[0467] PB1 captured image (image captured by the camera)

[0468] PF1 shooting surface

[0469] PS1, PS2, PS31, PS32, PS33, PS subject.

Claims

1. A camera system, characterized in that: include: a camera for photographing a cold storage having a storage room and an opening and closing body capable of opening and closing an opening of the storage room, and arranged on an upper surface of the cold storage; A subject to be photographed, which is arranged near the opening of the storage room or the opening and closing body, and the photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body; a state storage unit for storing a photographing state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and A selection unit selects a captured image when the opening and closing body is in the open state from among images captured by the camera, based on the captured state of the subject captured by the camera stored in the state storage unit.

2. The camera system according to claim 1, wherein: The object is arranged on the opening and closing body so that, when the opening and closing body is at the predetermined opening and closing angle, an imaging plane of the object is substantially orthogonal to an imaging direction of the camera.

3. The camera system according to claim 2, wherein: The photographed object is arranged on a door shelf of the opening and closing body. The camera system includes a fixing component for fixing the object to be photographed to the door shelf.

4. The camera system according to claim 1, wherein: A hinge member is provided for axially supporting the opening and closing body in a manner that the opening and closing body can rotate. The photographed object is configured as a hinge cover of the hinge member.

5. The camera system according to claim 4, characterized in that: A notch is formed on the hinge cover. The notch is fitted into an upper portion of a frame of a drawer below the storage chamber.

6. The imaging system according to any one of claims 1 to 5, characterized in that: forming a photographic subject image on a photographing surface of the object, the photographic subject image including identification information indicating at least one of a type of articles stored in the storage room and the storage room storing the articles, The state storage unit stores the subject image. The selection unit selects, based on the captured object image stored in the state storage unit, an image captured when the opening and closing body is in the open state from among the images captured by the camera.

7. The camera system according to claim 6, characterized in that: The photographed object image comprises: A rectangular frame image; and An identification image is formed inside the outer frame image and corresponds to identification information indicating the type of articles stored in the storage room in the refrigerator.

8. The camera system according to claim 7, characterized in that: The identification image is composed of a first rectangular image of a first color and a second rectangular image of a second color different from the first color. The arrangement pattern of the first rectangular image and the second rectangular image represents the identification information.

9. The camera system according to claim 8, characterized in that: The size of the first rectangular image is the same as the size of the second rectangular image. The first rectangular image and the second rectangular image are arranged along the long side direction of the outer frame image.

10. The camera system according to claim 9, characterized in that: The size of the first rectangular image and the size of the second rectangular image are determined based on the resolution of the camera and the distance between the camera and the captured object image at a predetermined opening and closing angle of the opening and closing body.

11. The camera system according to claim 7, wherein: When the outer frame image contained in the image taken by the camera can be detected and the identification information represented by the identification image of the photographed object contained in the image taken by the camera is consistent with the identification information represented by the identification image of the photographed object stored in the state storage unit, the selection unit selects the image taken by the camera as the image taken when the opening and closing body is in the open state.

12. The camera system according to claim 1, wherein: When the selection unit determines that the camera has captured the subject, the selection unit notifies a user that the camera has captured the subject.

13. A camera device, characterized in that: include: a camera for photographing a cold storage having a storage room and an opening and closing body capable of opening and closing an opening of the storage room, and arranged on an upper surface of the cold storage; A state storage unit that stores a photographing state of the subject photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and a selection unit that selects, from the images captured by the camera, a captured image when the opening and closing body is in the open state, based on the captured state of the subject captured by the camera stored in the state storage unit; The subject is arranged near the opening of the storage room or the opening and closing body, and a photographing state captured by the camera changes according to an opening and closing angle of the opening and closing body.

14. A cold storage, characterized in that: include: a camera for photographing a cold storage having a storage room and an opening and closing body capable of opening and closing an opening of the storage room, and arranged on an upper surface of the cold storage; A subject to be photographed, which is arranged near the opening of the storage room or the opening and closing body, and the photographing state photographed by the camera changes according to the opening and closing angle of the opening and closing body; a state storage unit for storing a photographing state of the photographed object photographed by the camera at a predetermined opening and closing angle of the opening and closing body; and A selection unit selects a captured image when the opening and closing body is in the open state from among images captured by the camera, based on the captured state of the subject captured by the camera stored in the state storage unit.

Images