Photographic apparatus

Abstract

Provided is a photographing device capable of high-precision photographing of articles in and out of a storage. The photographing device includes a main body provided on an upper surface of a storage, and a photographing member extending from the main body toward a front of the storage, including a camera that photographs a lower portion from a front upper portion of the storage, the main body including a position limiting member that abuts against a front end edge of the storage to limit a position of the photographing member in a front-rear direction of the storage.

Description

Technical Field

[0001] This invention relates to photographic apparatus. Background Technology

[0002] Patent document 1 discloses the following technology: a camera is installed on the upper part of a cold storage room, and a picture is taken when an object passing through the opening of the cold storage room is detected, and the food stored in the cold storage room is managed based on the picture results.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-070476 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] This invention provides a photographic device capable of capturing images of items entering and leaving a storage warehouse with high precision.

[0008] Methods for solving problems

[0009] The photographic apparatus of the present invention includes: a main body disposed on the upper surface of a storage cell; and a photographing component extending from the main body toward the front of the storage cell, including a camera that photographs a lower part of the storage cell from the front upper part of the storage cell. The main body includes a position limiting component that abuts against the front edge of the storage cell, limiting the position of the photographing component in the front-rear direction of the storage cell.

[0010] In addition, this specification includes the entire contents of Japanese Patent Application No. 2021-029014, filed on February 25, 2021.

[0011] Invention Effects

[0012] The photographic apparatus of the present invention restricts the position of the shooting component in the front-to-back direction of the storage compartment by means of a position limiting member, so the position of the camera is defined so that it can shoot from the front and above the storage compartment from below. Therefore, items entering and leaving the storage compartment can be included in the camera's shooting results with high precision, and items entering and leaving the storage compartment can be photographed with high precision. Attached Figure Description

[0013] Figure 1 This is a diagram showing the structure of the food management system in Implementation Method 1.

[0014] Figure 2 This is a side view of the camera unit installed in the cold storage, viewed from the right side, according to Embodiment 1.

[0015] Figure 3This is a perspective view of the position limiting component that abuts against the front edge of the main body, as seen from the left front side in Embodiment 1.

[0016] Figure 4 This is a front view of the cold storage and camera unit from the front side in Embodiment 1.

[0017] Figure 5 This is a top view of the cold storage and camera unit from the right side of Implementation Method 1.

[0018] Figure 6 This is a block diagram showing the structure of the camera unit in Embodiment 1.

[0019] Figure 7 This is a block diagram illustrating the structure of the terminal device and the food management server in Implementation Method 1.

[0020] Figure 8 This is a diagram illustrating an example of the cold storage management database in Implementation Method 1.

[0021] Figure 9 This is an example of an image showing the interior of a cold storage room in Embodiment 1.

[0022] Figure 10 This is a diagram illustrating an example of the drawer management database in Implementation Method 1.

[0023] Figure 11 This is an example of an adjustment screen for implementation method 1.

[0024] Figure 12 This is a diagram showing an example of the central position recording paper in Embodiment 1.

[0025] Figure 13 This is a flowchart illustrating the operation of the camera unit in Implementation Method 1.

[0026] Figure 14 This is a flowchart illustrating the operation of the food management system in Implementation Method 1.

[0027] Figure 15 This is a diagram used to illustrate the entry / exit determination image of Implementation Method 1.

[0028] Figure 16 This is an example of the main screen of the refrigerator compartment in Embodiment 1.

[0029] Figure 17 This is a diagram showing the input flow of the notes for Implementation Method 1.

[0030] Figure 18 This is an example of a storage list screen in Implementation Method 1.

[0031] Figure 19This is an example of a newly added list screen in Implementation Method 1.

[0032] Figure 20 This is a diagram illustrating the process of adding food information in Implementation Method 1.

[0033] Figure 21 This is an example of a screen showing the list of objects to be deleted in Implementation Method 1.

[0034] Figure 22 This is a flowchart illustrating the operation of the food management system in Implementation Method 1.

[0035] Figure 23 This is an example of the main screen of the vegetable room in Implementation Method 1.

[0036] Figure 24 This is an example of a food image list screen in Embodiment 1.

[0037] Figure 25 This is a flowchart illustrating the operation of the food management system in Implementation Method 1.

[0038] Figure 26 This is a block diagram illustrating the structure of the terminal device and the food management server in Embodiment 2. Detailed Implementation

[0039] (Knowledge and insights that form the basis of this invention, etc.)

[0040] When the inventors conceived of this invention, there existed techniques for photographing items entering and leaving storage rooms such as cold storage warehouses. In the prior art, sometimes a camera device separate from the storage room is used for photography. In this case, the camera device is mostly installed on the upper surface of the storage room. However, the inventors discovered that when the camera device is installed on the upper surface of the storage room, for example, by placing the camera device at the rear of the storage room, it may not be possible to accurately include the items entering and leaving the storage room in the camera's photographic results. To solve this problem, the subject of this invention is formed.

[0041] Therefore, the present invention provides a photographic device capable of capturing images of items entering and leaving a storage warehouse with high precision.

[0042] The embodiments will now be described in detail with reference to the accompanying drawings. However, sometimes the necessary detailed descriptions are omitted. For example, there may be instances where detailed descriptions of matters already known are omitted, or where substantially identical structures are described repeatedly.

[0043] Furthermore, the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the invention, and are not intended to limit the subject matter of the claims.

[0044] (Implementation Method 1)

[0045] First, implementation method 1 will be described.

[0046] [1-1. Structure]

[0047] [1-1-1. Structure of a Food Management System]

[0048] Figure 1 This is a diagram showing the structure of the food management system 1000.

[0049] The food management system 1000 is a system for managing the food stored in the cold storage 1, including the storage date and time.

[0050] Cold storage 1 corresponds to an example of the storage warehouse of the present invention. Food corresponds to an example of the article of the present invention.

[0051] The food management system 1000 includes a cold storage compartment 1. The cold storage compartment 1 includes a main body 10 with an open front surface. The main body 10 has a cold storage compartment 11, an ice-making compartment 12, a fresh-freezing compartment 13, a freezer compartment 14, and a vegetable compartment 15. A rotating left door 11A and a right door 11B are provided at the front opening of the cold storage compartment 11. Each of the ice-making compartment 12, the fresh-freezing compartment 13, the freezer compartment 14, and the vegetable compartment 15 has drawers 12A, 13A, 14A, and 15A for storing food. The drawers 14A and 15A of this invention include an upper box body JCA and a lower box body GCA, comprising two boxes CA.

[0052] The upper box JCA corresponds to an example of the first box of the present invention. The lower box GCA corresponds to an example of the second box of the present invention.

[0053] In the following description, without distinguishing between the left door 11A and the right door 11B, they are referred to as "doors" and given the symbol "11C". Furthermore, in the following description, without distinguishing between drawers 12A, 13A, 14A, and 15A, they are referred to as "drawers" and given the symbol "16A".

[0054] The food management system 1000 includes a camera unit 2. The camera unit 2 is a device for capturing images and detecting food entering and leaving the cold storage 1. The camera unit 2 is installed on the upper surface 10B of the main body 10 of the cold storage 1.

[0055] Camera unit 2 corresponds to an example of the photographic apparatus of the present invention.

[0056] The food management system 1000 includes a terminal device 3. The terminal device 3 is, for example, a smartphone or tablet. The terminal device 3 has an application installed for managing the food in the cold storage 1, and communicates with the food management server 4 through the functions of the application.

[0057] In the following description, the application will be referred to as the "Food Management Application" and given the symbol "311".

[0058] Food management application 311 is an example of the program and application of the present invention.

[0059] exist Figure 1 In the diagram, solid lines represent users P who are at home, and dashed lines represent users P who have left home (H). When terminal device 3 is used by user P who is at home, it communicates with food management server 4 via communication device 5 or without communication device 5 through the functions of food management application 311. Furthermore, when terminal device 3 is used by user P who has left home (H) and a communication connection with communication device 5 cannot be established, it communicates with food management server 4 without communication device 5 through the functions of food management application 311.

[0060] Communication device 5 connects to the global network GN, which includes the Internet, telephone network, and other communication networks, and communicates with the food management server 4 connected to the global network GN. Communication device 5 is an interface device for connecting terminal device 3 to the global network GN.

[0061] The food management system 1000 includes a food management server 4. The food management server 4 is a server device that manages the food in the cold storage 1 and is connected to the global network GN. Furthermore, while the food management server 4 is represented by a block in each diagram, this does not mean that the food management server 4 consists of a single server device. For example, the food management server 4 may also be composed of multiple server devices handling different tasks.

[0062] Next, the structure of camera unit 2, terminal device 3 and food management server 4 will be described.

[0063] [1-1-2. Structure of the Camera Unit]

[0064] Reference Figures 2-4 The camera unit 2 will be described.

[0065] exist Figures 2-4The diagram shows the X, Y, and Z axes. The X, Y, and Z axes are orthogonal to each other. The Z-axis represents the vertical direction. The X and Y axes are parallel to the horizontal direction. The X-axis represents the horizontal direction and also the width of the cold storage unit 1 and camera unit 2. The Y-axis represents the front-back direction and also the depth of the cold storage unit 1 and camera unit 2. The positive direction of the X-axis represents the rightward direction. The positive direction of the Y-axis represents the forward direction. The positive direction of the Z-axis represents the upward direction.

[0066] Figure 2 This is a side view of the camera unit 2 installed in the cold storage 1, viewed from the right side. Figure 3 This is a perspective view of the position limiting component 205 that abuts against the front edge 10A of the main housing 10, viewed from the left front side.

[0067] like Figure 2 and Figure 3 As shown, the camera unit 2 includes a box-shaped main body 201 disposed on the upper surface 10B of the main housing 10. Furthermore, the camera unit 2 includes a shooting component 202 extending forward from the main body 201, located above the front of the main body 201. The shooting component 202 includes a refrigerator compartment camera 203 and a drawer camera 204. The refrigerator compartment camera 203 and the drawer camera 204 are respectively positioned at a position further forward than the main body 201 in the lower part of the shooting component 202.

[0068] The main body 201 includes a position limiting member 205 that restricts the position of the camera unit 202 in the front-rear direction of the cold storage 1 by abutting against the front end edge 10A of the main housing 10. The position limiting member 205 is a position limiting piece that extends downward from the bottom surface 206 of the main body 201 in the set state. The position limiting member 205 extends along the left-right direction of the front surface of the main body 201 and has a mark 207 at the center in the left-right direction of the main body 201. This mark 207 indicates the center position of the camera unit 2 in the left-right direction.

[0069] The length of the position limiting member 205 in the installed state is shorter than the length from the upper surface 10B of the main body 10 to the liner provided between the cold storage 1 and the door 11C. More specifically, the length of the position limiting member 205 in the installed state is shorter than the length from the upper surface 10B of the main body 10 to the door liner provided on the upper part of the inner surface of the door 11C, which is substantially parallel to the upper surface 10B.

[0070] The main body 201 is fixed to the position limited by the position limiting component 205 by a built-in magnet 208. The means of fixing the main body 201 is not limited to the magnet 208, but can be other means such as double-sided tape.

[0071] Figure 4 This is a front view of the cold storage unit 1 and camera unit 2. Figure 4 The left door 11A and right door 11B of the cold storage 1 shown are open. Figure 5 This is a top view of the cold storage unit 1 and camera unit 2 from the right side. Figure 5 The refrigerator compartment 1 shown has drawer 15A in the open position (i.e., pulled out). Figure 4 and Figure 5 In the camera unit 2 shown, mark 207 is located approximately in the center of the cold storage 1 in the left-right direction, and position limiting component 205 abuts against the front edge 10A.

[0072] Camera unit 2 includes a refrigerator camera 203 and a drawer camera 204.

[0073] The camera 203 for the cold storage room captures images of food entering and leaving the cold storage room 11 from the top front and below.

[0074] For example, the shooting range of the camera 203 for the cold storage room includes in the front view. Figure 4 The range shown is A1. Range A1 is the area including the door shelf 111 provided on the door 11C and the opening on the front surface of the refrigerator compartment 11 when the door 11C is open. Furthermore, the shooting range of the refrigerator compartment camera 203, when viewed from the side, includes... Figure 5 The range A3 shown is the range extending from the open door 11C to the front end of the shelf 112 in the front-back direction, and the range extending from the opening of the refrigerator compartment 11 in the vertical direction.

[0075] The drawer-mounted camera 204 captures images of food entering and exiting the drawer-type storage compartment from the top front of the refrigerator 1.

[0076] For example, when looking forward, the shooting range of the drawer camera 204 includes... Figure 4 The area shown is A2. Area A2 includes drawer 16A when viewed from the front of the refrigerator 1. Furthermore, the area captured by camera 204 of the drawer when viewed from the side includes... Figure 5 The range shown is A4. Range A4 is the range of drawer 16A in the front-to-back direction, including the state when it is pulled out to its maximum extent.

[0077] The camera unit 2 has a range sensor 209 for the left door, a range sensor 210 for the right door, and a range sensor 211 for the drawer.

[0078] The left door distance sensor 209 is a distance sensor used to detect the open / closed state of the left door 11A. The left door distance sensor 209 detects the separation distance from the left door 11A as the open / closed state of the left door 11A.

[0079] The right door distance sensor 210 is a distance sensor used to detect the open / closed state of the right door 11B. The right door distance sensor 210 also detects the separation distance from the right door 11B as the open / closed state of the right door 11B.

[0080] The drawer distance sensor 211 is a distance sensor used to detect the open / closed state of drawer 16A. Furthermore, the open state of drawer 16A refers to its extension out of the corresponding drawer compartment. Conversely, the closed state of drawer 16A refers to its retraction into the corresponding drawer compartment. The drawer distance sensor 211 detects the distance between itself and drawer 16A as the open / closed state of drawer 16A.

[0081] In this embodiment, the drawer distance sensor 211 detects the opening and closing status of drawers 14A and 15A respectively.

[0082] The camera unit 2 has a human detection sensor 212. The human detection sensor 212 outputs the detected value to the camera unit control unit 20. The human detection sensor 212 is set at a predetermined position on the shooting component 202.

[0083] The human sensor 212 corresponds to an example of the "sensor" of the present invention.

[0084] in addition, Figure 4 and Figure 5 The arrangement and placement of the sensors and cameras are illustrated for the purpose of explaining camera unit 2, and their arrangement and placement are not limited to... Figure 4 and Figure 5 The arrangement order and setting position are shown.

[0085] Figure 6 This is a block diagram representing the structure of camera unit 2.

[0086] The camera unit 2 includes a camera unit control unit 20, a camera unit communication unit 21, a sensor unit 22, an imaging unit 23, and a power supply unit 24.

[0087] The camera unit control unit 20 includes a CPU (Central Processing Unit), an MPU (Microprocessor Unit), and other processors, namely a camera unit processor 220, and a camera unit storage unit 230. The camera unit control unit 20 reads and executes control programs stored in the camera unit storage unit 230 through the camera unit processor 220 to control various parts of the camera unit 2. The camera unit processor 220, by executing the control programs stored in the camera unit storage unit 230, functions as the camera unit communication control unit 221, detection value processing unit 222, image control unit 223, video recording control unit 224, and food detection unit 225.

[0088] The camera unit storage unit 230 includes a memory that stores the program executed by the camera unit processor 220 and the data processed by the camera unit processor 220. The camera unit storage unit 230 stores the control program executed by the camera unit processor 220, the identification information of the camera unit 2 (camera unit ID 4112), and various other data. The camera unit storage unit 230 has a non-volatile storage area. Alternatively, the camera unit storage unit 230 may include a volatile storage area, constituting the working area of ​​the camera unit processor 220.

[0089] The camera unit communication unit 21 is a communication interface that includes communication circuits, connectors, etc., according to a specified communication standard, and communicates with the food management server 4 and the terminal device 3 under the control of the camera unit control unit 20. In this embodiment, the communication standard used by the camera unit communication unit 21 is exemplified by a wireless communication standard, but it can also be a wired communication standard.

[0090] The sensor unit 22 includes a left door range sensor 209, a right door range sensor 210, a drawer range sensor 211, and a human sensor 212, and outputs the detection value of each sensor to the camera unit control unit 20.

[0091] The camera unit 23 includes a refrigerator compartment camera 203 and a drawer camera 204. The camera unit 23 outputs the shooting results of the refrigerator compartment camera 203 and the drawer camera 204 to the camera unit control unit 20. The refrigerator compartment camera 203 and the drawer camera 204 capture moving images.

[0092] Hereinafter, without distinguishing between the refrigerator compartment camera 203 and the drawer camera 204, they will be referred to as "camera" and given the symbol "213".

[0093] The power supply unit 24 includes hardware such as power supply circuitry, and supplies power to each part of the camera unit 2. In this embodiment, the power cable 7 of the camera unit 2 is connected to the power frequency AC power supply 8, and the power supply unit 24 supplies power to each of the camera units 2 based on the power supplied from the external power frequency AC power supply 8.

[0094] Furthermore, the power supply source for the power supply unit 24 is not limited to an external AC power source 8; it can be the refrigerator 1 or a battery. If the power source is the former, both the power supply unit 24 and the refrigerator 1 include hardware such as a port capable of USB power supply. Additionally, if the power source is the former, the USB cable of the camera unit 2 is connected to the USB port of the refrigerator 1, and the refrigerator 1 supplies power to the power supply unit 24 via the USB cable. If the power source is the latter, the camera unit 2 includes a battery.

[0095] As described above, the camera unit control unit 20 serves as the camera unit communication control unit 221, the detection value processing unit 222, the camera control unit 223, the video recording control unit 224, and the food detection unit 225.

[0096] Food Inspection Department 225 is an example corresponding to Article Inspection Department.

[0097] The camera unit communication control unit 221 communicates with the food management server 4 via the camera unit communication unit 21.

[0098] Based on the detection values ​​from each sensor output from the sensor unit 22, the detection value processing unit 222 determines whether the left door 11A, right door 11B, and drawers 14A and 15A are open or closed. If the detection value processing unit 222 determines that either the left door 11A or right door 11B is open, it outputs open status information indicating that door 11C is open to the video recording control unit 224 and the food detection unit 225. Similarly, if the detection value processing unit 222 determines that both the left door 11A and right door 11B are closed, it outputs closed status information indicating that door 11C is closed to the video recording control unit 224 and the food detection unit 225. Finally, if the detection value processing unit 222 determines that either drawer 14A or 15A is open, it outputs drawer open status information indicating that either drawer 14A or 15A is open to the video recording control unit 224 and the food detection unit 225. In addition, when the detection value processing unit 222 determines that drawers 14A and 15A are both closed, it outputs the drawer closure status information indicating that drawers 14A and 15A are closed to the video recording control unit 224 and the food detection unit 225.

[0099] Furthermore, the detection value processing unit 222 determines whether a person exists in the vicinity of the cold storage 1 based on the detection value output from the human detection sensor 212 from the sensor unit 22. If the detection value processing unit 222 determines that a person exists in the vicinity of the cold storage 1, it outputs information indicating the presence or absence of a person to the camera control unit 223. Conversely, if the detection value processing unit 222 determines that no person exists in the vicinity of the cold storage 1, it outputs information indicating the absence or absence of a person to the camera control unit 223.

[0100] When the detection value processing unit 222 outputs information indicating the presence or absence of a person, the camera control unit 223 causes the refrigerator camera 203 and the drawer camera 204 to start recording. When the detection value processing unit 222 outputs information indicating the absence or absence of a person, the camera control unit 223 causes the refrigerator camera 203 and the drawer camera 204 to stop recording.

[0101] When the detection value processing unit 222 outputs door open status information, the recording control unit 224 begins recording the footage captured by the refrigerator compartment camera 203. When the detection value processing unit 222 outputs door closed status information, the recording control unit 224 ends the recording of the refrigerator compartment camera 203. Furthermore, when the detection value processing unit 222 outputs drawer open status information, the recording control unit 224 begins recording the footage captured by the drawer camera 204. When the detection value processing unit 222 outputs drawer closed status information, the recording control unit 224 ends the recording of the drawer camera 204. Upon ending recording, the recording control unit 224 generates a video file 231 containing the dynamic images from the start to the end of recording and stores it in the camera unit storage unit 230.

[0102] The food inspection unit 225 inspects food entering and leaving the cold storage 1 based on the video file 231 stored in the camera unit storage unit 230.

[0103] Figure 7 This is a block diagram showing the structure of terminal device 3 and food management server 4.

[0104] [1-1-3. Structure of the Food Management Server]

[0105] First, let's explain the structure of the food management server 4.

[0106] The food management server 4 has a server control unit 40 and a server communication unit 41.

[0107] The server control unit 40 includes a processor such as a CPU or MPU, namely a server processor 400, and a server storage unit 410. The server control unit 40 reads and executes the control program stored in the server storage unit 410 to control the various parts of the food management server 4. The server processor 400, by executing the control program stored in the server storage unit 410, serves as the server communication control unit 401 and the information processing unit 402.

[0108] Server storage unit 410 has a memory that stores programs executed by server processor 400 and data processed by server processor 400. Server storage unit 410 stores control programs executed by server processor 400, refrigerator food management database 411, drawer food management database 412, and various other data. Server storage unit 410 has non-volatile storage areas. Server storage unit 410 may also have volatile storage areas, constituting the working area of ​​server processor 400.

[0109] Figure 8This is a diagram representing an example of a record stored in the cold storage food management database 411.

[0110] The cold storage food management database 411 is a database that stores various information related to the food stored in the cold storage 11.

[0111] In the following description, a record stored in the cold storage food management database 411 will be referred to as a "cold storage food management record" and assigned the symbol "RR".

[0112] The Cold Storage Food Management Record RR includes account ID 4111, camera unit ID 4112, cold storage model information 4113, cold storage image data 4114, entry / exit information 4115, food image data 4116, storage date and time information 4117, segmentation area number 4118, remarks input information 4119, remarks information 4130, deletion information 4131, and deletion date and time information 4132.

[0113] Account ID 4111 is identification information that identifies the account assigned to user P using the food management application 311.

[0114] The cold storage unit model information 4113 indicates the model of the cold storage unit 1 used by user P, represented by account ID 4111, which is created in the same record.

[0115] Cold storage compartment image data 4114 is image data representing an image inside the cold storage compartment 11 of the cold storage compartment 1. Hereinafter, the image representing the inside of the cold storage compartment 11 will be referred to as the "cold storage compartment image" and assigned the symbol "RG". Cold storage compartment image data 4114 represents the cold storage compartment image RG corresponding to the model represented by the cold storage model information 4113 established in the same record.

[0116] Image RG inside the cold storage corresponds to an example of an image inside the storage room.

[0117] Figure 9 This is an example of an image RG representing the interior of a cold storage room.

[0118] Figure 9 The image RG showing the interior of the cold storage compartment includes the left door shelf image RG1, the front surface image of the cold storage compartment RG2, and the right door shelf image RG3. Figure 9In the image RG showing the interior of the refrigerator compartment, the left door shelf image RG1, the front surface image RG2, and the right door shelf image RG3 are arranged sequentially from the left side of the image. The left door shelf image RG1 shows the door shelf 111 formed at the left door 11A. The front surface image RG2 is a frontal view of the refrigerator compartment 11 with the left door 11A and right door 11B open. The right door shelf image RG3 shows the door shelf 111 formed at the right door 11B.

[0119] The image region RG inside the cold storage room is segmented into multiple regions. Each region is called a "segmentation region" and assigned the symbol "BA". Figure 9 The image RG showing the interior of the cold storage room is divided into 15 segmented regions BA. Each segmented region BA is assigned a unique number. In the following description, this number will be referred to as the segmented region number and given the symbol "4118". Additionally, in... Figure 9 For convenience, the diagram shows which segmentation region BA is assigned which segmentation region number. However, segmentation region number 4118 is not shown in the image RG of the cold storage room displayed on the main screen GM2 of the cold storage room described later.

[0120] return Figure 8 The explanation for "4115" indicates whether there is any new information regarding inbound or outbound entries.

[0121] Food image data 4116 is image data of food. In this embodiment, the image represented by food image data 4116 is an image captured by camera 203 in the refrigerator compartment.

[0122] The storage date and time information 4117 indicates the storage date and time of the food in the cold storage 1.

[0123] The cold storage food management record RR has a segmented region number 4118 for each food image data 4116.

[0124] The "Notes" field (4119) indicates whether there is any information to include in the input.

[0125] The note information 4130 indicates the information of the entered note.

[0126] Deletion information 4131 indicates whether the corresponding sub-record was created from a sub-record that was the object of deletion in the Refrigerated Food Management Record RR. The sub-record is a record stored in the Refrigerated Food Management Record RR, which contains food image data 4116, storage date and time information 4117, segmentation area number 4118, whether there is a note input 4119, and note information 4130.

[0127] The deletion date and time information 4132 indicates the date and time when the corresponding sub-record was created and became the object of deletion. When the deletion information 4131 indicates that the object is to be deleted, the information processing unit 402, described later, deletes the sub-record and the deletion information 4131 and deletion date and time information 4132 corresponding to the sub-record from the cold storage food management record RR at a predetermined time.

[0128] Figure 10 This is a diagram representing a record stored in the drawer food management database 412.

[0129] The drawer food management database 412 is a database that stores various information related to food in drawer-type storage compartments. Additionally, in reference... Figures 23-25 In the description, drawer food management database 412 is a database related to vegetable compartment 15.

[0130] In the following description, a record stored in drawer food management database 412 will be referred to as a "drawer food management record" and assigned the symbol "HR".

[0131] The drawer food management record HR has account ID 4111, camera unit ID 4112, update date and time information 4121, upper box information 4122, and lower box information 4124.

[0132] The update date and time information 4121 indicates that the contents of the drawer food management record HR were updated on the date and time.

[0133] The upper box information 4122 includes upper box image data 4123, food image data 4116, remarks input information 4119, and remarks information 4130.

[0134] Image data 4123 for the upper box is the image data of the upper box JCA, i.e., the upper box image JCG. The upper box image JCG is an image captured by the drawer camera 204.

[0135] The upper box image JCG corresponds to an example of the first box image of the present invention.

[0136] In this embodiment, the image represented by the food image data 4116 in the upper box information 4122 is an image cut out from the upper box image JCG.

[0137] The lower box information 4124 includes lower box image data 4125, food image data 4116, remarks input information 4119, and remarks information 4130.

[0138] The lower box image data 4125 is the image data of the lower box GCA, i.e., the lower box image GCG. The lower box image GCG is the image captured by the drawer camera 204.

[0139] The lower-level box image GCG corresponds to an example of the second box image of the present invention.

[0140] Hereinafter, without distinguishing between the upper box image JCG and the lower box image GCG, they will be referred to as "box image" and labeled with the symbol "CG".

[0141] In this embodiment, the image represented by the food image data 4116 in the lower box information 4124 is an image cut out from the lower box image GCG.

[0142] return Figure 7 As explained, the server communication unit 41 includes a communication interface comprising a wireless circuit, antenna, and other communication-related structures in accordance with a prescribed communication standard, and communicates with devices connected to the global network GN in accordance with a prescribed communication standard. In this embodiment, the server communication unit 41 communicates with the camera unit 2 and the terminal device 3.

[0143] As described above, the server control unit 40 serves as both the server communication control unit 401 and the information processing unit 402.

[0144] The server communication control unit 401 communicates with the camera unit 2 and the terminal device 3 via the server communication unit 41.

[0145] The information processing unit 402 performs various information processing based on the information received from the camera unit 2 and the terminal device 3 by the server communication control unit 401.

[0146] [1-1-4. Structure of the Terminal Device]

[0147] Next, the functional structure of terminal device 3 will be explained.

[0148] The terminal device 3 includes a terminal control unit 30, a terminal communication unit 31, and a touch panel 32.

[0149] The touch panel 32 corresponds to an example of the display unit.

[0150] The terminal control unit 30 includes a CPU, an MPU, and other processors, namely a terminal processor 300, and a terminal storage unit 310. The terminal control unit 30 reads and executes control programs stored in the terminal storage unit 310 through the terminal processor 300 to control various parts of the terminal device 3. A food management application 311 is installed on the terminal device 3. The terminal processor 300 reads and executes the food management application 311, acting as an application execution unit 301.

[0151] The terminal processor 300 corresponds to an example of the computer of the present invention. Furthermore, the application execution unit 301 corresponds to an example of the display control unit of the present invention.

[0152] The terminal storage unit 310 includes a memory that stores the program executed by the terminal processor 300 and the data processed by the terminal processor 300. The terminal storage unit 310 stores the control program executed by the terminal processor 300, the food management application 311, the account ID 4111, and various other data. The terminal storage unit 310 has a non-volatile storage area. Alternatively, the terminal storage unit 310 may include a volatile storage area, constituting the working area of ​​the terminal processor 300.

[0153] The terminal communication unit 31 is a communication interface comprising a wireless circuit and an antenna constructed according to a prescribed communication standard, and communicates with devices connected to the Global Network (GN) according to the prescribed communication standard. In this invention, a wireless communication standard is exemplified as the communication standard used by the terminal communication unit 31.

[0154] The touch panel 32 has a display panel such as an LCD display panel, and a touch sensor disposed overlapping with or integrally with the display panel.

[0155] As described above, the terminal control unit 30 is used as the application execution unit 301.

[0156] Application execution unit 301 sends and receives various information with food management server 4 via terminal communication unit 31.

[0157] Furthermore, the application execution unit 301 displays a user interface related to the management of food in the cold storage 1 on the touch panel 32. In the following description, this user interface will be referred to as the "application UI," and the reference numeral "320" will be used. By displaying the application UI 320 on the touch panel 32, the application execution unit 301 provides the user P with various information related to the food in the cold storage 1, and also receives various inputs related to the food in the cold storage 1 from the user P.

[0158] [1-2. Actions]

[0159] Next, the operation of each part of the food management system 1000 will be explained.

[0160] [1-2-1. Camera Unit Setup]

[0161] Camera unit 2 is separate from cold storage 1. Therefore, camera unit 2 is installed by user P on the upper surface 10B of cold storage 1. As described above, the position limiting component 205 restricts the position of camera unit 2 in the front-back direction of cold storage 1, so user P adjusts the position of cold storage 1 in the left-right direction when setting up camera unit 2.

[0162] The following examples illustrate several methods for adjusting the position of camera unit 2 in the left-right direction of cold storage 1.

[0163] [1-2-1-1. Adjustment of the first position]

[0164] In the first position adjustment, user P uses application UI320.

[0165] During the first position adjustment, the application execution unit 301 causes the touch panel 32 to display the application UI 320 of the adjustment screen GM1.

[0166] Figure 11 This is an example of an adjustment screen GM1.

[0167] Adjust the image GM1 to set the shooting result display area HA1. The shooting result display area HA1 is a rectangular area that displays the shooting results of either the refrigerator camera 203 or the drawer camera 204. In the shooting result display area HA1, a line L1 extending vertically is set approximately in the center in the left-right direction as shown in the figure.

[0168] The adjustment screen GM1 includes adjustment method information J1. The adjustment method information J1 promotes position adjustment so that the center of the cold storage 1 reflected in the shooting result display area HA1 in the left-right direction becomes the line L1 set in the shooting result display area HA1.

[0169] User P installs camera unit 2 in the cold storage 1 according to the adjustment method indicated by adjustment method information J1. This allows food entering and leaving the cold storage 1 to be captured within the respective shooting ranges of the cold storage camera 203 and drawer camera 204 of camera unit 2. Furthermore, the range sensors included in camera unit 2 are capable of detecting the opening and closing of the corresponding door 11C or drawer 16A.

[0170] During the first position adjustment, camera unit 2 sends the capture results from either the refrigerator camera 203 or the drawer camera 204 to terminal device 3. Additionally, during the first position adjustment, camera unit 2 and terminal device 3 communicate directly via food management server 4 or through short-range wireless communication.

[0171] The camera unit communication control unit 221 of camera unit 2 sends the shooting results of either the refrigerator camera 203 or the drawer camera 204 to the terminal device 3. Then, the application execution unit 301 displays the received shooting results in the shooting result display area HA1.

[0172] [1-2-1-2. Second position adjustment]

[0173] In the second position adjustment, user P uses the central position to record paper CY.

[0174] Figure 12 This is a diagram illustrating an example of a paper CY with the central position recorded.

[0175] The central position recording paper CY is a paper that records a central mark MR2 indicating the central position of the cold storage 1 in the left-right direction. In addition to the central mark MR2, the central position recording paper CY also records a left end mark MR1 indicating the left end position of the cold storage 1 in the left-right direction, and a right end mark MR3 indicating the right end position of the cold storage 1 in the left-right direction.

[0176] The central position recording sheet CY is bundled together with the cold storage unit 1 and delivered to the user P. Alternatively, the camera unit 2 may be bundled together with the central position recording sheet CY and the cold storage unit 1, or it may not be bundled together. For the central position recording sheet CY bundled with the cold storage unit 1, the length from the left end marker MR1 to the right end marker MR3 is the length corresponding to the width of the cold storage unit 1.

[0177] In the second position adjustment, user P uses the center position recording paper CY to determine the center position of the cold storage 1 in the left-right direction. For example, user P attaches the center position recording paper CY to the cold storage 1, with the left end marked MR1 as the left end and the right end marked MR3 as the right end, and uses the center mark MR2 as a marker to determine the center position of the cold storage 1 in the left-right direction. Then, user P adjusts the position of the camera unit 2 so that the mark 207 on the position limiting component 205 of the camera unit 2 becomes the determined center position.

[0178] By adjusting the second position, camera unit 2 can perform various detections in the same way as when the first position is adjusted.

[0179] [1-2-1-3. Adjustment of the 3rd position]

[0180] In the third position adjustment, a mark indicating the center position of the cold storage 1 in the left-right direction is applied at a predetermined position of the cold storage 1. In the third position adjustment, the mark 207 of the position limiting component 205 and the mark applied to the cold storage 1 are used.

[0181] In the third position adjustment, user P adjusts the position of camera unit 2 in the left and right direction of cold storage 1 so that the position of the mark assigned to cold storage 1 is consistent with the position of mark 207 on position limiting component 205.

[0182] By adjusting the third position, camera unit 2 can perform various detections in the same way as when adjusting the first position.

[0183] Next, the operation of the food management system 1000, which has the camera unit 2 installed behind the cold storage 1, will be explained.

[0184] [1-2-2. Actions related to video recording]

[0185] First, the actions related to the recording will be described.

[0186] Figure 13 This is a flowchart FA representing the actions of camera unit 2.

[0187] exist Figure 13 In the illustrated operation, sensor unit 22 outputs the detection values ​​from each sensor to camera unit control unit 20. Furthermore, in Figure 13 In the operation shown, the detection value processing unit 222 performs processing based on various detection values ​​received from the sensor unit 22.

[0188] The detection value processing unit 222 determines whether there is a person in the vicinity of the cold storage 1 based on the detection value of the human sensor 212 (step SA1).

[0189] If the detection value processing unit 222 determines that there is a person in the vicinity of the cold storage 1 (step SA1: Yes), it outputs information indicating whether a person is present to the camera control unit 223 (step SA2).

[0190] When the camera control unit 223 receives information from the detection value processing unit 222 indicating whether a person is present, it causes the refrigerator camera 203 and the drawer camera 204 to start recording (step SA3).

[0191] Next, the detection value processing unit 222 determines whether any one of the left door 11A, right door 11B, drawer 14A, and 15A is open based on the detection values ​​of various sensors output from the sensor unit 22 (step SA4).

[0192] If the detection value processing unit 222 determines that the left door 11A, right door 11B, drawers 14A and 15A are not in the open state (step SA4: no), it will proceed to step SA16.

[0193] If the detection value processing unit 222 determines that any one of the left door 11A, right door 11B, drawer 14A, and 15A is open (step SA4: Yes), it determines whether door 11C is open or any one of drawers 14A and 15A is open (step SA5).

[0194] When the detection value processing unit 222 determines that the door 11C is in the open state (step SA5: door), it outputs the door open state information to the recording control unit 224 (step SA6).

[0195] Next, when the door open status information is input from the detection value processing unit 222, the video recording control unit 224 starts recording the shooting results of the refrigerator compartment camera 203 (step SA7).

[0196] Next, the detection value processing unit 222 determines whether the gate 11C is in a closed state (step SA8).

[0197] When the detection value processing unit 222 determines that the door 11C is in a closed state (step SA8: Yes), it outputs the door closed state information to the recording control unit 224 (step SA9).

[0198] Next, when the video recording control unit 224 receives door-closed status information from the detection value processing unit 222, it ends the recording of the camera footage from the refrigerator compartment camera 203 (step SA10). As a result, a video file 231 of the refrigerator compartment camera 203 is generated in the camera unit storage unit 230.

[0199] Returning to the explanation of step SA5, if the detection value processing unit 222 determines that either drawer 14A or 15A is in the open state (step SA5: drawer), it outputs the drawer open state information to the video recording control unit 224 (step SA11).

[0200] Next, when the drawer open status information is input from the detection value processing unit 222, the video recording control unit 224 starts recording the shooting results of the drawer camera 204 (step SA12).

[0201] Next, the detection value processing unit 222 determines whether either of the open drawers 14A and 15A is in a closed state (step SA13).

[0202] If the detection value processing unit 222 determines that either of the open drawers 14A or 15A is in a closed state (step SA13: Yes), it outputs the drawer closed state information to the video recording control unit 224 (step SA14).

[0203] Next, when the recording control unit 224 receives the drawer-closed state information from the detection value processing unit 222, it ends the recording of the drawer camera 204 (step SA15). As a result, a recording file 231 of the drawer camera 204 is generated in the camera unit storage unit 230.

[0204] The detection value processing unit 222 determines whether there are people around the cold storage 1 (step SA16).

[0205] If the detection value processing unit 222 determines that there are people around the cold storage 1 (step SA16: yes), it will perform the processing after step SA4 again.

[0206] On the other hand, if the detection value processing unit 222 determines that there are no people around the cold storage 1 (step SA16: no), it outputs information indicating whether there are no people to the camera control unit 223 (step SA17).

[0207] When the camera control unit 223 receives information from the detection value processing unit 222 indicating whether or not a person is present, it causes the refrigerator camera 203 and the drawer camera 204 to stop recording (step SA18).

[0208] [1-2-3. Actions related to food testing]

[0209] Next, we will explain the procedures related to food testing.

[0210] Figure 14 This is a flowchart illustrating the actions of the food management system 1000.

[0211] exist Figure 14 In the flowchart, FB represents the actions of camera unit 2, and FC represents the actions of food management server 4.

[0212] The food detection unit 225 of the camera unit 2 determines whether a video file 231 is stored in the camera unit storage unit 230 (step SB1).

[0213] If the food inspection unit 225 determines that a video file 231 is stored in the camera unit storage unit 230 (step SB1: Yes), it then determines whether the stored video file 231 is the video file 231 of the refrigerator camera 203 or the video file 231 of the drawer camera 204 (step SB2). The determination in step SB2 is based on the information output by the detection value processing unit 222.

[0214] If the food inspection unit 225 determines that the video file 231 is from the refrigerator camera 203 (step SB2: refrigerator camera), it inspects the food based on the stored video file 231 (step SB3).

[0215] In step SB3, for example, the food inspection unit 225 inspects the food based on features such as shape and color from the video file 231 recorded by the camera 203 in the refrigerator compartment. The data required for food inspection is stored in a storage area accessible to the food inspection unit 225.

[0216] The food inspection department 225 determines whether food can be detected from the video file 231 of the camera 203 in the refrigerator (step SB4).

[0217] If the food inspection unit 225 determines that food cannot be detected from the video file 231 of the refrigerator camera 203 (step SB4: no), the process is transferred to step SB10.

[0218] On the other hand, if the food inspection department 225 can detect food from the video file 231 of the camera 203 in the cold storage room (step SB4: Yes), it performs the entry and exit judgment process (step SB5).

[0219] The inbound / outbound determination process determines whether the food detected in step SB3 is food entering or leaving the cold storage 1. In this process, the inbound / outbound determination image HG is overlaid with the dynamic image represented by the video file 231 of the cold storage room camera 203.

[0220] Figure 15 This is a diagram used to illustrate the image HG used for determining whether an item enters or leaves the warehouse.

[0221] Figure 15 The image HG shown represents the case where user P is superimposed on the dynamic image that causes the egg package to be stored in cold storage 1.

[0222] The image HG used for inbound / outbound judgment is set with a boundary line L2.

[0223] Boundary line L2 includes boundary lines L21, L22, L23, L24, and L25. Boundary line L21 is located in the upper part of the inbound / outbound judgment image HG, extending from the left end of HG to the right. Boundary line L22 connects the right end of boundary line L21 to the left end of boundary line L23. Boundary line L23 is located approximately at the center of the vertical direction of the inbound / outbound judgment image HG, extending to the right from the right end of boundary line L21. Boundary line L24 connects the right end of boundary line L23 to the left end of boundary line L24. Boundary line L25 is located in the upper part of the inbound / outbound judgment image HG, extending to the right from the right end of boundary line L23 to the right end of the inbound / outbound judgment image HG.

[0224] The image region of the inbound / outbound judgment image HG is divided into a concave region (HG1) and a convex region (HG2) by the boundary line L2.

[0225] The food inspection unit 225 performs image analysis on a dynamic image superimposed with an entry / exit determination image HG to determine whether the food detected in step SB3 is moving from the convex region HG2 to the concave region HG1 or from the concave region HG1 to the convex region HG2. If the food inspection unit 225 determines that the food detected in step SB3 is moving from the convex region HG2 to the concave region HG1, it determines that the food detected in step SB3 is stored in the cold storage 1. Conversely, if the food inspection unit 225 determines that the food detected in step SB3 is moving from the concave region HG1 to the convex region HG2, it determines that the food detected in step SB3 is removed from the cold storage 1.

[0226] return Figure 14 The flowchart FB describes the process of determining the segmentation area number in Food Inspection Department 225 (step SB6).

[0227] The segmentation area number determination process is to determine the segmentation area BA corresponding to the food storage area detected in step SB3, and to determine the segmentation area number 4118 assigned to the determined segmentation area BA.

[0228] For example, the food detection unit 225 determines the storage area of ​​the cold storage compartment 11 where the food detected in step SB3 enters and exits the storage compartment through image analysis. Furthermore, for example, markings identifying the storage area are assigned to each front end of the shelf 112 and each door shelf 111. The food detection unit 225 determines the storage area based on the positional relationship between these markings and the detected food in the dynamic image. Next, the food detection unit 225 determines which segmentation region BA of the cold storage compartment image RG corresponds to the determined storage area, and assigns a segmentation region number 4118 to the determined segmentation region BA. The correspondence between the storage area, the segmentation region BA, and the segmentation region number 4118 is stored as data in the camera unit storage unit 230.

[0229] When performing the segmentation area numbering determination process, the food inspection unit 225 extracts an image of each food item detected in step SB3 from the video file 231 of the refrigerator camera 203 (step SB7). Hereinafter, the food images extracted from the capture results of the camera 213 will be referred to as "captured food images" and assigned the symbol "FG".

[0230] The food image FG corresponds to an example of the food image of the present invention.

[0231] Next, the food inspection department 225 generates the first update information (step SB8).

[0232] The first update information generated in step SB8 includes the camera unit ID 4112 of the camera unit storage unit 230.

[0233] In addition, the first update information generated in step SB8 includes date and time information indicating the date and time when the video file 231 of the refrigerator camera 203 was generated.

[0234] In addition, the first update information generated in step SB8 includes, for each food item detected in step SB3, food image data 4116 representing the food image FG captured in step SB7, entry and exit determination result information representing the determination result of the entry and exit determination process, and segmentation region number 4118 determined by the segmentation region number determination process.

[0235] Next, the camera unit communication control unit 221 sends the first update information generated by the food detection unit 225 to the food management server 4 (step SB9).

[0236] The food inspection unit 225 deletes the video file 231 from the camera unit storage unit 230 (step SB10).

[0237] Returning to the explanation of step SB2, if the food inspection unit 225 determines that the video file 231 is from the drawer camera 204 (step SB2: drawer camera), it determines whether at least one of the upper box JCA and the lower box GCA of the dynamic image of the video file 231 displayed in the drawer camera 204 has been pulled out above a predetermined threshold (step SB11).

[0238] For example, the food inspection unit 225 distinguishes box CA in a dynamic image based on features such as shape and color. Then, the food inspection unit 225 determines, through image analysis of the video file 231, whether the upper box JCA and the lower box GCA have been pulled above a predetermined threshold relative to the cold storage 1.

[0239] Furthermore, for example, the upper box JCA and the lower box GCA are each marked at a predetermined position to identify the box CA. The food inspection unit 225 distinguishes the box CA in the dynamic image by reading the mark from the dynamic image. Then, the food inspection unit 225 determines whether the box CA has been pulled beyond a predetermined threshold relative to the cold storage 1 by calculating the distance the mark has moved in the dynamic image.

[0240] If the food inspection department 225 determines that neither the upper box JCA nor the lower box GCA has been pulled above the specified threshold (step SB11: no), it proceeds to step SB18.

[0241] On the other hand, if the food inspection unit 225 determines that at least one of the upper box JCA and the lower box GCA has been pulled above a predetermined threshold (step SB11: Yes), it cuts out the box image CG from the video file 231 (step SB12).

[0242] In step SB12, if only the upper box JCA is pulled above a specified threshold, the food detection unit 225 cuts out the upper box image JCG from the video file 231.

[0243] In step SB12, the food detection unit 225 extracts the lower box image GCG from the video file 231 if only the lower box GCA is pulled above a specified threshold.

[0244] In step SB12, when the upper box JCA and the lower box GCA are pulled above a predetermined threshold, the food detection unit 225 cuts out the upper box image JCG and the lower box image GCG from the video file 231.

[0245] Next, the food inspection unit 225 inspects the food from the cut-out box image CG (step SB13).

[0246] Next, the food inspection unit 225 determines whether food can be detected from the box image CG (step SB14).

[0247] If the food inspection department 225 determines that no food can be detected (step SB14: No), it proceeds to step SB16.

[0248] On the other hand, when the food detection unit 225 determines that food can be detected (step SB14: Yes), it cuts out and captures food images FG from the box image CG for each detected food (step SB15).

[0249] Next, the food inspection department 225 generates the second update information (step SB16).

[0250] If only the upper box image JCG is cut out, the second update information generated in step SB16 includes the camera unit ID 4112 of the camera unit storage unit 230, the upper box image data 4123 indicating the cut-out upper box image JCG, and the food image data 4116 indicating the captured food image FG cut out from the upper box image JCG. However, if the determination in step SB14 is negative, the food image data 4116 is not included in this second update information.

[0251] Furthermore, when only the lower box image GCG is cut out, the second update information generated in step SB16 includes the camera unit ID 4112 of the camera unit storage unit 230, the lower box image data 4125 representing the cut-out lower box image GCG, and the food image data 4116 representing the captured food image FG cut out from the lower box image GCG. However, if the determination in step SB14 is negative, the food image data 4116 is not included in this second update information.

[0252] Furthermore, when the upper box image JCG and the lower box image GCG are cut out, the second update information generated in step SB16 includes the camera unit ID 4112 of the camera unit storage unit 230, the upper box image data group, and the lower box image data group. The upper box image data group includes upper box image data 4123 representing the cut-out upper box image JCG, and food image data 4116 representing the captured food image FG cut out from the upper box image JCG. The lower box image data group includes lower box image data 4125 representing the cut-out lower box image GCG, and food image data 4116 representing the captured food image FG cut out from the lower box image GCG. Additionally, if the determination in step SB14 is negative, the food image data 4116 is not included in this second update information.

[0253] Next, the camera unit communication control unit 221 sends the second update information generated by the food detection unit 225 to the food management server 4 (step SB17).

[0254] Next, the food inspection unit 225 deletes the video file 231 from the camera unit storage unit 230 (step SB18).

[0255] As shown in flowchart FC, the server communication control unit 401 of the food management server 4 determines whether it has received either the first update information or the second update information (step SC1).

[0256] If the server communication control unit 401 determines that update information has been received (step SC1: Yes), the information processing unit 402 updates the database based on the update information received by the server communication control unit 401 (step SC2).

[0257] Step SC2 is described in detail.

[0258] If the update information received by the server communication control unit 401 is the first update information, the information processing unit 402 determines the cold storage food management record RR that has the camera unit ID 4112 included in the first update information. Then, the information processing unit 402 updates the entry / exit information 4115 of the determined cold storage food management record RR to information indicating "present".

[0259] Next, if the inbound / outbound determination result information included in the first update information indicates that the item has been put into storage, the information processing unit 402 generates a sub-record and establishes a correspondence between the blank deletion information 4131 and the deletion date and time information 4132 and the generated sub-record, so as to save it in the determined cold storage food management record RR. The sub-record generated here includes the food image data 4116 included in the first update information, the storage date and time information 4117 indicating the same date and time information as the date and time information included in the first update information, the segmentation area number 4118 included in the first update information, the blank remarks input information 4119, and the blank remarks information 4130.

[0260] Furthermore, if the inbound / outbound determination result information included in the first update information indicates an outbound shipment, the information processing unit 402 determines from the cold storage food management record RR a sub-record containing the food image data 4116 included in the first update information, representing the captured food image FG, and food image data 4116 representing food images FG with high consistency. This determination is performed for each food image data 4116 included in the first update information. Then, the information processing unit 402 updates the deletion information 4131 corresponding to the determined sub-record to information indicating the deletion object. In addition, the information processing unit 402 updates the deletion date and time information 4132 corresponding to the determined sub-record to information indicating the current date and time as the deletion date and time.

[0261] Furthermore, step SC2 will be described in detail.

[0262] If the update information received by the server communication control unit 401 is the second update information, the information processing unit 402 determines the drawer food management record HR that has the camera unit ID 4112 included in the second update information.

[0263] In the case of the second update information when only the upper box image JCG is cut out, the information processing unit 402 updates the upper box image data 4123 and food image data 4116 of the upper box information 4122 of the determined drawer food management record HR to the data of the second update information.

[0264] In the case of the second update information when only the lower box image GCG is cut out, the information processing unit 402 updates the lower box image data 4125 and food image data 4116 of the determined drawer food management record HR with the data of the second update information.

[0265] In the case of the second update information when cutting out the upper box image JCG and the lower box image GCG, the information processing unit 402 appropriately updates the upper box image data 4123 and food image data 4116 of the determined drawer food management record HR's upper box information 4122 to the data included in the upper box image data group. Furthermore, the information processing unit 402 updates the lower box image data 4125 and food image data 4116 of the determined drawer food management record HR's lower box information 4124 to the data included in the lower box image data group.

[0266] In addition, the information processing unit 402 updates the determined update date and time information 4121 of the drawer food management record HR to information indicating that the drawer food management record HR has been updated.

[0267] [1-2-4. Actions related to the application UI]

[0268] Next, we will describe the actions related to the application UI320.

[0269] In this embodiment, the UI320 displays screens related to the refrigerator compartment 11 and the drawer-type storage compartment.

[0270] [1-2-4-1. Scenes related to the refrigerator compartment]

[0271] The screens related to the refrigerator compartment 11 include the refrigerator compartment home screen GM2, the refrigerator compartment list screen GM3, and the food information supplementary list screen GM4.

[0272] Figure 16 This is an example image representing the GM2 homepage of the refrigerator compartment.

[0273] The main screen of the refrigerator compartment, GM2, has a food image display area, HA2.

[0274] The food image display area HA2 is the area that displays the captured food image FG. The food image display area HA2 displays the cold storage image RG, and the captured food image FG is displayed overlapping with the cold storage image RG. Each captured food image FG is displayed overlapping within the cold storage image RG in the segmented region BA represented by segmented region number 4118, which corresponds to the segmented region number 4118 established with the food image data 4116.

[0275] Figure 16 Food image display area HA2 display Figure 9 The image RG shows the interior of a cold storage room. For example, in the cold storage food management record RR, the segmented area "4" is numbered 4118 and represents... Figure 16 The food image data 4116 corresponding to the captured food image FG1 is established. In this case, the captured food image FG1 is overlaid on the segmentation region BA, which is assigned the segmentation region number 4118 with "4".

[0276] The default food image display area HA2 displays the cold storage compartment image RG when the left door shelf image RG1 and the right door shelf image RG3 are not visible. Within the food image display area HA2, scrolling operations in the left and right directions of the image are received from user P. Based on the scrolling amount and direction of the received scrolling operations, the food image display area HA2 adjusts the display range of the cold storage compartment image RG.

[0277] For example, if the scrolling direction is to the right, it refers to the amount of scrolling up to the left end of the left door shelf image RG1. In this case, the food image display area HA2 displays the refrigerator compartment image RG, which completely obscures the right door shelf image RG3. Additionally, the captured food image FG and the annotation input icon MIC follow the change in the display range of the refrigerator compartment image RG; in other words, they move as the overlapping segmented area BA moves. The annotation input icon MIC will be described later.

[0278] When multiple food images FG are displayed in a segmented region BA, the food image display area HA2 displays them side-by-side in the left-right direction. The food image display area HA2 can simultaneously display up to five food images FG side-by-side in a segmented region BA. When a segmented region BA has six or more food images FG as display objects, the food image display area HA2 receives a scrolling operation from user P. When a segmented region BA receives a scrolling operation, it displays the sixth and subsequent food images FG. When a segmented region BA has six or more food images FG as display objects and receives a left-right scrolling operation, the food image display area HA2 maintains the maximum number of images that can be displayed side-by-side while displaying the sixth and subsequent food images FG according to the scroll amount. In this case, the food image display area HA2, based on the scroll amount and the number of food images FG as display objects, may either not display the first to fifth food images FG, or display the first and subsequent food images FG starting from the sixth and subsequent images FG. Additionally, the notes input icon MIC moves along with the associated food image FG. That is, if the associated food image FG is not displayed during scrolling, the notes input icon MIC will also not be displayed.

[0279] The food image display area HA2 displays captured food images FG, which receive input of annotations. These annotations are information that user P wants to leave for the food in cold storage 1.

[0280] Figure 17 This is a diagram showing the input process for the notes.

[0281] User P performs a prescribed touch operation, such as a long press, on a food image FG that they want to add notes to, on the refrigerator's main screen GM2. When the application execution unit 301 receives the prescribed touch operation on a food image FG, it displays a note editing image MHG associated with the food image FG that has undergone the prescribed touch operation.

[0282] The image editing feature MHG includes a note input area MNA, a note save button B1, a note delete button B2, and a food delete button B3.

[0283] The notes input area MNA is used for inputting notes. When the notes input area MNA is touched, the application execution unit 301 causes the touch panel 32 to display a soft keyboard. Then, the application execution unit 301 inputs the notes corresponding to the soft keyboard operation into the notes input area MNA.

[0284] The Notes Save button B1 is a software button used to save notes entered in the Notes Input Area MNA. The Notes Save button B1 cannot be touched if no notes are entered in the Notes Input Area MNA. When the Notes Save button B1 is touched, the application execution unit 301 displays the image of the food with notes entered in a different display mode than the display mode for the image of the food with no notes entered in the image related to the refrigerator compartment 11. In this embodiment, the application execution unit 301 displays the frame of the image of the food with notes entered in a different color, and displays the icon indicating that notes have been entered, namely the Notes Input Icon MIC, in association with the image of the food with notes entered in the image. The Notes Input Icon MIC can be touched. When the Notes Input Icon MIC is touched, the entered notes are displayed in the application UI 320.

[0285] The memo input icon MIC corresponds to an example of the memo input image of this invention.

[0286] The note deletion button B2 is used to delete the entered notes. When the note deletion button B2 is touched, the notes entered into the captured food image FG are deleted.

[0287] The food deletion button B3 is used to delete the captured food image FG from the food image display area HA2.

[0288] return Figure 16 The description of the refrigerator compartment home screen GM2 includes a list display button B4 in the food image display area HA2. The list display button B4 is a touch-operable software button.

[0289] When the list display button B4 is touched, the screen in application UI320 changes to the refrigerator list screen GM3.

[0290] The refrigerator list screen GM3 includes the storage list screen GM31, the newly added list screen GM32, and the deleted item list screen GM33.

[0291] This section explains the GM31 storage list screen.

[0292] Figure 18 This is an image representing the storage list screen of GM31.

[0293] The storage list screen GM31 has a food storage information display area HA3. The food storage information display area HA3 displays the first food information J21 for each food item stored in the refrigerator compartment 11. The first food information J21 includes a photographed food image FG, storage date and time information 4117, and segmentation area information J3. The segmentation area information J3 is an image RG of the refrigerator compartment, using color to represent the segmented area BA.

[0294] When a note is entered, the captured food image FG of the first food information J21 is displayed with a different colored frame, associated with the note input icon MIC. Additionally, the image can be scrolled vertically within the food information display area HA3. By scrolling through the food information display area HA3, user P can access the first food information J21, which cannot be fully displayed within HA3.

[0295] The storage list screen of GM31 has a first toggle button B5 and a second toggle button B6.

[0296] The first toggle button B5 is a software button used to switch the arrangement order of the first food information J21 displayed in the food information display area HA3 to the next arrangement order.

[0297] That is, when the first switch button B5 is touched, the food storage information display area HA3 displays the first food information J21 with the captured food image FG containing annotations above the first food information J21 with the captured food image FG without annotations. Furthermore, "displaying above" means displaying the information by decreasing the amount of scrolling during display. Additionally, when there are multiple first food information J21s with captured food images FG containing annotations, the food storage information display area HA3 displays the first food information J21 with the earlier storage date and time indicated by the storage date and time information 4117. Furthermore, when there are multiple first food information J21s with captured food images FG without annotations, the food storage information display area HA3 displays the first food information J21 with the earlier storage date and time indicated by the storage date and time information 4117.

[0298] in addition, Figure 18 The storage list screen GM31 on the left side of the image represents the storage list screen GM31 when the first switch button B5 has been touched.

[0299] The second switch button B6 is a software button used to switch the arrangement order of the first food information J21 displayed in the food information display area HA3 to the next arrangement order.

[0300] That is, when the second switch button B6 is touched, the food information display area HA3 displays the first food information J21 side by side in each segmented area BA. The arrangement order of each segmented area BA can be predetermined by the user P. In one segmented area BA, the food information display area HA3 displays the first food information J21 in the same arrangement as when the first switch button B5 is touched.

[0301] in addition, Figure 18 The storage list screen GM31 on the right side of the image represents the storage list screen GM31 when the second switch button B6 has been touched.

[0302] The GM31 storage list screen has a storage list display button B7, a new addition list display button B8, and a deleted object list display button B9.

[0303] The storage list display button B7 is a software button used to switch the screen displayed by the application UI320 to the storage list screen GM31.

[0304] The "New Added List Display" button B8 is a software button used to switch the screen displayed by the application UI320 to the "New Added List Display" screen of GM32.

[0305] The delete object list display button B9 is a software button used to switch the screen displayed by application UI320 to the delete object list screen GM33.

[0306] Figure 19 This is an example of a newly added list screen (GM32).

[0307] The newly added list screen GM32 has a newly added food information display area HA4. The newly added food information display area HA4 displays the first food information J21 regarding the food newly added to the refrigerator compartment 11. Furthermore, food newly added to the refrigerator compartment 11 refers to food stored within a specified period (e.g., three days) from the current date and time. The newly added food information display area HA4 displays the first food information J21 corresponding to the newly added food in a new order, with the storage date and time indicated by the storage date and time information 4117 as the top position.

[0308] The first food information J21 displayed in the newly added food information display area HA4 is each subject to a specified touch operation received from the captured food image FG. In the newly added list screen GM32, the first edit image HNG1 is displayed in association with the captured food image FG to which the specified touch operation was performed.

[0309] The first editable image HNG1 has a food deletion button B10, a region correction button B11, and a return button B12.

[0310] The food deletion button B10 is a software button that, when touched, associates the captured food image FG with the first edited image HNG1 as the food to be deleted from the refrigerator home screen GM2.

[0311] The area correction button B11 is a software button used to correct the segmented area BA represented by the segmented area information J3 when touched.

[0312] The return button B12 is a software button used to stop the display of the first edited image HNG1.

[0313] The newly added food information display area HA4 has a food information addition button B13 at the end of the overview of the first food information J21. The food information addition button B13 is a software button for manually adding new first food information J21.

[0314] Reference Figure 20 The procedure for adding information J21 to the first food information section is explained.

[0315] Figure 20 This is a diagram used to illustrate the additional process for the first food information J21.

[0316] When the food information addition button B13 is touched, the food information addition list screen GM4 is displayed in the application UI320. The food information addition list screen GM4 has multiple addition buttons B14 that function as software buttons. Furthermore, the food information addition list screen GM4 has a blank image BG and a food name input area SNA for entering food names. When the blank image BG is touched, an image representing the food name entered into the food name input area SNA appears within the blank image BG.

[0317] When the add button B14 is touched in the food information add list screen GM4, the image RG of the cold storage compartment is displayed in the application UI320. Here, the displayed image RG is an image showing the state of each segmented area BA with a segmented area number 4118 appended to it, and each segmented area BA can be selected by touch. When a segmented area BA is selected in the image RG, the next piece of food information J21 is added to the application UI320.

[0318] That is, the added first food information J21 consists of an image of the food corresponding to the added button B14 that was touched, storage date and time information 4117 indicating the date and time of adding the first food information J21 as the storage date and time, and segmentation area information J3 indicating the selected segmentation area BA. Furthermore, the image of the food represented by the added first food information J21 can be an icon. In this embodiment, the image overlapping the refrigerator compartment image RG on the refrigerator compartment home screen GM2 is assumed to be the captured food image FG. However, when the first food information J21 is added, an icon is displayed in the refrigerator compartment image RG along with the captured food image FG.

[0319] Additionally, in the process of adding the first food information J21, there may be a process where user P inputs the storage date and time. In this case, the added first food information J21 includes storage date and time information 4117 indicating the storage date and time input by user P.

[0320] return Figure 19 The GM32's new addition list screen has a list display button B7, a new addition list display button B8, and a deleted object list display button B9.

[0321] Figure 21 This is an example image of the GM33 screen showing the deletion of objects from the list.

[0322] The deletion list screen GM33 has a deletion object food information display area HA5. The deletion object food information display area HA5 displays the second food information J22 for each food item that is deleted. Additionally, the deleted food items include those deleted from the refrigerator main screen GM2 and those identified as having been removed from refrigerator 11. The second food information J22 includes a captured food image FG, deletion date and time information 4132, and segmentation information J3. The deletion object list screen GM33 displays the second food information J22 in a top-down order, with the deletion date and time as the new order.

[0323] The second food information J22 displayed in the food information display area HA5 for the deleted object receives a specified touch operation on the captured food image FG. In the deleted object list screen GM33, the second edit image HNG2 is displayed in association with the captured food image FG that has undergone the specified touch operation.

[0324] The second editable image HNG2 has a delete / cancel button B15 and a return button B16.

[0325] The delete / cancel button B15 is a software button used to associate the captured food image FG, which is linked to the second edited image HNG2, as an image displayed on the refrigerator home screen GM2.

[0326] The return button B16 is a software button used to stop the display of the second edited image HNG2.

[0327] [1-2-4-2. Actions of the Food Management System]

[0328] Next, the operation of the food management system 1000 when displaying a screen related to the refrigerator compartment 11 will be explained.

[0329] Figure 22 This is a flowchart illustrating the actions of the food management system 1000. Figure 22 In the flowchart, FD represents the actions of terminal device 3, and FE represents the actions of food management server 4.

[0330] The application execution unit 301 of the terminal device 3 determines whether to request the cold storage food management record RR from the food management server 4 (step SD1).

[0331] For example, if the application execution unit 301 receives a touch operation in the application UI 320 that instructs the display of the refrigerator compartment home screen GM2, it determines that it is in step SD1.

[0332] Furthermore, for example, if the application execution unit 301 performs a touch operation to indicate an update in the refrigerator compartment home screen GM2, it is determined to be yes in step SD1.

[0333] When the application execution unit 301 determines that a request for a cold storage food management record RR is made (step SD1: Yes), it sends the first record request information for the request for the cold storage food management record RR to the food management server 4 (step SD2). The first record request information sent in step SD2 includes the account ID 4111 stored in the terminal storage unit 310.

[0334] As shown in flowchart FF, the server communication control unit 401 of the food management server 4 receives the first record request information through the server communication unit 41 (step SE1).

[0335] Next, the information processing unit 402 determines the cold storage food management record RR, which includes the account ID 4111 included in the first record request information received by the server communication control unit 401, from the cold storage food management database 411 (step SE2).

[0336] Then, the server communication control unit 401 sends the cold storage food management record RR determined by the information processing unit 402 to the terminal device 3 (step SE3). In addition, the information processing unit 402 updates the entry / exit information 4115 of the sent cold storage food management record RR to an information indicating "none".

[0337] Referring to flowchart FD, when the application execution unit 301 of terminal device 3 receives the cold storage food management record RR from the food management server 4 (step SD3), it generates a screen related to the cold storage 11 based on the cold storage food management record RR (step SD4).

[0338] Step SD4 is described in detail.

[0339] Application execution unit 301 generates an image RG of the cold storage room, represented by the image data 4114 of the cold storage room included in the cold storage room food management record RR, and displays it on the cold storage room home screen GM2 of the food image display area HA2.

[0340] In addition, the application execution unit 301 generates a refrigerator home screen GM2 for each sub-record, where the segmented region BA represented by the segmented region number 4118 overlaps the captured food image FG represented by the food image data 4116.

[0341] In addition, if the information 4119 indicates that there is a note in the note input, the application execution unit 301 generates a refrigerator home screen GM2 that associates the note input icon MIC with the captured food image FG for each sub-record.

[0342] Furthermore, step SD4 will be described in detail.

[0343] The application execution unit 301 generates first food information J21 for each food image data 4116. This first food information J21 includes the captured food image FG represented by the food image data 4116, the storage date and time information 4117 corresponding to the food image data 4116, and the segmentation region information J3 representing the segmentation region BA indicated by the segmentation region number 4118, which is represented by color. Then, the application execution unit 301 generates a storage list screen GM31 that displays a summary of the first food information J21 related to the sub-records corresponding to the deletion information 4131 that does not indicate a deletion object. Furthermore, the application execution unit 301 generates a new addition list screen GM32 that displays a summary of the first food information J21 shown in the storage list screen GM31, representing the date and time within a specified period from the current moment, based on the storage date and time indicated by the storage date and time information 4117.

[0344] Furthermore, step SD4 will be described in detail.

[0345] The application execution unit 301 generates second food information J22 for each food image data 4116. The second food information J22 includes the captured food image FG represented by the food image data 4116, deletion date and time information 4132 corresponding to the food image data 4116, and segmentation region information J3 for the segmentation region BA represented by the segmentation region number 4118, indicated by color. Then, the application execution unit 301 generates a deletion object list screen GM33 that displays a summary of the generated second food information J22.

[0346] Returning to the explanation of flowchart FD, the application execution unit 301 displays a screen related to the refrigerator compartment 11 based on the touch operation (step SD5). Additionally, if the information 4115 regarding the presence or absence of inbound / outbound items in the refrigerator compartment food management record RR received in step SD3 indicates "yes," when the refrigerator compartment home screen GM2 is first displayed in step SD5, the application execution unit 301 displays a message on the refrigerator compartment home screen GM2 indicating that a new inbound / outbound item has occurred.

[0347] Next, the operation of the food management system 1000 when operating on the screen related to the refrigerator compartment 11 will be explained.

[0348] When the application execution unit 301 touches the note save button B1, it sends information to the food management server 4 to update the sub-record related to the captured food image FG, which is the object of the operation. This information includes note input information 4119 indicating whether there is a note, and note information 4130 indicating the entered note. The food management server 4 updates the sub-record appropriately based on the received information.

[0349] When the application execution unit 301 touches the note deletion button B2, it sends information to the food management server 4 to update the sub-record related to the captured food image FG, which is the object of the operation. This information includes note input presence / absence information 4119 indicating whether there is a note input without a note. The food management server 4 updates the sub-record appropriately based on the received information.

[0350] When the application execution unit 301 touches the food deletion buttons B3 and B10, it sends information to the food management server 4 to update the sub-record related to the captured food image FG, which is the object of the operation. This information includes deletion information 4131 indicating the object to be deleted. The food management server 4 updates the sub-record appropriately based on the received information.

[0351] When the application execution unit 301 touches the region correction button B11 and the segmented region BA is corrected, it sends information about updating the sub-record related to the captured food image FG, which is the object of the operation, to the food management server 4. This information includes the segmented region number 4118 of the corrected segmented region BA. The food management server 4 then appropriately updates the sub-record based on the received information.

[0352] Application Execution Unit 301 in use Figure 20 In the case where the append procedure shown appends the first food information J21, information about the sub-record related to the appended first food information J21 is sent to the food management server 4. The food management server 4 updates the sub-record appropriately based on the received information.

[0353] When the application execution unit 301 touches the delete cancel button B15, it sends updated information about the sub-record of the captured food image FG, which is the object of the operation, to the food management server 4. This information includes deletion information 4131 indicating a blank. The food management server 4 updates the sub-record appropriately based on the received information.

[0354] [1-2-4-3. Scenes of a drawer-style storage room]

[0355] Next, we will explain the images related to drawer-style storage compartments. When using... Figures 23-25 The description exemplifies vegetable compartment 15 as a drawer-type storage compartment. In this description, Figure 10 The database shown is a database that stores various information related to the food in the vegetable room 15.

[0356] The screens related to Vegetable Room 15 include the Vegetable Room home screen GM5 and the food image list screen GM6.

[0357] Figure 23This is an example image representing the GM5 homepage of the Vegetable Room.

[0358] The main screen of the Vegetable Room GM5 has a box-shaped image display area HA6.

[0359] The HA6 area is where the cassette image CG is displayed. The HA6 area displays the upper cassette image JCG, which overlaps with the lower cassette image GCG. By default, the HA6 area overlaps the lower cassette image GCG with the upper cassette image JCG, displaying only the upper cassette image JCG.

[0360] The HA6 image display area receives scrolling operations in the vertical direction of the received image. Based on the scrolling amount and direction of the received scrolling operation, the HA6 image display area changes the display range of the upper cassette image JCG. For example, if the scrolling direction of the received operation is upward, the HA6 image display area displays a portion of the upper cassette image JCG that is not currently visible. Within the HA6 image display area, when the upper cassette image JCG moves upward, the lower cassette image GCG, which overlaps with the upper cassette image JCG, is displayed.

[0361] The box image display area HA6 displays various box images CG that receive annotation input. Furthermore, the annotation input process is the same as in the refrigerator compartment 11, except that the image operated by user P is either a food image FG or a box image CG; the process itself is identical. Additionally, as... Figure 23 As shown, the annotation input icon MIC, which is associated with the upper box image JCG, moves along with the movement of the upper box image JCG.

[0362] The Vegetable Room homepage GM5 features a list display button B17. The list display button B17 is a touch-sensitive software button.

[0363] When the list display button B17 is touched, the food image list screen GM6 is displayed in the application UI320.

[0364] Additionally, the main screen GM5 of the vegetable compartment can have a switch button that allows switching the box image CG displayed in the box image display area HA6 to either the upper box image JCG or the lower box image GCG. In this configuration, the application execution unit 301 switches the displayed box image CG to either the upper box image JCG or the lower box image GCG based on the operation of the switch button.

[0365] Figure 24 This is an example of a food image list screen GM6.

[0366] The food image list screen GM6 has an update date and time information 4121. Figure 24 The food image list screen GM6 has update date and time information 4121, which indicates that the update date and time is "12 / 1 10:05".

[0367] The food image list screen GM6 displays captured food images (FG) for each container. The captured food images (FG) displayed in association with items in the upper container (JCA) are cropped from images in the upper container (JCG). The captured food images (FG) displayed in association with items in the lower container (GCA) are cropped from images in the lower container (GCG).

[0368] [1-2-4-5. Actions of the Food Management System]

[0369] Next, the operation of the food management system 1000 when displaying a screen related to the vegetable room 15 will be explained.

[0370] Figure 25 This is a flowchart illustrating the actions of the food management system 1000. Figure 25 In the flowchart, FF represents the actions of terminal device 3, and FJ represents the actions of food management server 4.

[0371] The application execution unit 301 of the terminal device 3 determines whether to request the drawer food management record HR from the food management server 4 (step SF1).

[0372] For example, if the application execution unit 301 receives a touch operation in the application UI 320 that instructs the display of the vegetable room home screen GM5, it determines that it is in step SF1.

[0373] Furthermore, for example, if the application execution unit 301 performs a touch operation to indicate an update in the vegetable room home screen GM5, it is determined to be yes in step SF1.

[0374] When the application execution unit 301 determines that a request for drawer food management record HR is received (step SF1: Yes), it sends the second record request information for the drawer management record HR to the food management server 4 (step SF2). The second record request information sent in step SF2 includes the account ID 4111 stored in the terminal storage unit 310.

[0375] Referring to flowchart FJ, the server communication control unit 401 of the food management server 4 receives the second record request information (step SJ1).

[0376] Next, the information processing unit 402 determines the drawer management record HR, which includes the account ID 4111 included in the second record request information received by the server communication control unit 401, from the drawer food management database 412 (step SJ2).

[0377] Then, the server communication control unit 401 sends the drawer management record HR determined by the information processing unit 402 to the terminal device 3 (step SJ3).

[0378] Referring to flowchart FJ, when the application execution unit 301 of terminal device 3 receives the drawer food management record HR from food management server 4 (step SF3), it generates a screen related to vegetable compartment 15 (step SF4).

[0379] Step SF4 is described in detail.

[0380] Application execution unit 301 generates vegetable compartment home screen GM5, which has a box image display area HA6 where the lower box image GCG represented by the lower box image data 4125 included in the drawer food management record HR overlaps with the upper box image JCG represented by the upper box image data 4123 included in the same record.

[0381] In addition, if the information 4119 indicates that there is a note in the note input, the application execution unit 301 generates a vegetable room homepage screen GM5 that associates the note input icon MIC with the corresponding box image CG.

[0382] Furthermore, step SF4 will be described in detail.

[0383] The application execution unit 301 generates a food image list screen GM6, which arranges the captured food images FG represented by the food image data 4116, according to each box CA.

[0384] Returning to the description of flowchart FF, the application execution unit 301 displays a screen related to the vegetable room 15 based on the touch operation (step SF5).

[0385] Next, the operation of the food management system 1000 when operating on the screen related to the vegetable room 15 will be explained.

[0386] When the application execution unit 301 touches the note save button B1, it sends information to the food management server 4 to update the note input presence / absence information 4119 and note information 4130. This information includes information indicating whether the operation object is the upper box image JCG or the lower box image GCG, the presence / absence of note input information 4119, and the note information 4130 indicating the entered note. Based on the received information, the food management server 4 appropriately updates the drawer food management record HR.

[0387] When the application execution unit 301 touches the note deletion button B2, it sends information to the food management server 4 to update the note input presence / absence information 4119 and note information 4130. This information includes whether the note to be deleted is the upper box image JCG or the lower box image GCG, and the note input presence / absence information 4119 indicating whether there is no note. Based on the received information, the food management server 4 appropriately updates the drawer food management record HR.

[0388] Additionally, as a use Figures 23-25 The illustrated drawer-style storage compartments include, for example, the vegetable compartment 15. However, Figures 23-25 The screen display and the operation of the food management system 1000 can also be performed in the same way regarding the freezer compartment 14. In this case, Figure 10 The database shown represents a database that stores various information related to the food in freezer compartment 14.

[0389] [1-3. Effects]

[0390] As described above, the camera unit 2 includes: a main body 201 disposed on the upper surface 10B of the cold storage 1; and a shooting component 202, including a camera 213 extending from the main body 201 toward the front of the cold storage 1 and shooting downwards from the front upper part of the cold storage 1. The main body 201 includes a position limiting member 205 that abuts against the front edge 10A of the cold storage 1 and limits the position of the shooting component 202 in the front-rear direction of the cold storage 1.

[0391] Therefore, by limiting the position of the camera 202 in the front-to-back direction of the cold storage 1 by the position limiting component 205, the camera 213 is positioned so that it can capture images from above the front of the cold storage 1. Thus, food entering and leaving the cold storage 1 can be included in the images captured by the camera 213 with high precision, and food entering and leaving the cold storage 1 can be captured with high precision.

[0392] The position limiting component 205 is a position limiting piece that extends downward from the bottom surface 206 of the main body 201 in the set state.

[0393] Therefore, the position of the camera unit 2 can be limited by the gap formed between the main body 10 and the door 11C. Thus, the camera unit 2 can be installed in the existing cold storage 1, so that food entering and leaving the cold storage 1 can be filmed with high precision in the existing cold storage 1.

[0394] The position limiting component 205 is formed to be shorter than the length of the gasket provided between the cold storage 1 and the door 11C of the cold storage 1.

[0395] Therefore, the position of the camera unit 2 can be limited by the gap formed between the main body 10 and the door 11C without hindering the cooling effect of the refrigerator compartment 11. Thus, food entering and leaving the refrigerator compartment 1 can be photographed with high precision without obstructing the cooling effect of the refrigerator compartment 11.

[0396] The position limiting component 205 extends in the width direction of the main body 201.

[0397] This ensures a larger contact area with the main housing 10, thus stabilizing the camera unit 2's orientation. Consequently, it enables more precise imaging of food entering and leaving the cold storage 1.

[0398] The position limiting component 205 has a mark 207 for adjusting the position of the camera unit 2 in the width direction of the cold storage 1.

[0399] Therefore, by setting mark 207 as a marker, user P can easily set up camera unit 2 in the left and right directions of cold storage 1. Thus, user P can easily set up camera unit 2 in a way that can capture food entering and leaving cold storage 1 with high precision.

[0400] The position limiting component 205 has a mark 207 at the center of the width direction of the main body 201.

[0401] Therefore, by setting mark 207 as a marker, user P can easily place camera unit 2 in the center of the cold storage 1 in the left-right direction. Thus, user P can easily set up camera unit 2 in a way that allows for more accurate imaging of food entering and leaving the cold storage 1.

[0402] Furthermore, as explained, the camera unit 2 includes: a camera 213 that captures images of the cold storage 1 from the front and above; a camera control unit 223 that, when a human sensor 212 detects the presence of a person in the vicinity of the cold storage 1, causes the camera 213 to start recording; and a recording control unit 224 that, at a predetermined time after the camera control unit 223 causes the camera 213 to start recording, begins recording the results of the camera 213's recording.

[0403] Furthermore, the control method of camera unit 2, when the human sensing sensor 212 detects the presence of a person in the vicinity of the cold storage 1, causes camera 213 to start recording, and at a predetermined time after camera 213 starts recording, it begins recording the recording results of camera 213.

[0404] According to camera unit 2 and its control method, by starting camera 213 to record when human sensor 212 detects the presence of a person in the vicinity of cold storage 1, the likelihood of including food entering and leaving cold storage 1 in the recording results of camera 213 before the door 11C or drawer 16A of cold storage 1 is opened can be increased. Therefore, food entering and leaving cold storage 1 can be recorded with high precision.

[0405] After the camera control unit 223 starts recording the camera 213, the recording control unit 224 starts recording the recording results of the camera 213 when the door 11C or drawer 16A of the cold storage 1 is open.

[0406] Therefore, since recording begins when the likelihood of food entering or leaving the cold storage is high, unnecessary recording can be prevented when food is not entering or leaving the cold storage 1. Furthermore, the recording control unit 224 can prevent the data size of the recording file 231 from increasing, thus preventing an increase in the number of items processed by the food inspection unit 225 and an increase in the processing time and workload of the food inspection unit 225.

[0407] After recording begins, the recording control unit 224 stops recording when the door 11C or drawer 16A of the open cold storage 1 is closed.

[0408] Therefore, since recording ends when the likelihood of food entering or leaving the warehouse is high, unnecessary recording can be prevented when food has not entered or left the cold storage 1. Furthermore, the recording control unit 224 can prevent the data size of the recording file 231 from increasing, thus preventing an increase in the number of items processed by the food inspection unit 225 and an increase in the processing time and workload of the food inspection unit 225.

[0409] When recording stops, the video control unit 224 generates a video file 231 showing the recording results of the door 11C or drawer 16A of the cold storage 1 from opening to closing.

[0410] Therefore, the video recording of the door 11C or drawer 16A of the cold storage 1 from opening to closing is generated as a video file 231. This prevents the data size of the video file 231 from increasing and allows for rapid generation of the video file 231. Furthermore, because the data size of the video file 231 can be prevented from increasing and a video file 231 can be generated rapidly, the food inspection unit 225 can quickly begin processing related to the video file 231 after food enters or leaves the cold storage 1.

[0411] After the recording control unit 224 stops recording, the camera control unit 223 causes the camera 213 to stop recording when the human sensor 212 detects that there are no people around the cold storage 1.

[0412] Therefore, since the camera 213 stops recording when the human detection sensor 212 detects that no one is around the cold storage 1, it is possible to prevent the camera 213 from recording when no food is entering or leaving the cold storage 1. Thus, it is possible to prevent increased power consumption and to record food entering and leaving the cold storage 1 with high precision.

[0413] The camera unit 2 includes a food detection unit 225 that detects food entering and leaving the cold storage 1 based on the recording results of the recording control unit 224.

[0414] Therefore, since the food can be detected based on the imaging results obtained by taking high-precision pictures of the food entering and leaving the cold storage 1, the food entering and leaving the cold storage 1 can be detected with high precision.

[0415] Furthermore, as explained, the food management application 311 uses the terminal processor 300 of the terminal device 3, which includes a touch panel 32, as the application execution unit 301. The application execution unit 301 displays an image RG inside the refrigerator compartment via the touch panel 32. In the displayed image RG inside the refrigerator compartment, a food image FG is displayed in the segmented area BA corresponding to the storage area of ​​the food stored in the refrigerator compartment 11.

[0416] In addition, the terminal device 3 includes a touch panel 32 and an application execution unit 301, which displays an image RG inside the refrigerator compartment via the touch panel 32. In the displayed image RG inside the refrigerator compartment, a food image FG is displayed in the segmented area BA corresponding to the storage area of ​​the food stored in the refrigerator compartment 11.

[0417] In addition, the control method of the terminal device 3 displays an image RG inside the refrigerator compartment. In the displayed image RG inside the refrigerator compartment, a food image FG is displayed in the segmented area BA corresponding to the storage area of ​​the food stored in the refrigerator compartment 11.

[0418] According to the food management application 311, terminal device 3, and control method of terminal device 3, the captured food image FG is displayed in the segmented area BA corresponding to the storage area of ​​the food stored in the cold storage within the image representing the interior of the cold storage. Therefore, user P can intuitively understand what kind of food is stored in the cold storage 11.

[0419] The food image FG is an image captured by the camera 203 used in the refrigerator.

[0420] Therefore, by providing user P with actual photographic images of the food stored in the cold storage 1, user P can more intuitively understand what kind of food is stored in the cold storage 11.

[0421] The image RG inside the cold storage room is an image of the model of the cold storage room 1 used by user P of terminal device 3.

[0422] Therefore, by providing user P with an image RG of the cold storage room 11 corresponding to the actual cold storage room 1 in use, user P can more intuitively understand what kind of food is stored in the cold storage room 11.

[0423] The image RG inside the cold storage room is segmented into multiple segmented regions BA. The number of segmented regions BA in the image RG inside the cold storage room is based on the number of models of cold storage room 1 used by user P of terminal device 3.

[0424] Therefore, user P can know what kind of food is stored in the storage area of ​​the cold storage compartment 11 of the actual cold storage 1. Thus, user P can more intuitively understand what kind of food is stored in the cold storage compartment 11.

[0425] When the application execution unit 301 displays multiple captured food images FG in a segmented region BA, it displays the multiple captured food images FG side by side.

[0426] Therefore, since multiple food images FG can be easily displayed in each segmented area BA, the user P can intuitively and easily grasp what kind of food is stored in the refrigerator compartment 11.

[0427] For a segmented region BA, there is an upper limit to the number of food images FG that can be displayed simultaneously.

[0428] This prevents reduced visibility when multiple food images (FG) are displayed in a single segmented area (BA). Therefore, user P can intuitively and easily understand what kind of food is stored in the refrigerator compartment (11).

[0429] If the number of captured food images FG that are displayed in a segmented region BA exceeds the upper limit, the application execution unit 301 can switch between displaying captured food images FG within the upper limit range.

[0430] This prevents reduced visibility when multiple food images FG are displayed in a segmented area BA, and allows the user P to visually confirm food images FG that cannot be fully displayed.

[0431] The application execution unit 301 receives annotation input from the user P of the terminal device 3 for each captured food image FG, and displays the annotation input icon MIC indicating that the annotation has been received in association with the captured food image FG.

[0432] Therefore, it is possible to determine which food item user P entered a note on. Thus, user P can intuitively understand what kind of food is stored in refrigerator compartment 11, and also know which food item a note was entered on.

[0433] Food management application 311 is an application that can be installed on terminal device 3.

[0434] Therefore, by installing the food management application 311, a terminal device 3 that does not have the function of displaying images RG inside the cold storage room and capturing images FG of food can become a terminal device 3 capable of displaying these images.

[0435] Furthermore, as explained, the food management application 311 uses the terminal processor 300 of the terminal device 3, including the touch panel 32, as an application execution unit 301 to display images captured by the drawer camera 204 that captures images of the drawer 16A of the refrigerator 1, which has an upper box JCA and a lower box GCA. The application execution unit 301 can display the upper box image JCG and the lower box image GCG, and receive annotation input from the user P of the terminal device 3 for each of the upper box image JCG and the lower box image GCG. When the application execution unit 301 receives annotation input from the user P for the upper box image JCG, it displays an annotation input icon MIC indicating that the annotation input has been received, associated with the upper box image JCG; when it receives annotation input from the user P for the lower box image GCG, it displays the annotation input icon MIC associated with the lower box image GCG.

[0436] Furthermore, the terminal device 3 includes a touch panel 32 and an application execution unit 301 that displays images captured by the drawer camera 204. The application execution unit 301 can display an upper box image JCG and a lower box image GCG, and receives annotation input from the user P of the terminal device 3 for each of the upper box image JCG and the lower box image GCG. When the application execution unit 301 receives annotation input from the user P for the upper box image JCG, it displays an annotation input icon MIC indicating that the annotation input has been received, associated with the upper box image JCG. When it receives annotation input from the user P for the lower box image GCG, it displays the annotation input icon MIC associated with the lower box image GCG.

[0437] Furthermore, the control method of the terminal device 3 can display the upper box image JCG and the lower box image GCG. For each of the upper box image JCG and the lower box image GCG, the terminal device 3 receives annotation input from the user P. When the user P receives annotation input for the upper box image JCG, the annotation input icon MIC, which indicates that the annotation input has been received, is displayed in association with the upper box image JCG. When the user P receives annotation input for the lower box image GCG, the annotation input icon MIC is displayed in association with the lower box image GCG.

[0438] Therefore, the food management application 311, the terminal device 3, and the control method of the terminal device 3 can display a captured image of the box CA in drawer 16A, and user P can input notes into the displayed captured image. Thus, user P can intuitively grasp the food stored in drawer 16A of the refrigerator 1, and user P's convenience is improved.

[0439] When the application execution unit 301 displays the upper cassette image JCG overlapping with the lower cassette image GCG, and receives an operation from the user P to move the upper cassette image JCG, it moves the first cassette image to display the lower cassette image GCG.

[0440] Therefore, the lower box image GCG can be displayed by moving the actual box CA. As a result, user P can more intuitively grasp the food stored in drawer 16A of the refrigerator 1, and the convenience of user P can be improved.

[0441] When the application execution unit 301 displays a note input icon MIC in association with the upper box image JCG, it moves the note input icon MIC together with the upper box image JCG.

[0442] Therefore, even when user P moves the upper box image JCG, the association between the upper box image JCG and the note input icon MIC can be maintained. Thus, even when user P moves the upper box image JCG, user P can accurately determine which image the note input icon MIC is associated with, improving user P's convenience.

[0443] The application execution unit 301 displays a switch button to switch the displayed captured image to either an upper-layer box image (JCG) or a lower-layer box image (GCG). By operating the switch button, the displayed captured image is switched to either an upper-layer box image (JCG) or a lower-layer box image (GCG).

[0444] Therefore, the convenience of the user P can be improved because the displayed box image CG can be easily switched by operating the switch button.

[0445] The upper box image JCG is an image taken when the upper box JCA is pulled out of the cold storage 1 above a predetermined threshold. The lower box image GCG is an image taken when the lower box GCA is pulled out of the cold storage 1 above a predetermined threshold.

[0446] Therefore, since as many images of the food as possible can be displayed as the box image CG, user P can intuitively grasp the food stored in drawer 16A of the refrigerator 1, and the convenience of user P can be improved.

[0447] Food management application 311 is an application that can be installed on terminal device 3.

[0448] Therefore, by installing the food management application 311, a terminal device 3 that does not have the function of displaying the box image CG and receiving the input of notes to the displayed box image CG can become a terminal device 3 with these functions.

[0449] (Implementation Method 2)

[0450] Next, Embodiment 2 will be described. In the description of Embodiment 2, the same reference numerals will be given to the constituent elements that are the same as the constituent elements of the food management system 1000 of Embodiment 1, and detailed descriptions will be omitted as appropriate.

[0451] [2-1. Structure]

[0452] Figure 26 This is a block diagram showing the structure of the terminal device 3 and the food management server 4 in embodiment 2.

[0453] Compare Figure 26 and Figure 7As can be seen, the server processor 400 in Embodiment 2 executes the control program stored in the server storage unit 410 and, in addition to the server communication control unit 401 and the information processing unit 402, also functions as a food identification unit 403.

[0454] The food recognition unit 403 performs food recognition processing based on the captured food image FG represented by the food image data 4116 included in the first update information and the second update information, respectively.

[0455] Food recognition processing involves identifying food from a captured food image (FG) and retrieving icons representing the same type of food from an icon database. This icon database stores an icon for each food item. Hereinafter, these icons will be referred to as food icons. The icon database can be stored on the food management server (4) or on an external device that the food management server (4) can communicate with.

[0456] In this embodiment, the food icon corresponds to an example of the food image of the present invention.

[0457] For example, the food identification unit 403 identifies food as follows.

[0458] The food recognition unit 403 determines the image with the highest consistency with the captured food image FG represented by the food image data 4116 included in the update information, based on images stored in the food image database. The food image database is a database that stores image data for each food item. The consistency is determined based on features such as shape and color. Then, the food recognition unit 403 identifies the food corresponding to the determined image as the food represented by the food image data 4116 included in the update information.

[0459] Furthermore, the food recognition unit 403 can recognize food using AI (artificial intelligence). For example, it can perform machine learning processing to learn features such as color and shape of the food from the trained photographed food image FG, and pre-build a learned model. This learned model is stored in the server storage unit 410. The food recognition unit 403 uses the food image data 4116 included in the update information as input data and refers to the learned model to recognize the food represented by the photographed food image FG. Here, the learned model is stored in a device other than the food management server 4 connected to the global network GN, and the structure of the food can be recognized in the food management server 4. In this configuration, the food recognition unit 403 sends the food image data 4116 included in the update information to the device storing the learned model and obtains the determination result of the food from the device. Then, the food recognition unit 403 identifies the food represented by the obtained determination result as the food represented by the photographed food image FG represented by the food image data 4116 included in the update information.

[0460] When identifying food, the food identification unit 403 retrieves the food icon of the identified food from the icon database. Furthermore, in Embodiment 2, the icon data of the food icons retrieved by the food identification unit 403 is appropriately stored, either in place of the food image data 4116 stored in the refrigerator food management record RR or together with it. Also in Embodiment 2, the icon data of the food icons retrieved by the food identification unit 403 is appropriately stored, either in place of the food image data 4116 stored in the drawer food management record HR or together with it.

[0461] Therefore, implementation method 2 can display food icons in the application UI320 instead of taking food images FG.

[0462] [2-2. Effects, etc.]

[0463] According to embodiment 2, it achieves the same effect as embodiment 1.

[0464] (Other implementation methods)

[0465] As described above, the embodiments disclosed in this application have been illustrated. However, the technology of the present invention is not limited thereto, and can be applied to embodiments with modifications, substitutions, additions, omissions, etc. Furthermore, new embodiments can be created by combining the constituent elements described in the above embodiments.

[0466] Therefore, other implementation methods are illustrated below.

[0467] In the above embodiments, a cold storage room 1 is exemplified as the storage room of the present invention. However, the storage room of the present invention is not limited to the cold storage room 1, and may be, for example, a warehouse set up in an office or the like. In the above embodiments, food is exemplified as the article of the present invention, but the article of the present invention is not limited to food, and may be other articles such as books, depending on the storage room.

[0468] In the above embodiment, the position limiting member 205 is shown to have a mark 207 at the center of the main body 201 in the left-right direction. However, the position of the mark 207 is not limited to the center of the main body 201 in the left-right direction. The position of the mark 207 can be other than the center of the main body 201 in the left-right direction. In this case, each mark is appropriately recorded in the central position recording paper CY, so that the camera unit 2, whose position is adjusted based on the mark 207, can properly perform various detections.

[0469] In the above embodiment, an example of the refrigerator compartment image of the present invention is an image of the refrigerator compartment 11 with the left door 11A and right door 11B open when viewed from the front, namely, the refrigerator compartment image RG. However, the refrigerator compartment image of the present invention is not limited to this refrigerator compartment image RG. It can be an image of the refrigerator compartment with one door open when viewed from the front, or it can be an image of the refrigerator compartment 1 in which a refrigerator compartment and a freezer compartment are included, with the door open when viewed from the front.

[0470] The shooting range of the drawer camera 204 can be set to the range of one or more arbitrary drawer-type storage rooms, depending on the type of drawer-type storage room displayed in the above screen of the application UI320.

[0471] In addition to storing the date and time, the Food Management System 1000 can also manage other elements such as remaining quantity and expiration date.

[0472] The main body 201 and the shooting component 202 of the camera unit 2 can be constructed separately.

[0473] The camera unit 2 can be configured to send the box image CG to the food management server 4 even when it is not pulled above a predetermined threshold. In this configuration, the food management server 4 stores the box image CG when it is not pulled above the predetermined threshold and provides it to the terminal device 3. Then, the application execution unit 301 displays the box image CG in the application UI 320 in a manner that allows it to be replaced by the upper box image JCG and the lower box image GCG respectively.

[0474] In the above embodiment, the case where user P sets up camera unit 2 in cold storage 1 is illustrated, but camera unit 2 can be shipped while set up in cold storage 1.

[0475] The configuration can be such that the server processor 400 is used as the food inspection unit 225, while the camera unit processor 220 is not used as the food inspection unit 225. In this configuration, the camera unit 2 sends the video recording file 231 to the food management server 4.

[0476] In the above embodiment, a human sensor 212 is exemplified as the sensor of the present invention. However, the sensor of the present invention is not limited to the human sensor 212, and may be an illumination sensor, an electrostatic sensor provided in the handle of the cold storage 1, etc. In the case of this electrostatic sensor, the cold storage 1 and the camera unit 2 can communicate, and the cold storage 1 sends the detection value of the electrostatic sensor to the camera unit 2.

[0477] The terminal device 3 can store the refrigerator food management record RR and the drawer food management record HR, while the food management application 311 manages various information related to the food detected by the camera unit 2. In this configuration, the application execution unit 301 updates the contents of each record stored in the terminal device 3 appropriately, just like the food management server 4 described above. Therefore, the food management system 1000 does not need to include a food management server 4, simplifying the system structure. Furthermore, in this configuration, the application execution unit 301 can function as a food recognition unit 403.

[0478] For example, the types of compartments formed in the main body 10 of the cold storage 1 are not limited to a cold storage compartment 11, an ice-making compartment 12, a fresh freezing compartment 13, a freezer compartment 14, and a vegetable compartment 15; other types of compartments may also be formed. In addition, the number of doors provided at the opening at the front of the cold storage compartment 11 may be one.

[0479] For example, drawers 14A and 15A may include more than two boxes. In this case, a certain box corresponds to the first box of the present invention, and the box below that box corresponds to the second box of the present invention.

[0480] For example, the camera unit processor 220, the terminal processor 300, and the server processor 400 can be composed of a single processor or multiple processors.

[0481] Figure 6 , Figure 7 and Figure 26 The parts shown are just one example, and there are no particular restrictions on the specific installation method. That is, it is not necessarily necessary to install hardware that corresponds independently to each part; of course, it is also possible to form a structure in which the functions of each part are implemented by a single processor executing a program. Furthermore, in the above embodiments, a part of the function implemented by software can be implemented as hardware, or a part of the function implemented by hardware can be implemented by software. In addition, the specific details of the configuration of the other parts of the camera unit 2, terminal device 3, and food management server 4 can be arbitrarily changed without departing from the spirit of the present invention.

[0482] In addition, for example, Figure 13 , Figure 14 , Figure 22 and Figure 25 The steps of the actions shown are divided according to the main processing content in order to make it easier to understand the actions of each part of the food management system 1000. This invention is not limited by the way or name of the processing units.

[0483] Furthermore, the above-described embodiments are examples of the technology of the present invention, and various changes, substitutions, additions, omissions, etc., can be made within the scope of the claims or their equivalents.

[0484] Industrial availability

[0485] As described above, the photographic apparatus and the control method of the photographic apparatus of the present invention can be used for photographing items entering and leaving a storage warehouse.

[0486] Explanation of reference numerals in the attached figures

[0487] 1. Cold storage warehouse (warehouse)

[0488] 2. Camera Unit (Photography Device)

[0489] 3. Terminal device

[0490] 4. Food Management Server

[0491] 10 Main box

[0492] 10A Front Edge

[0493] 10B upper surface

[0494] 11. Refrigeration compartment

[0495] 11A Left Door

[0496] 11B Right Door

[0497] 11C door

[0498] 12. Ice maker (drawer-style storage compartment)

[0499] Drawers 12A, 13A, 14A, 15A, and 16A

[0500] 13. Fresh Freezer Compartment (Drawer-style Storage)

[0501] 14. Freezer compartment (drawer-style storage compartment)

[0502] 15. Vegetable compartment (drawer-style storage compartment)

[0503] 32. Touch panel (display unit)

[0504] 201 Main Body

[0505] 202 Camera components

[0506] 203 Camera for cold storage

[0507] Camera for 204 drawers

[0508] 205 Position limiting component

[0509] 206 Bottom

[0510] 207 mark

[0511] 212 Human Sensing Sensor (Sensor)

[0512] 213 Cameras

[0513] 223 Camera Control Department

[0514] 224 Video Control Department

[0515] 225 Food Inspection Department (Item Inspection Department)

[0516] 231 Video Files

[0517] 300 Terminal Processor (Computer)

[0518] 301 Application Execution Unit (Display Control Unit)

[0519] 311 Food Management Applications (Programs, Applications)

[0520] 1000 Food Management System

[0521] BA (Database Analyzer) segmentation (region)

[0522] Food images captured by FG ​​and FG1 (food images)

[0523] GCA lower box (second box)

[0524] GCG lower-level box image (second box image)

[0525] JCA upper box (box 1)

[0526] JCG upper-level box image (box image 1)

[0527] MIC (Note Input Icon) (Note Input Image)

[0528] P user

[0529] Images of the interior of the RG cold storage room (interior images).

Claims

1. A photographic device, characterized in that, include: The main body is located on the upper surface of the cold storage. and The camera component extends from the main body towards the front of the cold storage, including a camera positioned above and in front of the cold storage to capture images of the area below. The main body includes a position limiting component that abuts against the front edge of the cold storage, thereby limiting the position of the imaging component in the front-rear direction of the cold storage. The position limiting component is a position limiting piece that extends downward from the bottom surface of the main body in the set state. The position limiting component is formed to a length shorter than the gasket located between the cold storage compartment and the door of the cold storage compartment.

2. The photographic apparatus as claimed in claim 1, characterized in that: The position limiting component extends in the width direction of the body.

3. The photographic apparatus as described in claim 2, characterized in that: The position limiting component has markings for adjusting the position of the photographic device in the width direction of the cold storage.

4. The photographic apparatus as claimed in claim 3, characterized in that: The position limiting component has the mark at the center of the width direction of the main body.

Images