Warehouse entry / exit management device, warehouse entry / exit management system, and warehouse entry / exit management method

Abstract

The warehouse-in / warehouse-out management device (30) includes: an imaging unit (31) that captures a plurality of images showing a posture of an object when the object is put into or taken out of a refrigerator (20) from above the refrigerator (20), the refrigerator (20) having a first storage structure and a second storage structure different in position in a vertical direction from the first storage structure, and including, in the plurality of images, a marker set in each of the first storage structure and the second storage structure as an imaging target; and a determination unit (37) that determines a storage position of the object based on whether the marker is hidden in the plurality of images.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a storage management device, a storage management system, and a storage management method. BACKGROUND

[0002] A technology related to a storage that stores an object has been conventionally proposed. A storage management system of a refrigerator is disclosed in Patent Literature 1. The storage management system disclosed in Patent Literature 1 judges the storage and retrieval of an object with respect to the refrigerator using a learning model. Such a learning model outputs a judgment result of the storage and retrieval when information obtained by a sensor provided to the refrigerator is input.

[0003] (Patent Literature)

[0004] (Patent Literature)

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2020-41761 SUMMARY

[0006] The present disclosure provides a storage management device, a storage management system, and a storage management method that can manage a storage position of an object in a storage.

[0007] A storage management device according to one embodiment of the present disclosure includes an imaging unit that captures a plurality of images showing a pattern of storage and retrieval of an object with respect to a storage, the storage having a first storage configuration and a second storage configuration different in position from the first storage configuration in a vertical direction, and including a photographing object that is a marker provided to each of the first storage configuration and the second storage configuration in the plurality of images; and a judging unit that judges a storage position of the object based on whether the marker is hidden in the plurality of images.

[0008] A storage management system according to one embodiment of the present disclosure includes an imaging unit that captures a plurality of images showing a pattern of storage and retrieval of an object with respect to a storage, the storage having a first storage configuration and a second storage configuration different in position from the first storage configuration in a vertical direction, and including a photographing object that is a marker provided to each of the first storage configuration and the second storage configuration in the plurality of images; and a judging unit that judges a storage position of the object based on whether the marker is hidden in the plurality of images.

[0009] One embodiment of the present disclosure relates to an inventory management method including: an imaging step of imaging a plurality of images showing a pattern when an object is put in or taken out of a storage, from above the storage, the storage having a first storage configuration and a second storage configuration different from the first storage configuration in a vertical direction, and including, in the plurality of images, a marker provided in each of the first storage configuration and the second storage configuration as an imaging target; and a determination step of determining a storage position of the object based on whether or not the marker is hidden in the plurality of images.

[0010] One embodiment of the present disclosure relates to an inventory management device, an inventory management system, and an inventory management method, which can manage a storage position of an object in a storage. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is a block diagram showing a functional configuration of an inventory management system of an embodiment.

[0012] Figure 2 is an appearance view of a refrigerator and an inventory management device included in an inventory management system of an embodiment.

[0013] Figure 3 is a flowchart showing an example 1 of an operation of an inventory management system of an embodiment.

[0014] Figure 4 is a diagram showing one example of a video imaged by an imaging section.

[0015] Figure 5 is a diagram showing one example of food management information.

[0016] Figure 6 is a diagram for explaining a determination method in a case where a food is put in an upper shelf.

[0017] Figure 7 is a diagram for explaining a determination method in a case where a food is put in a lower shelf.

[0018] Figure 8 is a diagram showing a video imaged by an imaging section in a state where a drawer of a refrigerator is pulled out.

[0019] Figure 9 is a sequence diagram showing an example 2 of an operation of an inventory management system of an embodiment.

[0020] Figure 10 is a diagram showing one example of a display screen showing contents of food management information.

[0021] Figure 11 is a sequence diagram showing an example 3 of an operation of an inventory management system of an embodiment.

[0022] Figure 12 is a drawing illustrating one example of a notification screen.

[0023] Figure 13 is a block diagram illustrating a functional configuration of a variation example of the warehouse management system.

[0024] Figure 14 is a drawing illustrating a setting example of a region mark.

[0025] Figure 15 is a drawing illustrating a state in which the food items packed in the storage containers are stored on the door-side shelves.

[0026] Figure 16 is a first drawing illustrating a state in which the door of the refrigerator is opened.

[0027] Figure 17 is a second drawing illustrating a state in which the door of the refrigerator is opened.

[0028] Figure 18 is a drawing illustrating one example of setting information. DETAILED DESCRIPTION

[0029] Embodiments will be specifically described below with reference to the drawings. Note that the embodiments described below are merely examples illustrating generality or specificity of the present disclosure. Values, shapes, materials, arrangement positions of constituent elements, connection modes, steps, orders of steps, and the like shown in the following embodiments are merely examples, and the gist of the present disclosure is not limited to them. Furthermore, among the constituent elements of the following embodiments, those not recited in the constituent elements of the independent technical means are described as arbitrary constituent elements.

[0030] Note that each drawing is a schematic view, and is not a strict illustration. Furthermore, in each drawing, the same symbols are given to substantially the same constituent elements, and there is a case where repeated description is omitted or simplified.

[0031] (Embodiment)

[0032] [Configuration]

[0033] First, the configuration of the warehouse management system of the embodiment will be described. Figure 1 is a block diagram illustrating a functional configuration of the warehouse management system of the embodiment.

[0034] Figure 1 The warehouse management system 10 illustrated is a system for managing food items stored in the refrigerator 20. The warehouse management system 10 is provided with the refrigerator 20, a warehouse management device 30, a server device 40, and an information terminal 50.

[0035] The refrigerator 20 is an example of a storage warehouse, is provided in a user's home or the like, and is used to refrigerate food. Figure 2 is an external view of the refrigerator 20 (and the in-out warehouse management device 30). As shown in Figure 2 , the refrigerator 20 is provided with a main body portion 21, a door 22 that is used when storing a first storage chamber 23 inside the main body portion 21, a plurality of shelves 24 provided in the first storage chamber 23, and a drawer 25. When the door 22 is opened, the refrigerator 20 becomes a state in which food can be placed on each of the plurality of shelves 24 (i.e., the food is stored in the first storage chamber 23). Also, when the door 22 is opened, the refrigerator 20 becomes a state in which food can be stored in a door-side shelf 22a provided on the inner side of the door 22. Also, when the drawer 25 is opened, the refrigerator 20 becomes a state in which food can be stored in a second storage chamber 26 inside the drawer 25. In Figure 2 , the refrigerator 20 is provided with two doors 22 that cover the first storage chamber 23, but the number of doors 22 that cover the first storage chamber 23 can also be one.

[0036] The in-out warehouse management device 30 is mounted on, for example, the upper surface of the refrigerator 20, and performs information processing for managing food stored in the refrigerator 20. The in-out warehouse management device 30 is attached to, for example, an existing refrigerator 20, but can also be built into the refrigerator 20. In other words, the in-out warehouse management device 30 can be a device formed separately from the refrigerator 20, or can be a device manufactured integrally with the refrigerator 20. As shown in Figure 1 , the in-out warehouse management device 30 is provided with an imaging portion 31, a distance measuring portion 32, an information processing portion 33, a storage portion 34, a communication portion 35, and a notification portion 36.

[0037] The imaging portion 31 captures a video showing the appearance of the refrigerator 20 when food is put in or taken out of the refrigerator 20 from above the refrigerator 20. In other words, the imaging portion 31 captures, for example, a video of the refrigerator 20 in a state in which the door 22 is opened from above the refrigerator 20. The imaging object included in the video is the door 22, the door-side shelf 22a, the first storage chamber 23, the shelves 24, the drawer 25, and the floor of the indoor space in front of the refrigerator 20, or the like. The imaging portion 31 is implemented by, for example, a camera having a fisheye lens, but can also be implemented by a camera having a normal lens.

[0038] In addition, the imaging portion 31 can capture the appearance of the refrigerator 20 when an object is put in or taken out of the refrigerator 20 at a predetermined time interval to generate a plurality of images (still images), and it is not necessary to necessarily capture a video. Furthermore, the imaging portion 31, although mounted on the frame of the in-out warehouse management device 30, can be mounted on the refrigerator 20 separately from the frame of the in-out warehouse management device 30. In other words, the imaging portion 31 can be attached to the refrigerator 20. In this case, as a configuration that can be attached to the refrigerator 20, the imaging portion 31 has a magnet, a suction cup, or a clip, or the like.

[0039] The distance measuring section 32 measures the distance from the distance measuring section 32 to the door 22 at the upper portion of the refrigerator 20. In other words, the distance measuring section 32 measures the opening degree of the door 22. The distance measuring section 32 is implemented by, for example, an infrared distance measuring sensor. The distance measuring section 32 can measure the distance from the distance measuring section 32 to the right door 22 and the distance from the distance measuring section 32 to the left door 22, respectively.

[0040] In addition, as described later, the distance measuring section 32 is used to determine the opening and closing of the door 22. The entrance / exit management device 30 can also be provided with another sensor such as an angle sensor for determining the opening and closing of the door 22 instead of the distance measuring section 32. Further, the sensor for determining the opening and closing of the door 22 can be provided by the refrigerator 20, in which case the entrance / exit management device 30 can not be provided with the sensor for determining the opening and closing of the door 22.

[0041] The information processing section 33 performs information processing for managing the food items stored in the refrigerator 20. The information processing section 33 is implemented by, for example, a microcomputer, but can also be implemented by a processor or a dedicated circuit. As a functional component, the information processing section 33 has a determination section 37, a tracking section 38, and a setting section 39. The functions of the determination section 37, the tracking section 38, and the setting section 39 are implemented by, for example, a microcomputer or the like executing a computer program stored in the storage section 34, which constitutes the information processing section 33. Details of the functions of the determination section 37, the tracking section 38, and the setting section 39 are described later.

[0042] The storage section 34 is a storage device for storing a computer program executed by the information processing section 33 and various information (setting information and food management information described later, etc.) required for the above information processing. The storage section 34 is implemented by, for example, a semiconductor memory.

[0043] The communication section 35 is a communication circuit for the entrance / exit management device 30 to communicate with the server device 40 via the wide area communication network 60. The communication section 35 is, for example, a wireless communication circuit that performs wireless communication, but can also be a wired communication circuit that performs wired communication. The communication standard with which the communication section 35 communicates is not particularly limited.

[0044] The notification section 36 notifies when the category of the food item put in or taken out of the refrigerator 20 cannot be determined. The notification section 36 is implemented by, for example, a speaker or a buzzer, etc., to notify in the form of sound, but can also be implemented by a light emitting element such as an LED (Light Emitting Diode) to notify in the form of light emission.

[0045] The server device 40 is a computer located outside the facility in which the refrigerator 20 is installed, specifically, a cloud server. The server device 40 performs information processing for presenting information about food stored in the refrigerator 20. Further, the server device 40 is an example of a notification device, and performs information processing for notifying information about food stored in the refrigerator 20. The server device 40 includes a communication section 41, an information processing section 42, and a storage section 43.

[0046] The communication section 41 is a communication circuit for the server device 40 to communicate with the warehouse management device 30 and the information terminal 50 via the wide area communication network 60. The communication section 41 is, for example, a wired communication circuit that performs wired communication, but can also be a wireless communication circuit that performs wireless communication. The communication standard with which the communication section 41 communicates is not particularly limited.

[0047] The information processing section 42 performs information processing for presenting or notifying information about food stored in the refrigerator 20. The information processing section 42 is realized by, for example, a microcomputer, but can also be realized by a processor or a dedicated circuit. As a functional component, the information processing section 42 has a presentation section 44 and a notification section 45. The functions of the presentation section 44 and the notification section 45 are realized by, for example, a microcomputer or the like executing a computer program stored in the storage section 43, which constitutes the information processing section 42. Details of the functions of the presentation section 44 and the notification section 45 will be described later.

[0048] The storage section 43 is a storage device for storing a computer program executed by the information processing section 42 and various information required for the above information processing. The storage section 43 is realized by, for example, a semiconductor memory.

[0049] The information terminal 50 is an information terminal possessed by a user. The user uses the information terminal 50 to receive a notification from the server device 40. The information terminal 50 is, for example, a portable information terminal such as a smartphone or a tablet terminal, but can also be a desktop information terminal such as a personal computer.

[0050] [Action Example 1]

[0051] Next, as Action Example 1 of the warehouse management system 10, the management action of the storage or withdrawal of food will be described. Figure 3 is a flowchart showing Action Example 1 of the warehouse management system 10.

[0052] First, the determination section 37 starts the imaging section 31 and starts video imaging by the imaging section 31 when it determines that at least one of the two doors 22 is open based on the measurement result of the distance measuring section 32 (S11).Figure 4 is an example of a video captured by the imaging section 31. As shown in Figure 4 , the imaging range of the imaging section 31 includes the main body 21 (the first storage chamber 23), the inner sides of the two doors 22 (the door-side shelves 22a), and the upper shelf 24a and the lower shelf 24b provided in the first storage chamber 23. In addition, in Figure 4 , a boundary line (dotted line) used in the in-storage determination or the out-of-storage determination of the food is also illustrated. This boundary line is not a line captured by the imaging section 31, but a virtual line.

[0053] Next, the tracking section 38 detects the food appearing in the video (S13), and tracks the detected food (S14). The detection of the food can use various existing algorithms such as R-CNN (Regions with Convolutional Neural Networks). The tracking section 38, for example, also detects the food in the next frame (hereinafter also referred to as a current frame) of the frame (hereinafter also referred to as a previous frame) in which the food detected in step S13 (hereinafter also referred to as a tracking target) appears. The tracking section 38 decides the moving destination of the tracking target as the food appearing in the current frame at a position closest to the position of the tracking target in the previous frame. By repeatedly performing such processing, the tracking section 38 can track the food detected in step S13.

[0054] In addition, the tracking section 38 can further extract an image feature amount of the food detected in step S13, and track the food using the image feature amount as auxiliary information. The tracking section 38, for example, can also decide the moving destination of the tracking target in accordance with the distance between the food detected in the current frame and the tracking target, and the similarity between the image feature amount detected in the current frame and the tracking target. Thus, the tracking accuracy of the food can be improved.

[0055] Furthermore, such a food tracking method is one example, and other existing algorithms can be used for the tracking of the food.

[0056] Next, the determination section 37 performs the in-storage determination or the out-of-storage determination of the food (S15). The determination section 37 determines that the food being tracked in the video has been taken out in a case where the food moves from the refrigerator 20 side to the indoor side and crosses the boundary line. Also, the determination section 37 determines that the food being tracked in the video has been put in in a case where the food moves from the indoor side to the refrigerator 20 side and crosses the boundary line. In addition, the boundary line is decided in advance on the basis of experience or experiment.

[0057] Next, the determination section 37 determines the category of the food put in or taken out (S16). The determination section 37 determines the category of the food, for example, using a machine learning model. The category of the food is, for example, a beverage, a vegetable, a meat food, or the like, but can be a more detailed category than these. In addition, the category of the food can be determined at the time of detection of the food in step S13 as well, in which case, step S16 can be omitted.

[0058] Next, the determination section 37 determines the storage position of the food (S17). As shown in FIG. 6, the determination section 37 determines the storage position of the food based on whether the plurality of first marks 27a, the plurality of second marks 27b, and the plurality of third marks 27c are hidden in the video. The plurality of first marks 27a are provided on the upper shelf 24a, the plurality of second marks 27b are provided on the lower shelf 24b, and the plurality of third marks 27c are provided on the bottom of the door-side shelf 22a. Details of the method of determining the storage position of the food in step S17 will be described later. Figure 4

[0059] Next, the determination section 37 updates the food management information stored in the storage section 34 based on the results of the determinations in steps S15 to S17 (S18). Figure 5 FIG. 7 is a diagram showing one example of food management information. The food management information is information showing detailed contents of the food stored in the refrigerator 20. As shown in FIG. 7, the food management information shows, for example, the category and the number of the food at each of the plurality of storage positions. The determination section 37, for example, in the case where it is determined in steps S15 to S17 that a beverage is taken out from the upper shelf 24a, decreases the number of the beverage on the upper shelf 24a in the food management information by 1. Also, the determination section 37, in the case where it is determined in steps S15 to S17 that a vegetable is put in the lower shelf 24b, increases the number of the vegetable on the lower shelf 24b in the food management information by 1. Figure 5

[0060] Thereafter, when it is determined by the determination section 37 that both of the doors 22 are closed based on the measurement result of the distance measuring section 32 (S19), the operation ends.

[0061] As described above, the warehouse management system 10 (the warehouse management device 30) can determine the putting and taking of the food with respect to the refrigerator 20 (the food is put in the refrigerator 20 or the food is taken out of the refrigerator 20) based on whether the food imaged in the video has crossed the set boundary line.

[0062] ​​Further, in a case where the determination of the category of the food in step S16 fails, in other words, in a case where the category of the food cannot be determined, the determination section 37 can notify that the category of the food cannot be determined by the notification section 36. For example, in a case where the notification section 36 is implemented by a speaker, the notification section 36 outputs an audio message or the like showing that the category of the food cannot be determined. In a case where the notification section 36 is implemented by a light emitting element, the notification section 36 notifies that the category of the food cannot be determined by emitting light.

[0063] In this way, if the case where the determination of the category of the food fails is notified, the user can take measures such as manual modification of the food management information.

[0064] [Details 1 of the determination method of the storage position]

[0065] Next, details of the determination method of the storage position of the food in the above step S17 will be described. First, the first marker 27a, the second marker 27b, and the third marker 27c will be described.

[0066] The first marker 27a is a two-dimensional code provided on the upper surface of the upper shelf 24a of the first storage compartment 23. A plurality of first markers 27a are arranged along the width direction of the refrigerator 20 on the user side (front side of the refrigerator 20) of the shelf 24a. The determination section 37 can recognize the plurality of first markers 27a as digital data by image processing of the video. The plurality of first markers 27a show, for example, mutually different digital data, but can also show the same digital data.

[0067] The second marker 27b is a two-dimensional code provided on the upper surface of the lower shelf 24b of the first storage compartment 23. A plurality of second markers 27b are arranged along the width direction of the refrigerator 20 on the user side (front side of the refrigerator 20) of the shelf 24b. The determination section 37 can recognize the plurality of second markers 27b as digital data by image processing of the video. The plurality of second markers 27b show, for example, mutually different digital data, but can also show the same digital data.

[0068] The third marker 27c is a two-dimensional code provided at the bottom of the door side shelf 22a. One or a plurality of third markers 27c can be provided in one door side shelf 22a. The judging section 37 performs image processing on the video, and thereby can recognize a plurality of third markers 27c as digital data. The plurality of third markers 27c, for example, show mutually different digital data. For example, the third marker 27c can also show digital data that differs for each door 22. In other words, the third marker 27c provided in the right door 22 and the third marker 27c provided in the left door 22 can show different digital data. Further, the third marker 27c can also show digital data that differs for each door side shelf 22a.

[0069] The arrangement positions of the plurality of first markers 27a, the plurality of second markers 27b, and the plurality of third markers 27c are stored in advance in the storage section 34.

[0070] Next, a method of judging the storage position of the food according to whether the plurality of first markers 27a, the plurality of second markers 27b, and the plurality of third markers 27c are hidden will be described. Figure 6 is a view for explaining a judging method in a case where the food is put in the upper shelf 24a. Figure 6 is a view showing a video captured by the imaging section 31.

[0071] As shown in Figure 6 In the case where the food is put in the upper shelf 24a, at least one of the plurality of first markers 27a provided in the upper shelf 24a is temporarily hidden by the food and cannot be seen. Then, the judging section 37 judges that the storage position of the food is on the shelf 24a in a case where the first marker 27a is hidden at the time of storage of the food. The judging section 37 judges that the storage position of the food stored is on the shelf 24a in accordance with the fact that the first marker 27a cannot be seen during the process of storage of the food even if the first marker 27a can be seen after the food is placed on the shelf 24a. The judging section 37 judges that the storage position of the food is on the shelf 24a when the first marker 27a is hidden regardless of whether the second marker 27b is hidden. Further, the judging section 37 can judge which area on the shelf 24a the food is stored in in accordance with which of the plurality of first markers 27a is hidden.

[0072] Although not shown, the same applies to a case where the food is taken out from the upper shelf 24a. The judging section 37 judges that the storage position of the food is on the shelf 24a in a case where the first marker 27a is hidden at the time of withdrawal of the food. The judging section 37 judges that the storage position of the food withdrawn is on the shelf 24a in accordance with the fact that the first marker 27a cannot be seen during the process of withdrawal of the food.

[0073] On the other hand,Figure 7 This diagram illustrates the method for determining the situation when food is placed on the lower shelf 24b. Figure 7 This is a diagram showing the video captured by camera unit 31. (Example) Figure 7 As shown, when food is placed on the lower shelf 24b, multiple first marks 27a on the upper shelf 24a are visible, while any one of the multiple second marks 27b on the lower shelf 24b is at least temporarily obscured by the food. Therefore, if the first marks 27a are not obscured and the second marks 27b are obscured when the food is stored, the determination unit 37 determines that the food is stored on the shelf 24b. Even if the second marks 27b are visible after the food is placed on the shelf 24b, the determination unit 37 considers that the second marks 27b were not visible during the food storage process, and thus determines that the stored food is stored on the shelf 24b. Furthermore, the determination unit 37 can also determine which area of ​​the shelf 24b the food is stored in based on which of the multiple second marks 27b is obscured.

[0074] Although not illustrated, the same applies when food is removed from the lower shelf 24b. If the first mark 27a is not obscured and the second mark 27b is obscured during food removal, the determination unit 37 determines that the food is stored on shelf 24b. The determination unit 37 also determines that the removed food is stored on shelf 24b because the second mark 27b is not visible during the food removal process.

[0075] Next, the method for determining the storage location when food is placed on the door-side shelf 22a will be explained. When food is placed on the door-side shelf 22a, the third mark 27c provided on that shelf is obscured by the food and cannot be seen. Therefore, when the third mark 27c is obscured by the food during storage, the determination unit 37 determines that the stored food is located on the door-side shelf 22a corresponding to the obscured third mark 27c. As described above, when the third mark 27c displays different numerical data for each door 22 or for each door-side shelf 22a, the determination unit 37 can determine the storage location of the food in more detail.

[0076] On the other hand, when food is removed from the door-side shelf 22a, the third mark 27c that was previously hidden by the food appears on the door-side shelf 22a. Therefore, when the third mark 27c appears when the food is removed from the warehouse, the determination unit 37 determines that the food is stored on the door-side shelf 22a corresponding to the third mark 27c.

[0077] As explained above, the judging section 37 judges the storage position of the food item based on whether the mark is hidden in the video. Specifically, the judging section 37 can judge which of the shelf 24a, the shelf 24b, and the door-side shelf 22a the storage position of the food item is in.

[0078] In addition, in a case where the mark is hidden in an unexpected manner, there can be a case where the judging section 37 fails to judge the storage position. Thus, the reporting section 36 can report when the judging of the storage position of the food item fails in step S17. Through such reporting, the user can take measures such as manually modifying the food management information.

[0079] [Details 2 of the judging method of the storage position]

[0080] The judging section 37 can also judge whether the storage position of the food item is in the first storage compartment 23 covered by the door 22 or in the second storage compartment 26 inside the drawer 25. Figure 8 is a view showing a video of the drawer 25 of the refrigerator 20 pulled out, which is captured by the imaging section 31.

[0081] As shown in Figure 8 , the fourth marks 27d are two-dimensional codes provided on the upper surface of the front plate of the drawer 25. The plurality of fourth marks 27d are arranged along the width direction of the refrigerator 20. The judging section 37 can recognize the plurality of fourth marks 27d as digital data by image processing of the video. The plurality of fourth marks 27d, for example, show mutually different digital data, or can show the same digital data.

[0082] The judging section 37, for example, judges whether the storage position of the food item is in the first storage compartment 23 or in the second storage compartment 26 based on whether the plurality of fourth marks 27d and the plurality of second marks 27b are hidden. In the following description of the judging method, the plurality of second marks 27b are described as being provided on the bottom (more specifically, the bottom surface) of the first storage compartment 23. The second marks 27b can be provided near the bottom of the first storage compartment 23, for example, as described above, on the shelf 24b.

[0083] For example, in a case where the food is put into the first storage chamber 23, any one of the plurality of second marks 27b provided on the bottom surface of the first storage chamber 23 is at least temporarily hidden by the food and cannot be seen. Then, the judging section 37 judges that the storage position of the food is in the first storage chamber 23 in a case where the second marks 27b are hidden at the time of storage of the food. Even if the second marks 27b can be seen after the food is stored in the first storage chamber 23, the judging section 37 judges that the storage position of the stored food is in the first storage chamber 23 according to the fact that the second marks 27b cannot be seen during the process of storage of the food. Whether or not the fourth marks 27d are hidden, the judging section 37 judges that the storage position of the food is in the first storage chamber 23 when the second marks 27b are hidden. Further, the judging section 37 can judge (speculate) which region in the first storage chamber 23 the storage position of the food is in, according to which of the plurality of second marks 27b is hidden.

[0084] Also, the same applies to a case where the food is taken out of the first storage chamber 23. The judging section 37 judges that the storage position of the food is in the first storage chamber 23 in a case where the second marks 27b are hidden at the time of withdrawal of the food. The judging section 37 judges that the storage position of the withdrawn food is in the first storage chamber 23 according to the fact that the second marks 27b cannot be seen during the process of withdrawal of the food.

[0085] On the other hand, in a case where the food is put into the second storage chamber 26, the plurality of second marks 27b are in a state where they can be seen, and any one of the fourth marks 27d is at least temporarily hidden by the food and cannot be seen. Then, the judging section 37 judges that the storage position of the food is in the second storage chamber 26 in a case where the second marks 27b are not hidden and the fourth marks 27d are hidden at the time of storage of the food. The judging section 37 judges that the storage position of the stored food is in the second storage chamber 26 according to the fact that the fourth marks 27d cannot be seen during the process of storage of the food, even if the fourth marks 27d can be seen after the food is stored in the second storage chamber 26. Further, the judging section 37 can judge (speculate) which region in the second storage chamber 26 the storage position of the food is in, according to which of the plurality of fourth marks 27d is hidden.

[0086] Also, the same applies to a case where the food is taken out of the second storage chamber 26. The judging section 37 judges that the storage position of the food is in the second storage chamber 26 in a case where the second marks 27b are not hidden and the fourth marks 27d are hidden at the time of withdrawal of the food. The judging section 37 judges that the storage position of the withdrawn food is in the second storage chamber 26 according to the fact that the fourth marks 27d cannot be seen during the process of withdrawal of the food.

[0087] As explained above, the judging section 37 can judge whether the storage position of the food is in the first storage chamber 23 or in the second storage chamber 26 according to whether the marks are hidden in the video.

[0088] [Action Example 2]

[0089] Next, as Action Example 2 of the stock management system 10, an action of prompting the food stored in the refrigerator 20 will be described. Figure 9 is a sequence chart showing Action Example 2 of the stock management system 10.

[0090] The communication section 35 of the stock management device 30 transmits the food management information to the server device 40 (S21). The communication section 41 of the server device 40 receives the food management information, and the prompting section 44 stores the received food management information in the storage section 43 (S22). The communication section 35 of the stock management device 30 transmits the food management information to the server device 40, for example, every time the stock-in or stock-out of the food is judged. Or the communication section 35 of the stock management device 30 transmits the food management information to the server device 40 every time the open door 22 is closed. Thus, the latest food management information is stored in the storage section 43 of the server device 40. In addition, the food management information or the like can be transmitted from the stock management device 30 to the server device 40 periodically at a prescribed time.

[0091] On the other hand, when the user performs a prescribed operation for confirming the food (the content of the food management information) stored in the refrigerator 20 at an outing place or the like, the information terminal 50 accepts the prescribed operation (S23), and transmits a prompt request to the server device 40 according to the accepted prescribed operation (S24).

[0092] The communication section 41 of the server device 40 receives the prompt request. The prompting section 44 transmits prompt information for prompting the content of the food management information stored in the storage section 43 to the information terminal 50 according to the received prompt request (S25). More specifically, the transmission of the prompt information is performed by the communication section 41.

[0093] The information terminal 50 receives the prompt information, and displays a display screen showing the content of the food management information on the display section provided in the information terminal 50 according to the received prompt information (S26). Figure 10 is a diagram showing one example of the display screen showing the content of the food management information.

[0094] Thus, the user of the stock management system 10 can confirm the food stored in the refrigerator 20 at home from an outing place or the like.

[0095] [Action Example 3]

[0096] Next, as Action Example 3 of the stock management system 10, a notification action when the food is put in or taken out of the refrigerator 20 while the user is not at home will be described.Figure 11 is a sequence chart showing an example 3 of the action of the warehouse entry / exit management system 10.

[0097] The communication section 35 of the warehouse entry / exit management apparatus 30 transmits the warehouse entry / exit information to the server apparatus 40, for example, at each time when the food storage or the food withdrawal is judged (S31). The warehouse entry / exit information is an example of information showing that the food is put into or taken out of the refrigerator 20.

[0098] The communication section 41 of the server apparatus 40 receives the warehouse entry / exit information, and the notification section 45 judges whether or not the user is away from home (S32). In other words, the notification section 45 judges whether or not the warehouse entry / exit information is received during the period when the user is not around the refrigerator 20.

[0099] For example, the information terminal 50 has a position measuring section such as a GPS (Global Positioning System) module, and transmits the position information to the server apparatus 40 periodically. In this case, the notification section 45 judges whether or not the user is away from home based on the received position information. In addition, the position (coordinates) of the user's home is registered in the storage section 43 in advance.

[0100] Further, the user can operate the information terminal 50 to report to the server apparatus 40 that he or she is away from home, instead of the method using the position information. In this case, the notification section 45 can consider that the user is away from home based on the above-mentioned report, and act in the away mode.

[0101] The notification section 45 transmits the notification information to the information terminal 50 when it is judged that the user is away from home (S33). More specifically, the transmission of the notification information is performed by the communication section 41.

[0102] The information terminal 50 receives the notification information, and displays the notification screen based on the received notification information (S34). Figure 12 is an example of the notification screen. On the other hand, in the case where it is judged in the step S32 that the user is at home (not away from home), the processing of the step S33 and the step S34 is not performed.

[0103] Thus, the warehouse entry / exit management system 10 notifies the information terminal 50 possessed by the user when the warehouse entry / exit information showing that the object is put into or taken out of the refrigerator 20 is received during the period when the user is not around the refrigerator 20. Accordingly, the user can recognize that something such as putting or taking out the food into or from the refrigerator 20 has occurred during the period when he or she is not around the refrigerator 20.

[0104] [Modified Example 1]

[0105] In the above embodiment, the processing from the detection of the food (step S13) to the update of the food management information (step S18) is performed by the warehouse-in / out management device 30, but can be performed by the server device 40. Figure 13 is a block diagram showing the functional configuration of the warehouse-in / out management system of this modification example 1.

[0106] As shown in Figure 13 , the warehouse-in / out management system 10a is provided with the refrigerator 20, the warehouse-in / out management device 30a, the server device 40a, and the information terminal 50. The warehouse-in / out management system 10a differs from the warehouse-in / out management system 10 not only in being provided with the warehouse-in / out management device 30a but also in the server device 40a (more specifically, the information processing section 42) being provided with the judging section 46 and the tracking section 47. The judging section 46 is a constitutional element having substantially the same function as the judging section 37, and the tracking section 47 is a constitutional element having substantially the same function as the tracking section 38.

[0107] In this warehouse-in / out management system 10a, the server device 40a is able to perform the processing from the detection of the food (step S13) to the update of the food management information (step S18) by obtaining the measurement result of the distance measuring section 32 and the video and the like taken by the camera section 31 from the warehouse-in / out management device 30a.

[0108] Alternatively, part of the processing from the detection of the food (step S13) to the update of the food management information (step S18) can be performed by the warehouse-in / out management device 30a, and the other part can be performed by the server device 40a.

[0109] [Modification Example 2]

[0110] In the above embodiment, the storage position of the food in the up-down direction is judged based on the first marker 27a provided to the shelf 24a, the second marker 27b provided to the shelf 24b, and the fourth marker 27d provided to the drawer 25. However, instead of the markers, the edges of the shelf 24a, the edges of the shelf 24b, and the edges of the drawer 25 can be used to judge the storage position of the food in the up-down direction. The edges here are the edges of the refrigerator 20 on the opening portion side (close to the user standing in front of the refrigerator 20).

[0111] The edge is detected (set) by the image processing of the video performed by the setting section 39, for example. Any algorithm can be used in the detection of the edge. Further, the edge (more specifically, the position of the pixel corresponding to the edge in the video) can also be set by the user after the determination of the positional relationship between the stock management device 30 (the imaging section 31) and the refrigerator 20. In this case, the information terminal 50 is used as the user interface, for example. In addition, the information terminal 50 is described as communicating with the stock management device 30 via the wide area communication network 60, but can communicate with the stock management device 30 via a local communication network in the case of being used as the user interface. Further, the information processing relating to the detection (setting) of the edge is performed by the setting section 39.

[0112] The method of judging the storage position of the food in the up-down direction using the edge can be the same as the method described in the above embodiment. The "mark" in the above embodiment can be appropriately replaced with the "edge".

[0113] In other words, the stock management device 30 includes the imaging section 31 that captures a plurality of images showing the outline of the refrigerator 20 from above the storage compartment, the refrigerator 20 having a first storage structure and a second storage structure different in position in the up-down direction from the first storage structure, and including, in the plurality of images, an imaging object that is an edge provided on each of the first storage structure and the second storage structure on the side of the opening of the refrigerator 20; and the judging section 37 that judges the storage position of the object based on whether the edge is hidden in the plurality of images. Each of the first storage structure and the second storage structure is, for example, any one of the shelves 24a, 24b, and the drawer 25.

[0114] Such a stock management device 30 can manage the storage position of the object in the refrigerator 20.

[0115] [Modified Example 3: Summary]

[0116] The stock management system 10 (stock management device 30) can also set a boundary line along the outline of the main body 21 and the door 22 in the video, and judge the storage and removal of the food in the refrigerator 20 (the food is stored in the refrigerator 20 or the food is removed from the refrigerator 20) based on whether the food shown in the video crosses the set boundary line. The stock management system 10 can improve the accuracy of the judgment of the storage and removal of the food in the refrigerator 20 by setting the boundary line along the outline of the main body 21 and the door 22, compared to the case where a boundary line independent of the outline is set. The stock management system 10 can particularly correctly judge the storage and removal of the food in the door shelf 22a. In addition, the boundary line is determined in advance based on experience or experiment.

[0117] Similarly, the warehouse entry / exit management system 10 (the warehouse entry / exit management apparatus 30) can divide the bottom surface of the door-side shelf 22a provided on the inner side of the two doors 22 in the video into prescribed regions and set region markers 22a1, 22a2, 22a3. Figure 14 is a diagram illustrating an example of setting of the region markers 22a1, 22a2, 22a3. Also, although the case in which the bottom of the door-side shelf 22a is divided into three is illustrated in Figure 14 , the present application is not limited thereto.

[0118] Further, the warehouse entry / exit management system 10 (the warehouse entry / exit management apparatus 30) can set the region markers 22a1, 22a2, 22a3 obtained by dividing the bottom surface of the door-side shelf 22a provided on the inner side of the two doors 22 into prescribed regions in advance, but can also be set by the user. In the case of being set by the user, the information terminal 50 is used as a user interface, for example. Also, the information terminal 50 is described as communicating with the warehouse entry / exit management apparatus 30 via the wide area communication network 60, but in the case of being used as a user interface, the warehouse entry / exit management apparatus 30 can communicate via a local communication network. The information processing relating to the setting of the region markers 22a1, 22a2, 22a3 is performed by the setting section 39. Also, the region markers 22a1, 22a2, 22a3 are not markers photographed by the imaging section 31, but are virtual markers.

[0119] An example of the operation using the region markers 22a1, 22a2, 22a3 will be described below. As with steps S11 and S12 of Action Example 1, the judging section 37 activates the imaging section 31 and starts the video photographing of the imaging section 31 when it judges from the measurement result of the distance measuring section 32 that at least one of the two doors 22 is opened. Here, the distance measuring section 32 is implemented by an infrared distance measuring sensor, for example. The distance measuring section 32 can measure the distance from the distance measuring section 32 to the right door 22 and the distance from the distance measuring section 32 to the left door 22, respectively.

[0120] The above Figure 14 is also a diagram illustrating one example of the video photographed by the imaging section 31, as illustrated in Figure 14 , the imaging range of the imaging section 31 includes the main body section 21 (the first storage chamber 23) and the inner sides of the two doors 22 (the door-side shelves 22a). Also, a boundary line (dotted line) for the entry judgment of the food or the exit judgment of the food is illustrated together. This boundary line is not a line photographed by the imaging section 31, but is a virtual line.

[0121] Next, as with step S17 of Action Example 1, the judging section 37 judges the storage position of the food. As Figure 14As shown, the judgment unit 37 determines the storage location of the food based on whether the multiple area marks 22a1, 22a2, and 22a3 set at the bottom of the door side shelf 22a shown in the video are covered (whether the food overlaps on the area marks).

[0122] Figure 15 This diagram shows the state in which food 70, contained in a storage container, is stored on the door-side shelf 22a. Three area markers 22a1, 22a2, and 22a3 are provided on the bottom surface of the door-side shelf 22a. However, the food 70 overlaps with area marker 22a2 and is stored there. The judgment unit 37 can see area markers 22a1 and 22a3 based on the state of the bottom of the door-side shelf 22a as shown in the video, but area marker 22a2 is obscured, thus determining that the food 70 is stored at the position of area marker 22a2.

[0123] [Variation 3: The Influence of Food Height]

[0124] In addition, Figure 15 Depending on the height of the food, sometimes it is not possible to visually confirm the food placed in area marker 22a3 (not shown) in the image of camera unit 31. However, as the opening angle of the door decreases, the food placed in area marker 22a3 can be visually confirmed. Therefore, it is also possible for determination unit 37 to determine the storage position of the food based on the opening angle of multiple doors.

[0125] [Variation Example 3: Modification of the position of area markers as doors open and close]

[0126] Alternatively, the setting unit 39 may dynamically change the area markings 22a1, 22a2, and 22a3 according to the degree of opening of the door 22. Figure 16 as well as Figure 17 This diagram shows door 22 in the open state. Figure 17 It shows the degree to which door 22 is opened compared to Figure 16 The diagram shows a larger state. The region markers 22a1, 22a2, and 22a3 are dynamically changed according to the degree of opening of door 22, so for example, it is only necessary to detect the opening angles of the left and right doors.

[0127] [Variation 3: Measuring the opening angle of the door using a distance measuring unit]

[0128] For example, setting information for setting area markers is pre-stored in the storage unit 34. Figure 18 This is a diagram illustrating an example of setting information. In Figure 18 The values ​​in the settings information are virtual values ​​used for explanation.

[0129] exist Figure 18In the setting information, the distance from the distance measuring section 32 to the right door 22 corresponds to the coordinate of the point R set in the right door 22, and the distance from the distance measuring section 32 to the left door 22 corresponds to the coordinate of the point L set in the left door 22. The setting section 39 determines the coordinates of the point R and the point L that match the measurement results of the distance measuring section 32 in the setting information, and determines the opening angle of the door 22 based on the determined coordinates, and is able to set the positions of the area marks 22a1, 22a2, 22a3 in accordance with the opening angle of the door 22. In addition, in the setting information, the opening degree of the door 22 is divided into four levels, but it can be divided into five or more levels, or it can be divided into three or fewer levels. Figure 18 In the setting information, the opening degree of the door 22 is divided into four levels, but it can be divided into five or more levels, or it can be divided into three or fewer levels.

[0130] In addition, in the present embodiment, one point, the point R, is defined on the right door 22 as viewed from the position of the user, and one point, the point L, is defined on the left door 22 as viewed from the position of the user. However, as long as the opening degree of the door can be determined, the positions and the number of the points as measurement targets can be determined as needed.

[0131] [Modified Example 3: Measurement of the opening angle of the door by an angle sensor]

[0132] Alternatively, the opening degree (opening angle) of the door 22 can be measured by an angle sensor or other sensor provided in the warehouse management device 30 in place of the distance measuring section 32. Further, the sensor that measures the opening degree of the door 22 can be provided in the refrigerator 20, in which case the warehouse management device 30 can not be provided with a sensor for measuring the opening degree of the door 22.

[0133] As described above, the setting section 39 is able to set the area marks 22a1, 22a2, 22a3 in accordance with the opening degree of the door 22 based on the setting information. In addition, the setting section 39, for example, calculates the average and the variance of the N (N is a natural number) closest measurement results (data), and adopts the average as the valid distance only in the case where the variance is equal to or less than a predetermined value.

[0134] [Other Modified Examples]

[0135] In the above-described embodiment, the refrigerator 20 can be a refrigerator for general household use, a refrigerator for product display used in a retail store such as a convenience store, or a refrigerator for other business use.

[0136] Further, in the case where a single category of food is stored in the refrigerator 20, the determination of the category of the food can not be performed.

[0137] Further, the refrigerator 20 is one example of a storage, and the present disclosure can also be implemented as another storage that does not have a refrigeration function. Further, the article (in other words, the object) stored in the storage is not limited to food, and can be another article. It is also possible to consider a case where an article other than food is stored in the refrigerator 20.

[0138] [Effects, etc.]

[0139] As described above, the storage management device 30 includes the imaging section 31 that captures a plurality of images showing a manner of putting or taking out an object with respect to the refrigerator 20 from above the refrigerator 20, the refrigerator 20 having a first storage configuration and a second storage configuration different in position in the vertical direction from the first storage configuration, and including, in the plurality of images, a photographing object that is a marker provided in each of the first storage configuration and the second storage configuration, and the determination section 37 that determines a storage position of the object based on whether the marker is hidden in the plurality of images. The first storage configuration and the second storage configuration are, for example, any one of a shelf, a drawer, and a door shelf, respectively. The marker is, specifically, the first marker 27a, the second marker 27b, the third marker 27c, or the fourth marker 27d, or the like. The refrigerator 20 is one example of a storage.

[0140] Such a storage management device 30 can manage the storage position of the object in the refrigerator 20.

[0141] Further, for example, the first storage configuration and the second storage configuration are, respectively, shelves (for example, the shelf 24a or the shelf 24b) provided in the same storage chamber (for example, the first storage chamber 23). The determination section 37 determines whether the storage position of the object is on the shelf 24a or on the shelf 24b based on whether the marker is hidden in the plurality of images.

[0142] Such a storage management device 30 can manage whether the storage position of the object in the refrigerator 20 is on the shelf 24a or on the shelf 24b.

[0143] Further, for example, the first storage configuration forms the first storage chamber 23, and the second storage configuration forms the second storage chamber 26, the second storage chamber 26 being different in position in the vertical direction from the first storage chamber 23. The determination section 37 determines whether the storage position of the object is in the first storage chamber 23 or in the second storage chamber 26 based on whether the marker is hidden in the plurality of images.

[0144] Such a storage management device 30 can manage whether the storage position of the object in the refrigerator 20 is in the first storage chamber 23 or in the second storage chamber 26.

[0145] Further, for example, the first storage configuration and the second storage configuration are, respectively, the door shelf 22a provided on the door 22 that the refrigerator 20 has.

[0146] Such a storage-in-out management device 30 can manage whether the storage position of the object in the refrigerator 20 is the first storage structure 23 or the second storage structure 26.

[0147] Further, for example, the first storage structure is positioned above the second storage structure. The judging section 37 judges that the storage position of the object is the position corresponding to the first storage structure in a case where the mark provided in the first storage structure is hidden in the plurality of images. The judging section 37 judges that the storage position of the object is the position corresponding to the second storage structure in a case where the mark provided in the first storage structure is not hidden and the mark provided in the second storage structure is hidden in the plurality of images.

[0148] Such a storage-in-out management device 30 can manage the storage position of the object in the refrigerator 20.

[0149] Further, for example, a plurality of marks are provided in each of the first storage structure and the second storage structure.

[0150] Such a storage-in-out management device 30 can further manage the storage position of the object in detail according to which of the plurality of marks is hidden.

[0151] Further, for example, the judging section 37 further judges the category of the object. The storage-in-out management device 30 further includes a notifying section 36 that notifies in a case where the category of the object cannot be judged.

[0152] Through the notification of such a storage-in-out management device 30, the user can take measures such as manual modification of the food management information.

[0153] Further, for example, the imaging section 31 has a structure that can be attached to the refrigerator 20.

[0154] Such an imaging section 31 can be attached to the refrigerator 20.

[0155] Further, the storage-in-out management system 10 includes: the storage-in-out management device 30; and a server device 40. The server device 40 includes: a communication section 41 that receives the storage-in-out information indicating that the object is put into or taken out of the refrigerator 20 from the storage-in-out management device 30; and a notification section 45 that notifies an information terminal 50 possessed by a user in a case where the storage-in-out information is received during a period in which the user is not present around the refrigerator 20.

[0156] Through the notification of such a storage-in-out management system 10, the user can recognize that something such as putting or taking out food from the refrigerator 20 has occurred during a period in which the user is not present around the refrigerator 20.

[0157] Further, the stocker management system 10 includes an imaging section 31 that captures a plurality of images showing a pattern of putting or taking out an object with respect to the refrigerator 20 from above the refrigerator 20, the refrigerator 20 having a first storage configuration and a second storage configuration different in position in the vertical direction from the first storage configuration, and including, in the plurality of images, a marker that is set at each of the first storage configuration and the second storage configuration as an imaging target, and a determination section 37 that determines a storage position of the object based on whether the marker is hidden in the plurality of images. As with the stocker management system 10a, the determination section 37 can be provided by the server device 40a.

[0158] Such a stocker management system 10 can manage the storage position of the object in the refrigerator 20.

[0159] Further, a stocker management method executed by a computer such as the stocker management system 10 includes an imaging step of capturing a plurality of images showing a pattern of putting or taking out an object with respect to the refrigerator 20 from above the refrigerator 20, the refrigerator 20 having a first storage configuration and a second storage configuration different in position in the vertical direction from the first storage configuration, and including, in the plurality of images, a marker that is set at each of the first storage configuration and the second storage configuration as an imaging target, and a determination step of determining a storage position of the object based on whether the marker is hidden in the plurality of images.

[0160] Such a stocker management method can manage the storage position of the object in the refrigerator 20.

[0161] (Other Embodiments)

[0162] The embodiments have been described above, but the present disclosure is not limited to the above-described embodiments.

[0163] For example, although the marker is described as a two-dimensional code in the above-described embodiments, the marker need not be a two-dimensional code, and the marker can be a figure or a line shape extending in the width direction of the refrigerator. Further, a pattern (for example, a pattern appearing through a groove provided on a shelf or the like) provided in the refrigerator for other purposes can be used as the marker. The specific manner of the marker is not particularly limited.

[0164] Further, the method of mounting the marker to the storage configuration is not particularly limited. For example, the marker can be mounted to the storage configuration by sticking a sticker on which the marker is printed to the storage configuration. According to this, the stocker management system becomes capable of managing an article stored in a storage rack (refrigerator) that does not originally have a marker mounted thereto.

[0165] Further, although the stock-in / stock-out management system is realized by a plurality of devices in the above-described embodiments, it can also be realized by a single device. For example, the stock-in / stock-out management system can be realized as a single device corresponding to the stock-in / stock-out management device of the above-described embodiments, or as a single device corresponding to the server device. In the case where the stock-in / stock-out management system is realized by a plurality of devices, the constituent elements possessed by the stock-in / stock-out management system can be arbitrarily distributed among the plurality of devices.

[0166] Further, for example, the communication method between the devices in the above-described embodiments is not particularly limited. Further, in the communication between the devices, a relay device not shown can also be interposed. Further, the transfer path of the information described in the above-described embodiments is not limited to the transfer path shown in the sequence chart.

[0167] For example, the processing performed by a specific processing section in the above-described embodiments can also be performed by another processing section. Further, the order of the plurality of processes can also be changed, and the plurality of processes can also be performed in parallel.

[0168] Further, in the above-described embodiments, each constituent element can also be realized by executing a software program suitable for each constituent element. Each constituent element can also be realized by a program execution section such as a CPU or a processor, which reads and executes a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0169] Further, each constituent element can also be realized by hardware. For example, each constituent element can also be a circuit (or an integrated circuit). These circuits can be constituted as one circuit as a whole, or can be different circuits respectively. Further, these circuits can be general-purpose circuits respectively, or can be dedicated circuits respectively.

[0170] Further, the entire or specific aspects of the present disclosure can also be realized by a system, a device, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the like. Further, the entire or specific aspects of the present disclosure can also be realized by any combination of a system, a device, a method, an integrated circuit, a computer program, and a recording medium.

[0171] For example, the present disclosure can also be realized as a stock-in / stock-out management method of an object executed by a computer, or as a program for causing a computer to execute a stock-in / stock-out management method. The present disclosure can also be realized as a computer-readable non-transitory recording medium in which such a program is recorded.

[0172] In addition to the above, modes obtained by various modifications conceived by those skilled in the art based on the modes described above, and modes obtained by arbitrarily combining constituent elements and functions of the embodiments, are also included in the present disclosure, provided that such modes do not exceed the spirit or scope of the present disclosure.

[0173] Legend

[0174] 10, 10a warehouse management system

[0175] 20 refrigerator (storage warehouse)

[0176] 21 main body

[0177] 22 door

[0178] 22a door side shelf

[0179] 23 first storage chamber

[0180] 24 shelf

[0181] 24a upper shelf

[0182] 24b lower shelf

[0183] 26 second storage chamber

[0184] 27a first mark

[0185] 27b second mark

[0186] 27c third mark

[0187] 27d fourth mark

[0188] 30, 30a warehouse management device

[0189] 31 imaging section

[0190] 32 distance measuring section

[0191] 33, 42 information processing section

[0192] 34, 43 storage section

[0193] 35, 41 communication section

[0194] 36 notification section

[0195] 37, 46 determination section

[0196] 38, 47 tracking section

[0197] 39 setting section

[0198] 40, 40a server device (notification device)

[0199] 44 prompter

[0200] 45 notifier

[0201] 50 information terminal

[0202] 60 wide area communication network

Claims

1. An inbound / outbound management device, comprising: A camera unit captures multiple images from above a storage compartment showing the appearance of objects being placed into and removed from the compartment. The storage compartment has a first storage structure and a second storage structure positioned vertically differently from the first storage structure. The multiple images include a camera object, which is a marker set in each of the first and second storage structures. The determination unit determines the storage location of the object based on whether the mark is obscured in the plurality of images. The first storage structure and the second storage structure are both door-side shelves, which are installed on the door of the storage unit. The markings are multiple region markings in the image, which are set by dividing the bottom surface of the door-side shelf, which is at least one of the first storage structure and the second storage structure, into defined regions. The area markings can change according to the opening angle of the door. The determination unit uses the area markers set according to the opening angle of the door to determine the storage location of the object.

2. The warehouse entry and exit management device as described in claim 1, The warehouse entry / exit management device also includes a distance measuring unit for measuring the distance from the distance measuring unit to the door. The opening angle of the door is determined using the distance from the measuring unit to the door, as measured by the measuring unit. The determination unit uses the area markers set according to the determined opening angle of the door to determine the storage location of the object.

3. The warehouse entry and exit management device as described in claim 2, The warehouse entry / exit management device also includes a setting unit for setting the area markers. The setting unit determines the distance used to determine the opening angle of the door by using the result obtained by the distance measuring unit measuring the distance to the door a predetermined number of times.

4. The warehouse entry and exit management device as described in claim 1, The warehouse entry / exit management device also includes a sensor for measuring the opening angle of the door. The determination unit uses the area marker set according to the opening angle of the door measured by the sensor to determine the storage location of the object.

5. The warehouse entry and exit management device as described in claim 1, The determination unit determines, on the door side shelf which is marked with the plurality of area markers, which in turn determines the storage location of the object on which area marker is located based on whether the area marker is obscured in the plurality of images.

6. The warehouse entry and exit management device as described in claim 1, The determination unit also determines the category of the object. The inbound / outbound management device also includes a notification unit that provides notification when it is unable to determine the category of the object.

7. An inbound and outbound management system, The inbound / outbound management system has the following features: The warehouse management device according to any one of claims 1 to 6; and Notification device, The notification device includes: The communication unit receives information from the inventory management device indicating that the object has been placed into the storage compartment or that the object has been removed from the storage compartment; and The notification department notifies the user's information terminal when the user of the storage room is not in the vicinity of the storage room.

8. An inbound / outbound management system, comprising: A camera unit captures multiple images from above a storage compartment showing the appearance of objects being placed into and removed from the compartment. The storage compartment has a first storage structure and a second storage structure positioned vertically differently from the first storage structure. The multiple images include a camera object, which is a marker set in each of the first and second storage structures. The determination unit determines the storage location of the object based on whether the mark is obscured in the plurality of images. The first storage structure and the second storage structure are both door-side shelves, which are installed on the door of the storage unit. The markings are multiple region markings in the image, which are set by dividing the bottom surface of the door-side shelf, which is at least one of the first storage structure and the second storage structure, into defined regions. The area markings can change according to the opening angle of the door. The determination unit uses the area markers set according to the opening angle of the door to determine the storage location of the object.

9. A method for managing inbound and outbound inventory, comprising: The camera step involves capturing multiple images from above the storage compartment showing the appearance of objects being placed into and taken out of the storage compartment. The storage compartment has a first storage structure and a second storage structure located in a different position in the vertical direction from the first storage structure. The multiple images include a camera object, which is a marker set in each of the first storage structure and the second storage structure. as well as The determination step involves identifying the storage location of the object based on whether the mark is obscured in the multiple images. The first storage structure and the second storage structure are both door-side shelves, which are installed on the door of the storage unit. The markings are multiple region markings in the image, which are set by dividing the bottom surface of the door-side shelf, which is at least one of the first storage structure and the second storage structure, into defined regions. The area markings can change according to the opening angle of the door. In the judgment step, the storage location of the object is determined by using the area markers set according to the opening angle of the door.

10. A recording medium having a program recorded thereon for causing a computer to execute the inventory management method of claim 9.

Images