Measurement apparatus, measurement system, and measurement method

Abstract

To calculate filling rate of a shelf even when an opening of the shelf cannot be detected from measurement by a distance measuring sensor.SOLUTION: A measurement device that measures filling rate related to an object arranged on a shelf is configured to: detect a shelf frame from distance measurement data obtained by three dimensionally scanning the shelf; set a virtual shelf frame, which is a virtual shelf frame, in a region adjacent to the detected shelf frame and in which the shelf frame is not detected in the distance measurement data; and calculate the filling rate related to the object in the virtual shelf frame.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a measuring device, a measuring system, and a measuring method.

Background Art

[0002] In the logistics and distribution sites, it is required to measure the filling rate of measurement objects such as luggage in the storage space and improve the utilization efficiency of the storage space.

[0003] Patent Document 1 discloses a filling rate measurement method in which a three-dimensional space model obtained by measurement through an opening by a distance measuring sensor facing a storage unit having an opening is acquired, a storage three-dimensional model that is a three-dimensional model of the storage unit is acquired, an object part that is a part of the measurement object in the three-dimensional space model is extracted, a line segment indicating the shape of the opening is specified from a two-dimensional image of the opening generated by measurement in a specific direction from the position of the distance measuring sensor, and an object three-dimensional model that is a three-dimensional model of the measurement object is estimated based on a three-dimensional coordinate system based on the position of the opening in the three-dimensional space specified based on the position of the distance measuring sensor, the specific direction, and the shape of the opening, and the object part, and the filling rate of the measurement object with respect to the storage space is calculated.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When the method disclosed in Patent Document 1 is used, for example, to measure the filling rate of luggage stored on a shelf in a warehouse, the opening of the shelf is not always detectable from the measurement by the distance measuring sensor. In that case, the filling rate of the shelf cannot be calculated.

[0006] An object of the present disclosure is to provide a technique for calculating the filling rate of a shelf even when the opening of the shelf cannot be detected from the measurement by a distance measuring sensor.

Means for Solving the Problems

[0007] A measuring device according to an aspect of the present disclosure is a measuring device that measures the filling rate of an object arranged on a shelf, and includes a processor and a memory. The processor, in cooperation with the memory, detects a shelf frame from distance measurement data obtained by three-dimensionally scanning the shelf, sets a virtual shelf frame, which is a virtual shelf frame, in an area where no shelf frame is detected and which is adjacent to the detected shelf frame in the distance measurement data, and calculates the filling rate of the object within the virtual shelf frame.

[0008] A measurement system according to an aspect of the present disclosure is a measurement system including a measurement device that measures the filling rate of an object arranged on a shelf and a server device capable of communicating with the measurement device. The measurement device detects a shelf frame from distance measurement data obtained by three-dimensionally scanning the shelf, sets a virtual shelf frame, which is a virtual shelf frame, in an area where no shelf frame is detected and which is adjacent to the detected shelf frame in the distance measurement data, calculates the filling rate of the object within the virtual shelf frame, transmits the distance measurement data and the calculated filling rate to the server device, and the server device displays the distance measurement data and the filling rate received from the measurement device.

[0009] A measurement method according to an aspect of the present disclosure is a measurement method that measures the filling rate of an object arranged on a shelf. The method includes detecting a shelf frame from distance measurement data obtained by three-dimensionally scanning the shelf, setting a virtual shelf frame, which is a virtual shelf frame, in an area where no shelf frame is detected and which is adjacent to the detected shelf frame in the distance measurement data, calculating the filling rate of the object within the virtual shelf frame, and displaying the distance measurement data and the filling rate.

[0010] These general or specific aspects may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium, or may be implemented in any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

Advantages of the Invention

[0011] According to the present disclosure, even when the opening of the shelf cannot be detected from the measurement by the distance measuring sensor, the filling rate of the shelf can be calculated.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

MODE FOR CARRYING OUT THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, a more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and duplicate descriptions of substantially the same configurations may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art. The attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0014] (Embodiment 1) <Measurement System> FIG. 1 is a schematic diagram for explaining the outline of the operation of the measuring device 20 according to Embodiment 1 for three-dimensionally scanning the shelf 1 in the warehouse and the luggage 9 stored in the shelf 1. FIG. 2 is a block diagram showing a configuration example of the measurement system 10 according to Embodiment 1. FIG. 3 is a view of the shelf 1, the luggage 9, and the measuring device 20 according to Embodiment 1 as seen from the side. FIG. 4 is a view of the shelf 1 and the luggage 9 according to Embodiment 1 as seen from the front. Next, the configuration and operation of the measurement system 10 according to Embodiment 1 will be described with reference to FIGS. 1 to 4. For the sake of convenience of explanation, as shown in FIGS. 1, 3, and 4, the width direction of the shelf 1 is defined as the X-axis, the depth direction of the shelf 1 is defined as the Y-axis, and the height direction of the shelf 1 is defined as the Z-axis.

[0015] The measurement system 10 is a system that measures the filling rate of the goods 9 on the shelf 1. The filling rate is a value indicating the ratio of the volume of the goods 9 actually stored in the shelf 1 to the volume of the space (hereinafter referred to as the storage possible space) that can accommodate the goods 9 in the shelf 1. Note that the filling rate may not be an exact value and may be an approximate value or an estimated value.

[0016] As shown in FIG. 2, the measurement system 10 includes at least one measurement device 20 and a server device 30. The measurement device 20 is connected to the server device 30 via a predetermined communication network 11. Examples of the predetermined communication network 11 include a wireless LAN, a wired LAN, a mobile communication network, the Internet, and a Virtual Private Network (VPN). However, the measurement device 20 does not necessarily have to be directly connected to the server device 30. For example, it may be indirectly connected to the server device 30 via a relay device (not shown). In this case, the measurement device 20 may transmit information to the relay device, and the relay device may transmit (transfer) the information received from the measurement device to the server device.

[0017] <Measurement device> The measurement device 20 is a device that measures the filling rate of each shelf 1 in the warehouse. As shown in FIGS. 1 and 2, the measurement device 20 includes at least one distance measurement sensor 21 (21A, 21B, 21C) and an information processing device 22. The distance measurement sensor 21 is connected to the information processing device 22 via a predetermined electrical cable. An example of the electrical cable is a USB cable. However, the distance measurement sensor 21 may be connected to the information processing device 22 by wireless communication.

[0018] The distance measurement sensor 21 captures an object to generate an RGB image and measures the distance (depth) to the object to generate a depth image. The RGB image has color information for each pixel. The depth image has depth (distance) information for each pixel. Hereinafter, the RGB image and the depth image are collectively referred to as distance measurement data 100. The distance measurement sensor 21 may be at least one of a Time of Flight (ToF) sensor, a Light Detection And Ranging (LiDAR), and a stereo camera, etc.

[0019] As shown in FIG. 2, the information processing device 22 includes a processor 23, a memory 24, a communication device 25, a device connection device 26, an input device 27, and an output device 28.

[0020] The processor 23 realizes the functions of the measurement device 20 by executing a computer program in cooperation with the memory 24. Details of the functions of the measurement device 20 will be described as appropriate.

[0021] The memory 24 stores a computer program and data for realizing the functions of the measurement device 20. The memory 24 may be composed of a volatile storage medium (e.g., RAM) and / or a non-volatile storage medium (e.g., ROM, flash memory, Solid State Drive (SSD), etc.).

[0022] The communication device 25 is connected to the communication network 11 and controls the transmission and reception of information via the communication network 11.

[0023] The device connection device 26 is connected to the distance measurement sensor 21 and controls the transmission and reception of information with the distance measurement sensor 21.

[0024] The input device 27 is a device that receives input from a user, and is, for example, a touch panel, a keyboard, a mouse, a microphone, etc.

[0025] The output device 28 is a device that outputs information, and is, for example, a display, a speaker, a lamp, etc.

[0026] Next, the operation of the measuring device 20 will be described.

[0027] As shown in FIG. 1, the measuring device 20 moves along the passage in the warehouse and performs three-dimensional scanning (hereinafter referred to as 3D scanning) on a plurality of shelves 1 arranged along the passage with a distance measuring sensor 21 to generate distance measurement data 100. The movement of the measuring device 20 may be manually moved by a person or may be automatically (autonomously) moved by the measuring device 20.

[0028] For example, as shown in FIGS. 3 and 4, when the shelf 1 has a two-stage configuration of a lower stage 2A and an upper stage 2B, the measuring device 20 includes a distance measuring sensor 21A capable of 3D scanning the lower stage 2A, a distance measuring sensor 21B capable of 3D scanning the upper stage 2B, and a distance measuring sensor 21C capable of 3D scanning the space above the shelf 1 on the upper stage 2B (that is, above the uppermost shelf 1). Note that it is not essential to perform 3D scanning of one shelf stage with one distance measuring sensor. For example, the lower stage 2A and the upper stage 2B may be 3D scanned with one distance measuring sensor. In the present embodiment, an example in which the number of overlapping shelves 1 is two is described, but the number of overlapping shelves 1 may be any number. Hereinafter, the space above the uppermost shelf 1 is referred to as above-shelf 2C.

[0029] The distance measuring sensor 21A transmits the distance measurement data 100A obtained by 3D scanning the lower stage 2A to the information processing device 22. The distance measuring sensor 21B transmits the distance measurement data 100B obtained by 3D scanning the upper stage 2B to the information processing device 22. The distance measuring sensor 21C transmits the distance measurement data 100C obtained by 3D scanning the above-shelf 2C to the information processing device 22.

[0030] The processor 23 of the information processing apparatus 22 detects the shelf frame 3A (opening) of the lower stage 2A from the distance measurement data 100A, and detects the luggage 9 existing within the range surrounded by the shelf frame 3A of the lower stage 2A (hereinafter referred to as the measurement range 101A). Then, the processor 23 calculates the ratio of the volume of the detected luggage 9 to the volume of the accommodatable space in the measurement range 101A, and uses it as the filling rate in the shelf frame 3A (measurement range 101A) of the lower stage 2A. The processor 23 may calculate the filling rate by the method described in Patent Document 1. For example, by the method described in Patent Document 1, the processor 23 specifies the foremost depth of the luggage 9 from the depth image, and calculates the volume of the luggage 9 assuming that the luggage 9 is arranged from the specified foremost depth of the luggage 9 to the rearmost depth of the accommodatable space of the shelf frame 3A. Note that the depth may be read as distance or position.

[0031] Similarly, the processor 23 detects the shelf frame 3B (opening) of the upper stage 2B from the distance measurement data 100B, and detects the luggage existing within the range surrounded by the shelf frame 3B of the upper stage 2B (hereinafter referred to as the measurement range 101B). Then, the processor 23 calculates the ratio of the volume of the detected luggage 9 to the volume of the accommodatable space in the measurement range 101B, and uses it as the filling rate in the shelf frame 3B (measurement range 101B) of the upper stage 2B.

[0032] However, in the method described in Patent Document 1, since there is no opening (shelf frame) on the upper shelf 2C, the filling rate of the luggage 9 arranged on the upper shelf 2C cannot be calculated. Therefore, hereinafter, a method for calculating the filling rate of the luggage 9 arranged on the upper shelf 2C will be described with reference to FIGS. 4 and 5. FIG. 5 is a diagram for explaining the virtual shelf frame 50A of the upper shelf 2C according to the first embodiment.

[0033] As shown in FIGS. 4 and 5, the distance measuring sensor 21C is set to be capable of 3D scanning at least a part of the upper stage 2B and the space where the luggage 9 on the upper shelf 2C is placed. As shown in FIG. 5, the processor 23 of the information processing device 22 sets a virtual shelf frame (hereinafter referred to as a virtual shelf frame 50A) for the upper shelf 2C of the distance measurement data 100C received from the distance measuring sensor 21C. The processor 23 may set the virtual shelf frame 50A by the process shown in FIG. 6 below. FIG. 6 is a flowchart showing an example of a process for setting the virtual shelf frame 50A according to the first embodiment.

[0034] The processor 23 of the information processing device 22 detects the shelf board 4B of the upper stage 2B (the uppermost stage) and the left and right shelf columns 5B from the distance measurement data 100C (S11).

[0035] The processor 23 extends the line segments 41 of the detected left and right shelf columns 5B upward to set the left and right line segments 51 of the virtual shelf frame 50A (S12).

[0036] The processor 23 parallel - moves the line segment 42 of the detected shelf board 4B of the upper stage 2B (the uppermost stage) upward to set the upper line segment 52 of the virtual shelf frame 50A (S13). The amount of parallel movement, that is, the height of the virtual shelf frame 50A may be a value pre - input by the user. Alternatively, the height of the virtual shelf frame 50A may be the upper limit value of the height at which the luggage 9 can be loaded on the upper shelf 2C (that is, the loading is permitted for safety reasons).

[0037] The processor 23 parallel - moves the line segment 42 of the detected shelf board 4B of the upper stage 2B (the uppermost stage) upward by the thickness of the shelf board 4B to set the lower line segment 53 of the virtual shelf frame 50A (S14).

[0038] The processor 23 sets the depth of the shelf 1 of the upper stage 2B (the uppermost stage) as the depth of the virtual shelf frame 50A. Note that the depth of the shelf 1 may be input by the user (S15).

[0039] The process shown in Fig. 6 can be expressed as follows. That is, the width of the virtual shelf frame 50A is set based on the width of the topmost shelf board 4B, the height of the virtual shelf frame 50A is set based on a predetermined value or the upper limit value of the height that can be loaded on the upper shelf 2C, and the depth of the virtual shelf frame 50A is set based on the depth of the topmost shelf 1.

[0040] Through the process shown in Fig. 6, the processor 23 of the information processing device 22 can set the virtual shelf frame 50A on the upper shelf 2C for the ranging data 100C.

[0041] The processor 23 of the information processing device 22 sets the virtual shelf frame 50A for the ranging data 100C, and detects the luggage 9 existing within the range surrounded by the virtual shelf frame 50A (hereinafter referred to as the measurement range 101C), that is, the luggage 9 arranged on the upper shelf 2C. Then, the processor 23 calculates the ratio of the volume of the detected luggage 9 to the volume of the accommodable space of the measurement range 101C (that is, the accommodable space of the virtual shelf frame 50A), and uses it as the filling rate of the upper shelf 2C. The processor 23 may calculate the filling rate by the method described in Patent Document 1.

[0042] In addition, when the processor 23 of the information processing device 22 detects the luggage 9 loaded exceeding the height of the virtual shelf frame 50A, it may output a warning sound and / or a warning image to the output device 28. Thereby, the measuring device 20 can notify the user of the existence of the luggage 9 loaded exceeding the upper limit value of the height that can be loaded on the upper shelf 2C. Further, when the processor 23 of the information processing device 22 detects the luggage 9 loaded exceeding the height of the virtual shelf frame 50A, it may transmit a notification of overloading on the upper shelf 2C and the number of the shelf 1 to the server device 30. Thereby, the measuring device 20 can notify the server device 30 of the existence of the luggage 9 loaded exceeding the upper limit value of the height that can be loaded on the upper shelf 2C and the number of the shelf 1.

[0043] Through the above processing, the information processing apparatus 22 can calculate the filling rate not only for the packages 9 stored in the shelf 1 but also for the packages 9 arranged on the upper shelf 2C. The information processing apparatus 22 transmits the filling rates calculated for each stage (lower stage 2A, upper stage 2B) and the upper shelf 2C, together with the number of the shelf 1 and the ranging data 100 of the shelf 1, to the server apparatus 30. Thereby, the server apparatus 30 can collect the filling rates of the respective shelves 1 and the upper shelf 2C in the warehouse.

[0044] FIG. 7 is a flowchart showing an example of processing performed by the information processing apparatus 22 according to Embodiment 1. The information processing apparatus 22 may receive the ranging data 100 from each ranging sensor 21 and perform the processing shown in FIG. 7.

[0045] The processor 23 of the information processing apparatus 22 receives the ranging data 100 from the ranging sensor 21 (S101).

[0046] The processor 23 detects the shelf frame 3 from the ranging data 100 (S102).

[0047] The processor 23 determines whether the shelf frame 3 has been detected from the ranging data 100 in step S102 (S103).

[0048] If the shelf frame 3 cannot be detected from the ranging data 100 (S103: NO), the processor 23 returns the process to step S101.

[0049] If the shelf frame 3 has been detected from the ranging data 100 (S103: YES), the processor 23 performs the following processing. That is, the processor 23 detects the packages 9 within the shelf frame 3 (S104) and calculates the filling rate of the packages 9 within the shelf frame 3 (S105).

[0050] The processor 23 determines whether the shelf frame 3 detected in step S102 is the uppermost shelf frame 3 (S106). Note that if it has been previously determined that the ranging data 100C obtained by 3D scanning the upper shelf 2C does not include other shelf frames (for example, the shelf frame 3B of the upper stage 2B), this determination process may be omitted.

[0051] If the shelf frame 3 detected in step S102 is not the uppermost shelf frame 3 (S106: NO), the processor 23 returns the process to step S101.

[0052] If the shelf frame 3 detected in step S102 is the uppermost shelf frame 3 (S106: YES), the processor 23 advances the process to step S110.

[0053] As step S110, the processor 23 sets a virtual shelf frame 50A on the upper shelf 2C of the distance measurement data 100C by the process of FIG. 6 above (S110). The processor 23 detects the luggage 9 within the virtual shelf frame 50A (S111), and calculates the filling rate of the luggage 9 within the virtual shelf frame 50A (S112). The processor 23 returns the process to step S101.

[0054] Through the above process, the information processing apparatus 22 can set the virtual shelf frame 50A for the space of the upper shelf 20C of the distance measurement data 100, and calculate the filling rate in the virtual shelf frame 50A.

[0055] <Server device> As shown in FIG. 2, the server device 30 includes a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35.

[0056] The processor 31 realizes the functions of the server device 30 by executing a computer program in cooperation with the memory 32. Details of the functions of the server device 30 will be described as appropriate.

[0057] The memory 32 stores a computer program and data for realizing the functions of the server device 30. The memory 32 is composed of a volatile storage medium (for example, RAM) and / or a non-volatile storage medium (for example, ROM, flash memory, SSD, etc.).

[0058] The communication device 33 is connected to the communication network 11 and controls the transmission and reception of information via the communication network 11.

[0059] The input device 34 is a device that receives input from the user, and examples thereof include a touch panel, a keyboard, a mouse, a microphone, and the like.

[0060] The output device 35 is a device that outputs information, and examples thereof include a display, a speaker, a lamp, and the like.

[0061] The server device 30 receives information such as the filling rate and the ranging data 100 of each shelf 1 from the measuring device 20, and generates information indicating the status of each shelf in the warehouse based on the received information. The server device 30 displays the generated information indicating the status of each shelf in the warehouse on the output device 35 (display). For example, the server device 30 displays the filling rates of each stage (lower stage 2A, upper stage 2B) and the upper part 2C of the shelf. For example, the server device 30 calculates the average value of the filling rates of each stage (lower stage 2A, upper stage 2B) and the upper part 2C of the plurality of overlapping shelves, and displays it as the average filling rate of the plurality of overlapping shelves. Thereby, the server device 30 can notify the user of the storage status of the goods on each shelf 1 in the warehouse.

[0062] When the server device 30 receives a notification of overloading on the upper part 2C of the shelf from the measuring device 20, it may display the number of the shelf 1 with the overloading. Thereby, the user can know the shelf 1 with overloading on the upper part 2C of the shelf.

[0063] Note that other display examples of the server device 30 will be described in Embodiment 2 (see FIGS. 13 to 15).

[0064] (Embodiment 2) FIG. 8 is a side view of the shelf 1, the goods 9, and the measuring device according to Embodiment 2. FIG. 9 is a front view of the shelf 1 and the goods 9 according to Embodiment 2.

[0065] In Embodiment 1, an example in which the upper shelf 2C is 3D scanned by the distance measurement sensor 21C was described. However, due to reasons such as cost reduction or safety restrictions on the height at which sensors or the like can be installed, it may not be possible to install the distance measurement sensor 21C at a high position. Further, even if the measurement device 20 is moved as far as possible from the shelf 1 because the aisle width of the warehouse is narrow, it may not be possible to include the entire upper shelf 2C within the angular field of view of the distance measurement sensor. Therefore, in Embodiment 2, as shown in FIG. 8, a method for calculating a provisional filling rate of the upper shelf 2C will be described in a configuration in which the measurement device 20 includes the distance measurement sensors 21A and 21B but does not include the distance measurement sensor 21C. Note that the configurations of the measurement device 20 and the server device 30 may be the same as those in Embodiment 1, and thus the description thereof will be omitted.

[0066] The measurement device 20 includes a distance measurement sensor A capable of 3D scanning the lower stage 2A, and a distance measurement sensor 21B capable of 3D scanning the upper stage 2B and a partial space of the upper shelf 2C. The distance measurement sensor 21A transmits distance measurement data 100A obtained by 3D scanning the lower stage 2A to the information processing device 22. The distance measurement sensor 21B transmits distance measurement data 100B obtained by 3D scanning the upper stage 2B and a partial space of the upper shelf 2C to the information processing device 22. Since the angular field of view of the distance measurement sensor 21B cannot include all of the space where the goods 9 on the upper shelf 2C can be loaded, as shown in FIG. 9, a partial space of the upper shelf 2C is imaged in the distance measurement data 100B.

[0067] FIG. 10 is a diagram for explaining the virtual shelf frame 50B of the upper shelf 2C according to Embodiment 2.

[0068] As shown in FIG. 10, the processor 23 of the information processing device 22 sets a virtual shelf frame 50B on the upper shelf 2C in the distance measurement data 100B received from the distance measurement sensor 21B. The processor 23 sets the virtual shelf frame 50B by the process shown in FIG. 11 below. FIG. 11 is a flowchart showing an example of a process for setting the virtual shelf frame 50B according to Embodiment 2.

[0069] The processor 23 of the information processing apparatus 22 detects the upper shelf board 4B (the uppermost shelf board) and the left and right shelf columns 5B from the distance measurement data 100B (S22).

[0070] The processor 23 extends the line segments 41 of the detected left and right shelf columns 5B upward to set the left and right line segments 51 of the virtual shelf frame 50D (S22).

[0071] The processor 23 translates the line segment 42 of the detected upper shelf board 4B (the uppermost shelf board) parallel to the upper end of the distance measurement data 100B to set the upper line segment 52 of the virtual shelf frame 50B (S23). Note that the translation amount is not limited to the upper end of the distance measurement data 100B and may be a previously input amount.

[0072] The processor 23 translates the line segment 42 of the detected upper shelf board 4B (the uppermost shelf board) upward by the thickness of the shelf board 4B to set the lower line segment 53 of the virtual shelf frame 50B (S24).

[0073] The processor 23 sets the depth of the upper shelf 1 (the uppermost shelf) to the depth of the virtual shelf frame 50B. Note that the depth of the shelf 1 may be input by the user (S25).

[0074] Through the above processing, the virtual shelf frame 50B can be set for a part of the space on the shelf 2C with respect to the distance measurement data 100B.

[0075] The processor 23 of the information processing apparatus 22 sets the virtual shelf frame 50B in the distance measurement data 100B and detects a part of the load 9 existing within the range (measurement range 101D) surrounded by the virtual shelf frame 50B (that is, a part of the load 9 arranged on the shelf 2C). Then, the processor 23 calculates the occupancy rate of the load 9 in the measurement range 101D. When the occupancy rate is equal to or greater than a predetermined threshold value, it is determined that the load 9 exists on the shelf 2C, or the virtual filling rate of the shelf 2C is calculated as 100%. Note that the method for calculating the occupancy rate is the same as the method for calculating the filling rate of the virtual shelf frame 50A described in the first embodiment.

[0076] Through the above processing, even when the distance measurement sensor 21B is installed at an angle of view that cannot cover all of the loadable space on the upper shelf 2C, the information processing apparatus 22 can determine whether there is a load 9 on the upper shelf 2C or calculate the provisional filling rate of the upper shelf 2C.

[0077] FIG. 12 is a flowchart showing an example of processing performed by the information processing apparatus 22 according to the second embodiment. The information processing apparatus 22 may receive distance measurement data 100 from each distance measurement sensor 21 and perform the processing shown in FIG. 12.

[0078] The processor 23 of the information processing apparatus 22 receives distance measurement data 100 from the distance measurement sensor 21 (S201).

[0079] The processor 23 detects the shelf frame 3 from the distance measurement data 100 (S202).

[0080] The processor 23 detects the load 9 within the shelf frame 3 (S203) and calculates the filling rate of the load 9 within the shelf frame 3 (S204).

[0081] The processor 23 determines whether the shelf frame 3 detected in step S202 is the uppermost shelf frame 3 (S205).

[0082] If the detected shelf frame 3 is not the uppermost shelf frame 3 (S205: NO), the processor 23 returns the process to step S201.

[0083] If the detected shelf frame 3 is the uppermost shelf frame 3 (S205: YES), the processor 23 sets a virtual shelf frame 50B on the upper shelf 2C of the distance measurement data 100 by the processing of FIG. 6 above (S206). Then, the processor 23 detects the load 9 within the virtual shelf frame 50B (S207) and calculates the occupancy rate of the load 9 within the virtual shelf frame 50B (S208). The method of calculating the occupancy rate is the same as the method of calculating the filling rate of the virtual shelf frame 50A described in the first embodiment.

[0084] The processor 23 determines whether the occupancy rate of the load 9 with respect to the virtual shelf frame 50B is equal to or higher than a predetermined threshold value (S209).

[0085] When the occupancy rate of the load 9 with respect to the virtual shelf frame 50B is equal to or higher than the predetermined threshold value (S209: YES), the processor 23 determines that there is a load on the upper shelf 2C, or calculates the temporary filling rate of the upper shelf 2C as 100% (S210). Then, the process returns to step S201.

[0086] When the occupancy rate of the load 9 with respect to the virtual shelf frame 50B is less than the predetermined threshold value (S209: NO), the processor 23 determines that there is no load on the upper shelf 2C, or calculates the temporary filling rate of the upper shelf 2C as 0% (S211). Then, the process returns to step S201.

[0087] Through the above processing, the information processing apparatus 22 can set the virtual shelf frame 50B with respect to the space of the upper shelf 2C of the distance measurement data 100, and calculate the presence or absence of a load in the virtual shelf frame 50B or the temporary filling rate.

[0088] Note that the information processing apparatus 22 may detect and calculate the filling rate of the shelf frame 3B and the temporary filling rate of the virtual shelf frame 50B from one distance measurement data 100B at a time. Thereby, the number of processing steps can be reduced.

[0089] FIG. 13 is a diagram showing an example of a utilization efficiency screen 200 displayed by the server apparatus 30 according to the second embodiment. The server apparatus 30 displays the utilization efficiency screen 200 as shown in FIG. 13 on the output device 35 (display).

[0090] The utilization efficiency screen 200 includes an area 201 for the filling rate inside the shelves of the entire warehouse, an area 202 for the filling rate on the upper shelves of the entire warehouse, an area 203 for the filling rate inside the shelves by column, and an area 204 for the filling rate by row.

[0091] In the area 201 of the shelf filling rate of the entire warehouse, the average value of the filling rates of all the lower shelves 2A and upper shelves 2B in the warehouse is displayed. In the second embodiment, since the filling rate of the upper shelves 2C is a provisional filling rate, the provisional filling rate of the upper shelves 2C is not included in the calculation of the average value. However, in the first embodiment, the filling rate of the upper shelves 2C may be included in the calculation of the average value. Also, in the area 201 of the shelf filling rate of the entire warehouse, a histogram showing the shelf filling rate on the horizontal axis and the number of shelves on the vertical axis is displayed.

[0092] In the area 202 of the upper shelf filling rate of the entire warehouse, the average value of the filling rates of all the upper shelves 2C in the warehouse is displayed.

[0093] In the area 203 of the shelf filling rate by column, the average value of the filling rates of the lower shelves 2A and upper shelves 2B of one column in the warehouse is displayed.

[0094] In the area 204 of the filling rate by stage, the average value of the filling rates of each stage (lower stage 2A, upper stage 2B, upper shelves 2C) in the warehouse is displayed.

[0095] By looking at the utilization efficiency screen 200, the user can easily grasp the utilization efficiency of the shelves 1 in the warehouse.

[0096] FIG. 14 is a diagram showing an example of a cargo consolidation support screen 300 displayed by the server device according to the second embodiment. The server device 30 displays the cargo consolidation support screen 300 as shown in FIG. 14 on the output device 35 (display).

[0097] The cargo consolidation support screen 300 has a filling rate distribution area 301 and a shelf number list area 302.

[0098] In the filling rate distribution area 301, a histogram showing the shelf filling rate bins of the shelves 1 on the horizontal axis and the number of shelves on the vertical axis is displayed. The bar graph of this histogram may display the number of lower shelves 2A and the number of upper shelves 2B separately distinguishable.

[0099] In the area 302 of the list of shelf numbers, the numbers of the shelves included in the bar graph selected in the area 301 of the filling rate distribution are displayed in a list format.

[0100] By looking at the luggage consolidation support screen 300, the user can grasp Shelf 1 with a low filling rate or Shelf 1 with a high filling rate. Therefore, by looking at the luggage consolidation support screen 300, the user can consolidate Luggage 9 on a specific Shelf 1 to increase the number of empty shelves 1 or equalize the filling rate of the luggage on each Shelf 1.

[0101] FIG. 15 is a diagram showing an example of the shelf vacancy status screen 400 displayed by the server device 30 according to the second embodiment. The server device 30 displays the shelf vacancy status screen 400 as shown in FIG. 15 on the output device 35 (display).

[0102] The shelf vacancy status screen 400 has an area 401 for the upper limit threshold of the filling rate, an area 402 for the area map, an area 403 for the filling rate of each shelf, and an area 404 for the distance measurement data.

[0103] In the area 401 for the upper limit threshold of the filling rate, an interface (for example, a slide bar and a numerical input field) through which the user can set the upper limit threshold of the filling rate is displayed.

[0104] In the area 402 of the area map, an icon indicating the position of Shelf 1 in the warehouse (hereinafter referred to as the shelf icon 411) is displayed. The shelf icon 411 may be displayed in a different manner (for example, a different color) when the filling rate of the corresponding shelf 1 is below the upper limit threshold of the filling rate set by the user than when it is greater than the upper limit threshold. Thereby, the user can easily know Shelf 1 whose filling rate is below the set upper limit threshold of the filling rate by looking at the area 402 of the area map.

[0105] In the area 403 for the filling rate of each shelf, the filling rate of the shelf (lower stage 2A, upper stage 2B, above-shelf 2C) corresponding to the shelf icon 411 selected by the user in the area 402 of the area map is displayed.

[0106] In the ranging data area 404, an RGB image of the shelf (a part of the lower shelf 2A, the upper shelf 2B, and the upper part of the shelf 2C) corresponding to the shelf icon 411 selected by the user in the area map area 402 is displayed. As shown in FIG. 15, the server device 30 may superimpose and display the shelf frame 3 and the filling rate of the shelf frame 3 on the RGB image. Further, the server device 30 may superimpose and display the virtual shelf frame 50B and the provisional filling rate of the virtual shelf frame 50B on the RGB image. Note that in the first embodiment, instead of the provisional filling rate, the filling rate of the upper part of the shelf 2C may be superimposed and displayed.

[0107] The user can view the empty shelf status screen 400 to search for a shelf 1 with a low filling rate or check the RGB image of the photographed shelf 1.

[0108] (Embodiment 3) FIG. 16 is a diagram for explaining the setting of the virtual shelf frame 50C according to the third embodiment.

[0109] For example, when at least a part of the shelf 1 (for example, the shelf post 5B in FIG. 16(a)) is silver, the light irradiated from the ranging sensor 21 is reflected by the silver, and the shelf 1 cannot be ranged. As a result, the shelf frame 3 may not be detected from the ranging data 100. For example, when at least a part of the shelf 1 (for example, the shelf post 5A in FIG. 16(b)) is black, the light irradiated from the ranging sensor 21 is absorbed by the black, and the shelf 1 cannot be ranged. As a result, the shelf frame 3 may not be detected from the ranging data 100. In the third embodiment, a method of setting a virtual shelf frame and measuring the filling rate when the ranging sensor 21 cannot detect at least a part of the shelf 1 will be described.

[0110] FIG. 16(a) is a diagram showing a case where the shelf column 5B of the upper stage 2B cannot be distance-measured and the shelf frame 3 of the upper stage 2B cannot be detected. In this case, the information processing device 22 sets a virtual shelf frame 50C in the upper stage 2B based on the detected shelf frame 3A of the lower stage 2A adjacent to the upper stage 2B. For example, the information processing device 22 sets a virtual shelf frame 50C in the upper stage 2B with the same size as the shelf frame 3A of the lower stage 2A by the method described in the first embodiment. Alternatively, the information processing device 22 sets a virtual shelf frame 50C having the same width and depth as the shelf frame 3A of the lower stage 2A and a predetermined height for the upper stage 2B. Thereby, the information processing device 22 can calculate the filling rate of the goods 9 in the upper stage 2B by the method described in the first embodiment using the virtual shelf frame 50C set in the upper stage 2B.

[0111] FIG. 16(b) is a diagram showing a case where the shelf column 5A of the lower stage 2A cannot be distance-measured and the shelf frame 3 of the lower stage 2A cannot be detected. In this case, the information processing device 22 sets a virtual shelf frame 50C in the lower stage 2A based on the detected shelf frame 3B of the upper stage 2B adjacent to the lower stage 2A. The method for determining the size of the virtual shelf frame 50C may be the same as that in the case of FIG. 16(a) above. Thereby, the information processing device 22 can calculate the filling rate of the goods 9 in the lower stage 2A by the method described in the first embodiment using the virtual shelf frame 50C set in the lower stage 2A.

[0112] When the information processing device 22 receives the distance measurement data 100 in which both the lower stage 2A and the upper stage 2B are photographed, and when one of the shelf frames 3 of the lower stage 2A or the upper stage 2B can be detected and the other shelf frame 3 cannot be detected, a virtual shelf frame 50C may be set in the shelf stage that cannot be detected based on the detected shelf frame 3 of the detected shelf stage.

[0113] Alternatively, when the information processing device 22 receives the distance measurement data 100A in which the lower stage 2A is photographed and the distance measurement data 100B in which the upper stage 2B is photographed respectively, and when the shelf frame 3 can be detected from one of the distance measurement data 100 of the lower stage 2A or the upper stage 2B and the shelf frame 3 cannot be detected from the other distance measurement data 100, a virtual shelf frame 50C may be set for the distance measurement data 100 of the shelf stage that cannot be detected based on the detected shelf frame 3 of the detected shelf stage.

[0114] That is, when the information processing apparatus 22 cannot detect the shelf frame 3 for one of the shelf levels adjacent to each other, the information processing apparatus 22 may set a virtual shelf frame 50C for the shelf level where the shelf frame 3 cannot be detected, based on the detected shelf frame 3 adjacent to that shelf level.

[0115] Thereby, the information processing apparatus 22 can set the virtual shelf frame 50C and calculate the filling rate of the luggage 9 even for a shelf level where the shelf frame 3 cannot be detected.

[0116] (Summary of the present disclosure) According to the description of the above Embodiment 1, the following techniques are disclosed.

[0117] <Technology 1> A measuring device (20) for measuring the filling rate of an object (for example, luggage 9) arranged on a shelf (1) according to the present disclosure includes a processor (23) and a memory (24). The processor, in cooperation with the memory, detects a shelf frame (3) from distance measurement data (100) obtained by three-dimensionally scanning the shelf, sets virtual shelf frames (50A, 50B, 50C), which are virtual shelf frames, in an area where no shelf frame is detected and which is adjacent to the detected shelf frame in the distance measurement data, and calculates the filling rate of the object within the virtual shelf frame. Thereby, the measuring device (20) can set a virtual shelf frame and calculate the filling rate even for an area where the shelf frame cannot be detected (for example, shelf upper 2C or a shelf level at least a part of which is not recognized by the three-dimensional scan).

[0118] <Technology 2> In the measuring device according to Technology 1, the processor sets the virtual shelf frame in an area where no shelf frame is detected and which is adjacent to above or below the detected shelf frame. Thereby, the measuring device can set a virtual shelf frame and calculate the filling rate even for, for example, shelf upper 2C adjacent to above the detected shelf frame of the upper stage 2B, or the upper stage 2B at least a part of which is not recognized by the three-dimensional scan and which is adjacent to above the detected shelf frame of the lower stage 2A, or the lower stage 2A at least a part of which is not recognized by the three-dimensional scan and which is adjacent to below the detected shelf frame of the upper stage 2B.

[0119] <Technology 3> In the measuring device according to Technology 1 or 2, the processor determines the size of the virtual shelf frame based on the detected size of the shelf frame. Thereby, the measuring device can appropriately determine the size of the virtual shelf frame.

[0120] <Technology 4> In the measuring device according to any one of Technologies 1 to 3, the processor sets the virtual shelf frame in an area where the shelf frame is not detected, adjacent to the uppermost shelf frame of which at least a part is detected. Thereby, the measuring device can set a virtual shelf frame on the shelf.

[0121] <Technology 5> In the measuring device according to Technology 4, the processor determines the width of the virtual shelf frame based on the width of the uppermost shelf frame of which at least a part is detected. Thereby, the measuring device can appropriately determine the width of the virtual shelf frame.

[0122] <Technology 6> In the measuring device according to Technology 4 or 5, the processor determines the height of the virtual shelf frame to a predetermined height. Thereby, the measuring device can appropriately determine the height of the virtual shelf frame.

[0123] <Technology 7> In the measuring device according to Technology 6, the predetermined height is a value based on the upper limit of the loading height of the object above the uppermost shelf frame. Thereby, the measuring device can appropriately determine the height of the virtual shelf frame.

[0124] <Technology 8> In the measuring device according to any one of Technologies 4 to 7, when the ratio of the object occupying the virtual shelf frame is equal to or greater than a predetermined threshold, the processor determines that there is an object above the uppermost shelf frame, or calculates the virtual filling rate of the virtual shelf frame as 100%. As a result, even when the angular range of the distance measurement data does not cover the entire space above the shelves, the measuring device can measure the situation of the objects on the shelves.

[0125] <Technology 9> In the measuring device according to Technology 8, from one piece of distance measurement data, calculate the filling rate of the uppermost shelf frame and the provisional filling rate of the virtual shelf frame. As a result, the measuring device can calculate the filling rate of the uppermost shelf frame and the provisional filling rate of the virtual shelf frame from one piece of distance measurement data at once.

[0126] <Technology 10> In the measuring device according to any one of claims 1 to 8, the distance measurement data includes at least a depth image, and the processor detects the shelf frame using the depth image. As a result, the measuring device can detect the shelf frames of the shelves.

[0127] <Technology 11> In the measuring device according to any one of Technologies 1 to 10, the processor detects the shelf frame corresponding to the shelf posts extending in the height direction and the shelf boards extending in the width direction that constitute the shelf from the distance measurement data. As a result, the measuring device can detect the shelf frame from the shelves constituted by the shelf posts and the shelf boards and overlapping in the height direction.

[0128] <Technology 12> In a measurement system (10) comprising a measurement device (20) for measuring a filling rate of an object (e.g., a package 9) arranged on a shelf (1) according to the present disclosure, and a server device (30) capable of communicating with the measurement device, the measurement device detects a shelf frame from distance measurement data obtained by three-dimensionally scanning the shelf, sets a virtual shelf frame, which is a virtual shelf frame, in a region adjacent to the detected shelf frame in the distance measurement data where no shelf frame is detected, calculates the filling rate for the object within the virtual shelf frame, transmits the distance measurement data and the calculated filling rate to the server device, and the server device displays the distance measurement data and the filling rate received from the measurement device. Thereby, the measurement device (20) can set a virtual shelf frame and calculate the filling rate even for a region where a shelf frame cannot be detected (e.g., on the upper part 2C of the shelf or a shelf level at least partially not recognized by the three-dimensional scan). Further, the server device 30 can display the filling rate calculated in this way and the distance measurement data used for calculating the filling rate.

[0129] <Technology 13> In the measurement system according to Technology 12, the distance measurement data includes at least an RGB image, and the server device superimposes and displays at least one of the shelf frame and the virtual shelf frame and the filling rate on the RGB image of the shelf taken. Thereby, since at least one of the shelf frame and the virtual shelf frame and the filling rate are superimposed and displayed on the image of the shelf taken, the user can easily know the filling rate on each shelf or on the shelf.

[0130] <Technology 14> A measurement method for measuring a filling rate of an object (e.g., a package 9) arranged on a shelf (1) according to the present disclosure detects a shelf frame from distance measurement data obtained by three-dimensionally scanning the shelf, sets a virtual shelf frame, which is a virtual shelf frame, in a region adjacent to the detected shelf frame in the distance measurement data where no shelf frame is detected, calculates the filling rate for the object within the virtual shelf frame, and displays the distance measurement data and the filling rate received from the measurement device. As a result, even for areas where the shelf frame cannot be detected (for example, above the shelf 2C or a shelf step that is not recognized by the three-dimensional scan at least in part), a virtual shelf frame can be set and the filling rate can be calculated. Further, the filling rate calculated in this way and the distance measurement data used for the calculation of the filling rate can be displayed.

[0131] As described above, the embodiments have been described with reference to the accompanying drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive various modification examples, correction examples, substitution examples, addition examples, deletion examples, and equivalent examples within the scope described in the claims, and it is understood that they also belong to the technical scope of the present disclosure. Further, within the scope not departing from the gist of the invention, the respective components in the above-described embodiments may be arbitrarily combined.

Industrial Applicability

[0132] The technology of the present disclosure is useful for measuring the storage status of goods on a shelf.

Explanation of Signs

[0133] 1 Shelf 2A Lower stage 2B Upper stage 2C Above the shelf 3, 3A, 3B Shelf frame 4, 4B Shelf board 5, 5A, 5B Shelf post 9 Goods 10 Measurement system 11 Communication network 20 Measuring device 21, 21A, 21B, 21C Distance measurement sensor 22 Information processing device 23 Processor 24 Memory 25 Communication device 26 Equipment connection device 27 Input device 28 Output device 30 Server device 31 Processor 32 Memory 33 Communication device 34 Input device 35 Output device 50A, 50B, 50C Virtual shelf frame 41, 42, 51, 52, 53 Line segment 100, 100A, 100B, 100C Distance measurement data 101A, 101B, 101C, 101D Measurement range 200 Utilization efficiency screen 201 Area of the filling rate inside the shelves of the entire warehouse 202 Area of the filling rate above the shelves of the entire warehouse 203 Area of the filling rate inside the shelves by column 204 Area of the filling rate by level 300 Cargo aggregation support screen 301 Area of the filling rate distribution 302 Area of the list of shelf numbers 400 Shelf vacancy status screen 401 Area of the upper threshold value of the filling rate 402 Area of the area map 403 Area of the filling rate of each shelf 404 Area of the distance measurement data 411 Shelf icon

Claims

1. A measuring device for measuring the filling rate of an object placed on a shelf, comprising a processor and a memory, wherein the processor, in cooperation with the memory, detects a shelf frame from ranging data obtained by three-dimensionally scanning the shelf, sets a virtual shelf frame, which is a virtual shelf frame, in an area where no shelf frame is detected and which is adjacent to the detected shelf frame in the ranging data, calculates the filling rate of the object within the virtual shelf frame, measuring device.

2. The processor sets the virtual shelf frame in an area where no shelf frame is detected and which is adjacent above or below the detected shelf frame, The measuring device according to claim 1.

3. The processor determines the size of the virtual shelf frame based on the size of the detected shelf frame, The measuring device according to claim 2.

4. The processor sets the virtual shelf frame in an area where no shelf frame is detected and which is adjacent above the uppermost shelf frame of which at least a part is detected, The measuring device according to claim 1.

5. The processor determines the width of the virtual shelf frame based on the width of the uppermost shelf frame of which at least a part is detected, The measuring device according to claim 4.

6. The processor determines the height of the virtual shelf frame to a predetermined height, The measuring device according to claim 4.

7. The predetermined height is a value based on the upper limit of the stacking height of the object above the uppermost shelf frame, The measuring device according to claim 6.

8. When the ratio of the object occupying the virtual shelf frame is equal to or greater than a predetermined threshold, the processor determines that there is an object above the uppermost shelf frame, or calculates the virtual filling rate of the virtual shelf frame as 100%, The measuring device according to claim 4.

9. The processor calculates the filling rate of the uppermost shelf frame and the virtual filling rate of the virtual shelf frame from one piece of ranging data, The measuring device according to claim 8.

10. The ranging data includes at least a depth image, The processor detects the shelf frame using the depth image, The measuring device according to claim 1.

11. The processor detects the shelf frame corresponding to the shelf posts extending in the height direction and the shelf boards extending in the width direction that constitute the shelf from the ranging data, The measuring device according to any one of claims 1 to 10.

12. A measurement system comprising a measurement device for measuring the filling rate of an object placed on a shelf and a server device capable of communicating with the measurement device, wherein the measurement device, detects a shelf frame from distance measurement data obtained by three-dimensionally scanning the shelf, sets a virtual shelf frame, which is a virtual shelf frame, in an area where no shelf frame is detected and adjacent to the detected shelf frame in the distance measurement data, calculates the filling rate of the object within the virtual shelf frame, transmits the distance measurement data and the calculated filling rate to the server device, wherein the server device, displays the distance measurement data and the filling rate received from the measurement device, measurement system.

13. The distance measurement data includes at least an RGB image, and the server device superimposes and displays at least one of the shelf frame and the virtual shelf frame and the filling rate on the RGB image of the shelf that has been photographed, The measurement system according to claim 12.

14. A measurement method for measuring the filling rate of an object placed on a shelf, detecting a shelf frame from distance measurement data obtained by three-dimensionally scanning the shelf, setting a virtual shelf frame, which is a virtual shelf frame, in an area where no shelf frame is detected and adjacent to the detected shelf frame in the distance measurement data, calculating the filling rate of the object within the virtual shelf frame, displaying the distance measurement data and the filling rate, measurement method.

Images