Method and apparatus

Abstract

The present invention provides a method for calculating the filling rate of an object to be measured. [Solution] The method involves obtaining a first filling rate of the object to be measured for each of the one or more first storage units stored in the second storage space of a second storage unit having a second storage space capable of storing a plurality of first storage units, obtaining the number of first storage units that can be additionally stored in the second storage space, and calculating a second filling rate of the object to be measured for the second storage space based on the obtained first filling rate and the number of first storage units that can be additionally stored.

Description

Technical Field

[0001] This disclosure relates to a method and an apparatus.

Background Art

[0002] Patent Document 1 discloses a three-dimensional shape measurement apparatus that acquires a three-dimensional shape using a three-dimensional laser scanner.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Sufficient consideration has not been given to application examples of the measured three-dimensional shape. For example, sufficient consideration has not been given to the calculation of the filling rate indicating how much of the measurement object is stored in a predetermined storage space.

[0005] This disclosure provides a filling rate measurement method and the like that can calculate the filling rate of a measurement object.

Means for Solving the Problems

[0006] The method according to one aspect of this disclosure acquires, for each of one or more first storage units stored in the second storage space of a second storage unit having a second storage space capable of storing a plurality of first storage units, a first filling rate of a measurement object with respect to the first storage space, acquires the number of the first storage units that can be additionally stored in the second storage space, and calculates a second filling rate of the measurement object with respect to the second storage space based on the acquired first filling rate and the number of the first storage units that can be additionally stored.

[0007] An apparatus according to one aspect of the present disclosure is an apparatus for calculating a second filling rate of an object to be measured in a second storage space having a second storage space capable of storing a plurality of first storage units, comprising a memory and a processor, wherein the processor uses the memory to obtain a first filling rate of the object to be measured in the first storage space for each of the one or more first storage units stored in the second storage space, obtains the number of first storage units that can be additionally stored in the second storage space, and calculates a second filling rate of the object to be measured in the second storage space based on the obtained first filling rate and the number of first storage units that can be additionally stored.

[0008] A filling rate measurement method according to one aspect of the present disclosure includes a first storage space in which an object to be measured is stored from an opening, and a first storage unit having a plurality of through holes and an opening / closing unit arranged to cover the opening, and a spatial three-dimensional model obtained by measuring the first storage unit through the plurality of through holes with a distance measuring sensor facing the first storage unit, a storage three-dimensional model of the first storage unit when the object to be measured is not stored therein, an object three-dimensional model of the object to be measured in the first storage space is estimated using the acquired spatial three-dimensional model and the storage three-dimensional model, a first filling rate of the object to be measured in the first storage space is calculated using the storage three-dimensional model and the object three-dimensional model, and in the estimation, the shape of the second part of the object to be measured hidden from the distance measuring sensor is estimated based on the shape of the first part of the object to be measured measured by the distance measuring sensor through the plurality of through holes in the direction from the distance measuring sensor toward the object to be measured, and the object three-dimensional model is estimated using the first part, the second part and the storage three-dimensional model.

[0009] An information processing device according to one aspect of the present disclosure comprises a processor and a memory, the processor using the memory to acquire a spatial three-dimensional model obtained by measuring a first storage space in which an object to be measured is stored through an opening, and a first storage unit having a plurality of through holes and an opening / closing unit having a plurality of through holes and being arranged to cover the opening in a closed state, through the plurality of through holes by a distance measuring sensor facing the first storage unit, and acquiring a storage three-dimensional model of the first storage unit when the object to be measured is not stored there, and using the acquired spatial three-dimensional model and the storage three-dimensional model, The three-dimensional model of the object to be measured within the first storage space is estimated, and the first filling rate of the object to be measured in the first storage space is calculated using the storage three-dimensional model and the object three-dimensional model. In the estimation, the shape of the second part of the object to be measured, which is hidden from the distance measuring sensor, is estimated based on the shape of the first part of the object to be measured measured by the distance measuring sensor through the plurality of through holes in the direction from the distance measuring sensor toward the object to be measured, and the three-dimensional model of the object is estimated using the first part, the second part and the storage three-dimensional model.

[0010] A method for measuring the filling rate according to one aspect of the present disclosure involves obtaining a spatial three-dimensional model obtained by measuring a first storage unit having a first storage space in which an object to be measured is stored and having an opening formed therein, through the opening by a distance measuring sensor facing the first storage unit, obtaining a storage three-dimensional model which is a three-dimensional model of the first storage unit in which the object to be measured is not stored, extracting an object portion which is the part of the spatial three-dimensional model corresponding to the object to be measured, using the acquired spatial three-dimensional model and the storage three-dimensional model, estimating an object three-dimensional model which is a three-dimensional model of the object to be measured in the first storage space using the extracted object portion, and calculating a first filling rate of the object to be measured in the first storage space using the storage three-dimensional model and the object three-dimensional model.

[0011] An information processing device according to one aspect of the present disclosure comprises a processor and a memory, wherein the processor uses the memory to obtain a spatial three-dimensional model obtained by measuring a first storage unit having a first storage space in which an object to be measured is stored and having an opening formed therein, through the opening from the first direction side by a distance measuring sensor facing the first storage unit; obtains a storage three-dimensional model which is a three-dimensional model of the first storage unit in which the object to be measured is not stored; uses the obtained spatial three-dimensional model and the storage three-dimensional model to extract an object portion which corresponds to the object to be measured from the storage three-dimensional model; uses the extracted object portion to estimate an object three-dimensional model which is a three-dimensional model of the object to be measured in the first storage space; and uses the storage three-dimensional model and the object three-dimensional model to calculate a first filling rate of the object to be measured in the first storage space.

[0012] Furthermore, this disclosure may be implemented as a program that causes a computer to execute the steps included in the above-described filling rate measurement method. Alternatively, this disclosure may be implemented as a non-temporary recording medium such as a CD-ROM readable by the computer on which the program is recorded. Furthermore, this disclosure may be implemented as information, data, or signals representing the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet. [Effects of the Invention]

[0013] According to this disclosure, a method for measuring the filling rate of an object to be measured can be provided. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 is a diagram illustrating the outline of the filling rate measurement method according to an embodiment. [Figure 2] Figure 2 is a block diagram showing the characteristic configuration of the three-dimensional measurement system according to the embodiment. [Figure 3]FIG. 3 is a diagram for explaining a first example of the configuration of the distance measurement sensor. [Figure 4] FIG. 4 is a diagram for explaining a second example of the configuration of the distance measurement sensor. [Figure 5] FIG. 5 is a diagram for explaining a third example of the configuration of the distance measurement sensor. [Figure 6] FIG. 6 is a block diagram showing the configuration of the coordinate system calculation unit in the first example. [Figure 7] FIG. 7 is a diagram for explaining a method of calculating the measurement coordinate system by the coordinate system calculation unit in the first example. [Figure 8] FIG. 15 is a diagram for explaining an example of a method of calculating the filling rate by the filling rate calculation unit. [Figure 9] FIG. 8 is a block diagram showing the configuration of the coordinate system calculation unit in the second example. [Figure 10] FIG. 9 is a diagram for explaining a method of calculating the measurement coordinate system by the coordinate system calculation unit in the second example. [Figure 11] FIG. 10 is a block diagram showing the configuration of the coordinate system calculation unit in the third example. [Figure 12] FIG. 11 is a diagram for explaining a method of calculating the measurement coordinate system by the coordinate system calculation unit in the third example. [Figure 13] FIG. 12 is a block diagram showing an example of the configuration of the model generation unit. [Figure 14] FIG. 13 is a flowchart of a process for calculating the volume of the storage space by the model generation unit. [Figure 15] FIG. 14 is a block diagram showing an example of the configuration of the filling rate calculation unit. [Figure 16] FIG. 16 is a diagram for explaining another example of a method of calculating the filling rate by the filling rate calculation unit. [Figure 17] FIG. 17 is a flowchart of a filling rate measurement method performed by the information processing apparatus. [Figure 18] FIG. 18 is a flowchart of a process for calculating the measurement coordinate system by the coordinate system calculation unit in the first example. [[ID=4s7]] [Figure 19]Figure 19 is a flowchart of the process for calculating the measurement coordinate system by the coordinate system calculation unit in the second example. [Figure 20] Figure 20 is a flowchart of the process for calculating the measurement coordinate system by the coordinate system calculation unit in the third example. [Figure 21] Figure 21 is a diagram illustrating the method for calculating the filling rate. [Figure 22] Figure 22 is a block diagram showing an example of the configuration of the calculation unit of the filling rate calculation unit according to Modification Example 1. [Figure 23] Figure 23 is a flowchart of the filling rate calculation process of the calculation unit of the filling rate calculation unit according to Modified Example 1. [Figure 24] Figure 24 shows an example of storing two or more shelves in a storage space such as the back of a truck. [Figure 25] Figure 25 is a table showing the relationship between the shelves stored in the cargo bed's storage space and their occupancy rate. [Figure 26] Figure 26 is a block diagram showing an example of the configuration of the calculation unit of the filling rate calculation unit according to Modification Example 2. [Figure 27] Figure 27 is a flowchart of the filling rate calculation process of the calculation unit of the filling rate calculation unit according to Modified Example 2. [Figure 28] Figure 28 is a diagram illustrating the configuration of the cage trolley according to Modification 3. [Figure 29] Figure 29 is a block diagram showing an example of the configuration of the filling rate calculation unit according to Modification Example 3. [Figure 30] Figure 30 is a flowchart of the filling rate calculation process of the filling rate calculation unit according to Modification Example 3. [Figure 31] Figure 31 illustrates an example of a second method for calculating the filling rate. [Figure 32] Figure 32 illustrates another example of a second method for calculating the filling rate. [Figure 33] Figure 33 is a diagram illustrating the method for generating a three-dimensional spatial model related to Modification 4. [Figure 34]Figure 34 is a diagram illustrating the method for generating a three-dimensional spatial model related to Modification 5. [Modes for carrying out the invention]

[0015] (Background leading to this disclosure) In logistics and distribution, there is a need to measure the occupancy rate of goods and other items in storage spaces to improve the efficiency of storage space utilization. Furthermore, since many items are stored in containers and other storage units in logistics and distribution settings, there is a need to measure a large number of occupancy rates in a short amount of time. However, methods for easily measuring occupancy rates have not been sufficiently considered.

[0016] Therefore, this disclosure provides a method for measuring the filling rate of a large number of storage units in a short time by applying a method for generating a three-dimensional model of the storage unit in which the object to be measured is stored.

[0017] A method for measuring the filling rate according to one aspect of the present disclosure involves obtaining a spatial three-dimensional model obtained by measuring a first storage unit having a first storage space in which an object to be measured is stored and having an opening formed therein, through the opening by a distance measuring sensor facing the first storage unit, obtaining a storage three-dimensional model which is a three-dimensional model of the first storage unit in which the object to be measured is not stored, extracting an object portion which is the part of the spatial three-dimensional model corresponding to the object to be measured, using the acquired spatial three-dimensional model and the storage three-dimensional model, estimating an object three-dimensional model which is a three-dimensional model of the object to be measured in the first storage space using the extracted object portion, and calculating a first filling rate of the object to be measured in the first storage space using the storage three-dimensional model and the object three-dimensional model.

[0018] According to this method, the three-dimensional object model of the object to be measured is estimated using the three-dimensional spatial model of the first storage unit with the object to be measured stored inside, and the object portion extracted using the three-dimensional storage model of the first storage unit without the object to be measured stored inside. As a result, the first filling rate of the object to be measured in the first storage space can be easily calculated simply by measuring the first storage unit with the object to be measured stored inside.

[0019] Furthermore, in the estimation described above, the three-dimensional model of the object may be estimated based on a first three-dimensional coordinate system that uses the shape of a part of the first storage unit as a reference.

[0020] Therefore, the processing load for estimating the three-dimensional model of the object can be reduced.

[0021] Furthermore, the first three-dimensional coordinate system may be calculated based on only the shape of a part of the first storage unit.

[0022] Therefore, the shape of only a portion of the first storage unit, which is easily extracted from the image, can be used to calculate the first three-dimensional coordinate system. As a result, the processing speed of estimating the three-dimensional model of the object can be improved, and the accuracy of calculating the first three-dimensional coordinate system can be improved.

[0023] Furthermore, the aforementioned partial shape may be the shape of the opening.

[0024] Therefore, a coordinate system based on the shape of the opening can be easily calculated, and a three-dimensional model of the object can be estimated based on the calculated coordinate system.

[0025] Furthermore, in the estimation described above, the three-dimensional model of the object may be estimated based on a first three-dimensional coordinate system that uses the position of the marker installed in the first storage unit as a reference.

[0026] Therefore, a coordinate system based on the marker can be easily calculated, and a three-dimensional model of the object can be estimated based on the calculated coordinate system.

[0027] Furthermore, in the estimation method described above, the three-dimensional model of the object may be estimated by estimating the shape of a second portion that does not face the object, based on the shape of a first portion that faces the object in the direction from the distance measuring sensor toward the object.

[0028] Therefore, even if the distance measuring sensor has a second part of its shape that does not face the object being measured, it is possible to estimate the three-dimensional model of the object.

[0029] Furthermore, the first storage section further has an opening / closing section having a through hole and being arranged to cover the opening when closed, wherein the first portion is the portion of the opening / closing section in the closed state that faces the through hole in the direction, and the second portion is the portion hidden by the opening / closing section in the closed state in the direction, and the filling rate measurement method further determines whether the opening / closing section is in an open state or a closed state, estimates the three-dimensional model of the object by performing the extraction and estimation when the opening / closing section is in the open state, estimates the second portion based on the first portion, and estimates the three-dimensional model of the object using the first portion, the estimated second portion and the storage three-dimensional model.

[0030] According to this, even when the object to be measured is stored in a first storage compartment equipped with an opening / closing mechanism, the method for estimating the object's three-dimensional model is switched according to the opening / closing state of the mechanism, thereby enabling the appropriate estimation of the object's three-dimensional model.

[0031] Furthermore, the aforementioned direction may also be aligned with the horizontal direction.

[0032] Therefore, since there is no need to adjust the position of the distance measuring sensor so that measurements can be taken from a direction without an opening or closing part having a through hole, there is a high degree of flexibility in the installation of the distance measuring sensor. Thus, even if the position of the distance measuring sensor has not been fully adjusted, measurement results for estimating a three-dimensional model of the object using the distance measuring sensor can be obtained.

[0033] Furthermore, in the above calculation, the first filling rate may be calculated as the ratio of the volume of the object to be measured stored in the first storage space to the volume of the space in the first storage space in which the object to be measured can be stored.

[0034] Therefore, a first filling rate can be calculated to appropriately determine how many objects to be measured can be stored in the empty space of the first storage space.

[0035] Furthermore, the first storage unit and the additional first storage unit are stored in the second storage space of the second storage unit, and the filling rate measurement method may further calculate the second filling rate of the first storage unit and the additional first storage unit relative to the second storage space.

[0036] Therefore, the second filling rate can be appropriately calculated when one or more first storage units are stored in the second storage space.

[0037] Furthermore, the stored three-dimensional model may be a three-dimensional model measured by the distance measuring sensor and additional distance measuring sensors.

[0038] Therefore, it is possible to generate a stored three-dimensional model with minimal occlusion.

[0039] Furthermore, the distance measuring sensor has at least two cameras for generating the spatial three-dimensional model and is fixed on the upper side of the first storage unit.

[0040] In this way, when the distance measuring sensor is fixed to the upper side of the first storage unit, the two cameras of the distance measuring sensor capture only the ground or the base (bottom) of the first storage unit, and since there is nothing that can move other than the first storage unit, it is easy to isolate the object to be measured from the background.

[0041] An information processing device according to one aspect of the present disclosure comprises a processor and a memory, wherein the processor uses the memory to obtain a spatial three-dimensional model obtained by measuring a first storage unit having a first storage space in which an object to be measured is stored and having an opening formed therein, through the opening from the first direction side by a distance measuring sensor facing the first storage unit; obtains a storage three-dimensional model which is a three-dimensional model of the first storage unit in which the object to be measured is not stored; uses the obtained spatial three-dimensional model and the storage three-dimensional model to extract an object portion which corresponds to the object to be measured from the storage three-dimensional model; uses the extracted object portion to estimate an object three-dimensional model which is a three-dimensional model of the object to be measured in the first storage space; and uses the storage three-dimensional model and the object three-dimensional model to calculate a first filling rate of the object to be measured in the first storage space.

[0042] According to this method, the three-dimensional object model of the object to be measured is estimated using the three-dimensional spatial model of the first storage unit with the object to be measured stored inside, and the object portion extracted using the three-dimensional storage model of the first storage unit without the object to be measured stored inside. As a result, the first filling rate of the object to be measured in the first storage space can be easily calculated simply by measuring the first storage unit with the object to be measured stored inside.

[0043] Furthermore, this disclosure may be implemented as a program that causes a computer to execute the steps included in the above-described filling rate measurement method. Alternatively, this disclosure may be implemented as a non-temporary recording medium such as a CD-ROM readable by the computer on which the program is recorded. Furthermore, this disclosure may be implemented as information, data, or signals representing the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

[0044] In the following, each embodiment of the filling rate measurement method, etc., relating to this disclosure will be described in detail with reference to the drawings. Note that each embodiment described below is merely one specific example of this disclosure. Therefore, the numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in each embodiment below are examples only and are not intended to limit this disclosure.

[0045] Furthermore, each figure is a schematic diagram and not necessarily a strictly accurate representation. Also, in each figure, substantially identical components are given the same reference numerals, and redundant explanations may be omitted or simplified.

[0046] (Embodiment) Referring to Figure 1, an overview of the filling rate measurement method according to the embodiment will be described.

[0047] Figure 1 is a diagram illustrating the outline of the filling rate measurement method according to an embodiment.

[0048] In the filling rate measurement method, as shown in Figure 1, the amount of luggage 103 stored in a shelf 102 having a storage space 101 is measured using a distance measuring sensor 210. Then, the filling rate of the luggage 103 relative to the storage space 101 is calculated using the obtained measurement results. The shelf 102 has an opening 102a for loading and unloading luggage 103 into and out of the storage space 101. The distance measuring sensor 210 is positioned opposite the opening 102a of the shelf 102 and is oriented to measure the shelf 102 including the opening 102a, measuring the measurement area R1 including the inside of the storage space 101 through the opening 102a.

[0049] Note that the shelf 102 has a box-like shape, for example, as shown in Figure 1. The shelf does not have to have a box-like shape, as long as it has a mounting surface on which the cargo 103 is placed and a storage space 101 above the mounting surface where the cargo 103 is stored. The shelf 102 is an example of a first storage section. The storage space 101 is an example of a first storage space. Although the storage space 101 is described as the internal space of the shelf 102, it is not limited to this and may be a space in a warehouse where the object to be measured, such as cargo 103, is stored. Cargo 103 is an example of an object to be measured. The object to be measured is not limited to cargo 103, but may be merchandise. In other words, the object to be measured may be any portable object.

[0050] Figure 2 is a block diagram illustrating a characteristic configuration of a three-dimensional measurement system according to an embodiment. Figure 3 is a diagram illustrating a first example of the configuration of a distance measuring sensor. Figure 4 is a diagram illustrating a second example of the configuration of a distance measuring sensor. Figure 5 is a diagram illustrating a third example of the configuration of a distance measuring sensor.

[0051] As shown in Figure 2, the three-dimensional measurement system 200 includes a distance measuring sensor 210 and an information processing device 220. The three-dimensional measurement system 200 may have multiple distance measuring sensors 210 or it may have one distance measuring sensor 210.

[0052] The distance measuring sensor 210 obtains measurement results including the shelf 102 and its storage space 101 by measuring the three-dimensional space including the storage space 101 of the shelf 102 through the opening 102a of the shelf 102. Specifically, the distance measuring sensor 210 generates a three-dimensional spatial model represented by a set of three-dimensional points that indicate the three-dimensional position of each of several measurement points on the shelf 102 or the luggage 103 (hereinafter referred to as the measurement target). The set of three-dimensional points is called a three-dimensional point cloud. The three-dimensional position indicated by each three-dimensional point in the three-dimensional point cloud is represented, for example, by three-dimensional coordinates of ternary information consisting of the X, Y, and Z components of a three-dimensional coordinate space consisting of XYZ axes. Note that the three-dimensional model may include not only three-dimensional coordinates but also color information indicating the color of each point, or shape information representing the surface shape of each point and its surroundings. The color information may be represented, for example, in the RGB color space, or in another color space such as HSV, HLS, or YUV.

[0053] A specific example of the distance measuring sensor 210 will be explained using Figures 3 to 5.

[0054] The distance measuring sensor 210 in the first example generates a three-dimensional spatial model by emitting electromagnetic waves and acquiring reflected waves that are reflected from the object being measured, as shown in Figure 3. Specifically, the distance measuring sensor 210 measures the time it takes for the emitted electromagnetic waves to be reflected from the object being measured and return to the distance measuring sensor 210, and uses the measured time and the wavelength of the electromagnetic waves used for measurement to calculate the distance between the distance measuring sensor 210 and a point P1 on the surface of the object being measured. The distance measuring sensor 210 emits electromagnetic waves in a predetermined radial direction from a reference point of the distance measuring sensor 210. For example, the distance measuring sensor 210 may emit electromagnetic waves at a first angular interval around the horizontal direction and at a second angular interval around the vertical direction. Therefore, the distance measuring sensor 210 can calculate the three-dimensional coordinates of multiple points on the object being measured by detecting the distance between the distance measuring sensor 210 and the object being measured in each of the multiple directions around the distance measuring sensor 210. Therefore, the distance measuring sensor 210 can calculate positional information indicating multiple three-dimensional positions on the object to be measured, and can generate a three-dimensional spatial model containing the positional information. The positional information may also be a three-dimensional point cloud containing multiple three-dimensional points indicating multiple three-dimensional positions.

[0055] The first example of the distance measuring sensor 210, as shown in Figure 3, is a three-dimensional laser measuring instrument having a laser irradiation unit 211 that irradiates laser light as an electromagnetic wave, and a laser light receiving unit 212 that receives reflected light reflected from the object to be measured. The distance measuring sensor 210 scans the object to be measured with laser light by rotating or oscillating a unit comprising the laser irradiation unit 211 and the laser light receiving unit 212 on two different axes, or by installing a movable mirror (MEMS (Micro Electro Mechanical Systems) mirror) that oscillates on two axes in the path of the irradiated or received laser. As a result, the distance measuring sensor 210 can generate a highly accurate and high-density three-dimensional model of the object to be measured.

[0056] The distance measuring sensor 210 is exemplified as a three-dimensional laser measuring instrument that measures the distance to the object by irradiating it with laser light, but it is not limited to this, and may also be a millimeter-wave radar measuring instrument that measures the distance to the object by emitting millimeter waves.

[0057] Furthermore, the distance measuring sensor 210 may generate a three-dimensional model having color information. The first color information is color information generated using an image captured by the distance measuring sensor 210, and is color information indicating the color of each of the multiple first three-dimensional points included in the first three-dimensional point cloud.

[0058] Specifically, the distance measuring sensor 210 may have a built-in camera that photographs the object to be measured around the distance measuring sensor 210. The camera built into the distance measuring sensor 210 generates an image by photographing the area that includes the irradiation range of the laser beam emitted by the distance measuring sensor 210. Furthermore, the shooting range captured by the camera is pre-associated with the irradiation range. Specifically, multiple directions in which the laser beam is emitted by the distance measuring sensor 210 are pre-associated with each pixel in the image captured by the camera, and the distance measuring sensor 210 sets the pixel value of the image associated with the direction of the three-dimensional point as color information indicating the color of each of the multiple three-dimensional points included in the three-dimensional point cloud.

[0059] The second example of the distance measuring sensor 210A is a distance measuring sensor that uses structured light, as shown in Figure 4. The distance measuring sensor 210A has an infrared pattern projection unit 211A and an infrared camera 212A. The infrared pattern projection unit 211A projects a predetermined infrared pattern 213A onto the surface of the object to be measured. The infrared camera 212A acquires an infrared image by photographing the object to be measured onto which the infrared pattern 213A is projected. The distance measuring sensor 210A searches for the infrared pattern 213A included in the obtained infrared image and calculates the distance from the infrared pattern projection unit 211A or the infrared camera 212A to the position of point P1 on the object to be measured based on a triangle formed by connecting the position of a point P1 in the infrared pattern on the object to be measured in real space, the position of the infrared pattern projection unit 211A, and the position of the infrared camera 212A. In this way, the distance measuring sensor 210A can acquire a three-dimensional point of the measurement point on the object to be measured.

[0060] Furthermore, the distance measuring sensor 210A can acquire a high-density three-dimensional model by moving the unit of the distance measuring sensor 210A, which has an infrared pattern irradiation unit 211A and an infrared camera 212A, or by making the infrared pattern irradiated by the infrared pattern irradiation unit 211A a fine texture.

[0061] Furthermore, the distance measuring sensor 210A may generate a three-dimensional model with color information by using the visible light region of the color information acquired by the infrared camera 212A, and by associating the obtained visible light region with three-dimensional points, taking into account the position or orientation of the infrared pattern irradiation unit 211A or the infrared camera 212A. The distance measuring sensor 210A may also be configured to further include a visible light camera for adding color information.

[0062] The third example, the distance measuring sensor 210B, is a distance measuring sensor that measures three-dimensional points using stereo camera measurement, as shown in Figure 5. The distance measuring sensor 210B is a stereo camera having two cameras 211B and 212B. The distance measuring sensor 210B acquires a stereo image with parallax by photographing the object to be measured at synchronized timings with the two cameras 211B and 212B. The distance measuring sensor 210B uses the obtained stereo image (two images) to perform feature point matching processing between the two images and acquires alignment information between the two images with pixel accuracy or fractional pixel accuracy. The distance measuring sensor 210B calculates the distance from either camera 211B or 212B to the matching position on the object to be measured (i.e., point P1) based on a triangle formed by connecting the matching position of a point P1 on the object to be measured in real space and the positions of the two cameras 211B and 212B. This allows the distance measuring sensor 210B to acquire the three-dimensional point of the measurement point on the object being measured.

[0063] Furthermore, the distance measuring sensor 210B can acquire a highly accurate three-dimensional model by moving the unit of the distance measuring sensor 210B, which has two cameras 211B and 212B, or by increasing the number of cameras mounted on the distance measuring sensor 210B to three or more, and photographing the same measurement target and performing matching processing.

[0064] Furthermore, by using visible light cameras 211B and 212B in the distance measuring sensor 210B, it becomes easy to add color information to the acquired three-dimensional model.

[0065] In this embodiment, the information processing device 220 is described as an example that includes the distance measuring sensor 210 of the first example, but it may also be configured to include the distance measuring sensor 210A of the second example or the distance measuring sensor 210B of the third example instead of the distance measuring sensor 210 of the first example.

[0066] The two cameras 211B and 212B can capture monochrome images including visible light images or infrared images. In this case, the matching process between the two images in the three-dimensional measurement system 200 can be performed using, for example, SLAM (Simultaneous Localization And Mapping) or S This may also be done using fM (Structure from Motion). The position and orientation information of cameras 211B and 212B obtained may be used to increase the point cloud density of the measurement space model using MVS (Multi View Stereo).

[0067] Returning to Figure 2, the configuration of the information processing device 220 will be explained.

[0068] The information processing device 220 includes an acquisition unit 221, a coordinate system calculation unit 222, a model generation unit 223, a filling rate calculation unit 224, and a storage unit 225.

[0069] The acquisition unit 221 acquires the spatial three-dimensional model and image generated by the distance measuring sensor 210. Specifically, the acquisition unit 221 may acquire the spatial three-dimensional model and image from the distance measuring sensor 210. The spatial three-dimensional model and image acquired by the acquisition unit 221 may be stored in the storage unit 225.

[0070] The coordinate system calculation unit 222 calculates the positional relationship between the distance measuring sensor 210 and the shelf 102 using a three-dimensional spatial model and images. Based on this, the coordinate system calculation unit 222 calculates a measurement coordinate system based on a portion of the shape of the shelf 102. The coordinate system calculation unit 222 may also calculate a measurement coordinate system based on only a portion of the shape of the shelf 102. Specifically, the coordinate system calculation unit 222 calculates the measurement coordinate system based on the shape of the opening 102a of the shelf 102 as a portion of the shape used as the basis for calculating the measurement coordinate system. The shape of the opening 102a used as the basis for calculating the measurement coordinate system may be a corner of the opening 102a or a side of the opening 102a, as shown in the embodiment when the opening 102a is rectangular.

[0071] The measurement coordinate system is a three-dimensional Cartesian coordinate system, and is an example of a first three-dimensional coordinate system. By calculating the measurement coordinate system, the relative position and orientation of the distance measuring sensor 210 relative to the shelf 102 can be determined. In other words, this allows the sensor coordinate system of the distance measuring sensor 210 to be aligned with the measurement coordinate system, and calibration can be performed between the shelf 102 and the distance measuring sensor 210. The sensor coordinate system is a three-dimensional Cartesian coordinate system.

[0072] In this embodiment, the rectangular shelf 102 has an opening 102a on one side, but it is not limited to this. The shelf may have openings on multiple sides of its rectangular shape, such as the front and rear sides, or the front and top sides. If the shelf has multiple openings, a predetermined reference position, described later, may be set for one of the multiple openings. The predetermined reference position may be set as a three-dimensional point in the storage three-dimensional model, which is a three-dimensional model of the shelf 102, or in a space where no voxels exist.

[0073] Here, the coordinate system calculation unit 222 of the first example will be explained using Figures 6 and 7.

[0074] Figure 6 is a block diagram showing the configuration of the coordinate system calculation unit in the first example. Figure 7 is a diagram illustrating the method for calculating the measurement coordinate system using the coordinate system calculation unit in the first example.

[0075] The coordinate system calculation unit 222 calculates the measurement coordinate system. The measurement coordinate system is a three-dimensional coordinate system that serves as the reference for the three-dimensional spatial model. For example, the distance measuring sensor 210 is installed at the origin of the measurement coordinate system and is positioned to face the opening 102a of the shelf 102. In this case, the measurement coordinate system may be set with the upward direction of the distance measuring sensor 210 as the X-axis, the rightward direction as the Y-axis, and the forward direction as the Z-axis. The coordinate system calculation unit 222 includes an auxiliary unit 301 and a calculation unit 302.

[0076] As shown in Figure 7(a), the auxiliary unit 301 sequentially acquires images 2001, which are measurement results from the distance measuring sensor 210 acquired by the acquisition unit 221, in real time, and superimposes adjustment markers 2002 onto each sequentially acquired image 2001. The auxiliary unit 301 sequentially outputs superimposed images 2003, which are obtained by superimposing the adjustment markers 2002 onto the image 2001, to a display device (not shown). The display device sequentially displays the superimposed images 2003 output by the information processing device 220. Note that the auxiliary unit 301 and the display device may be provided integrally with the distance measuring sensor 210.

[0077] The adjustment marker 2002 is a marker that assists the user in moving the distance measuring sensor 210 so that its position and orientation relative to the shelf 102 are in a specific position and orientation. The user can position the distance measuring sensor 210 in a specific position and orientation relative to the shelf 102 by changing the position and orientation of the distance measuring sensor 210 so that the adjustment marker 2002 overlaps with a predetermined reference position on the shelf 102, while viewing the superimposed image 2003 displayed on the display device. The predetermined reference position on the shelf 102 is, for example, the position of the four corners of the rectangular opening 102a of the shelf 102.

[0078] When the distance measuring sensor 210 is positioned at a specific location and orientation relative to the shelf 102, a superimposed image 2003 is generated in which four adjustment markers 2002 are superimposed at four positions corresponding to the four corners of the opening 102a of the shelf 102. For example, the user can align the four adjustment markers 2002 with the four corners of the opening 102a, as shown in Figure 7(b), by moving the distance measuring sensor 210 so that the adjustment markers 2002 move in the direction of the arrows shown in Figure 7(a).

[0079] Although the auxiliary unit 301 is described as superimposing the adjustment marker 2002 onto the image 2001, the adjustment marker may also be superimposed onto the three-dimensional spatial model, and the three-dimensional spatial model with the adjustment marker superimposed may be displayed on the display device.

[0080] As shown in Figure 7(c), the calculation unit 302 calculates a rotation matrix 2005 and a translation vector 2006 that show the positional relationship between the distance measuring sensor 210 and the shelf 102 when the four adjustment markers 2002 are aligned to the four corners of the opening 102a. The calculation unit 302 uses the calculated rotation matrix 2005 and translation vector 2006 to transform the sensor coordinate system 2004 of the distance measuring sensor 210, thereby calculating a measurement coordinate system 2000 with an arbitrary corner (one of the four corners) of the opening 102a as the origin. When the four adjustment markers 2002 are aligned to the four corners of the opening 102a, the user may make an input to an input device (not shown). The information processing device 220 may determine when the four adjustment markers 2002 are aligned to the four corners of the opening 102a by acquiring the time when such an input is received from the input device. The information processing device 220 may also analyze the image 2001 to determine whether the four adjustment markers 2002 are aligned with the four corners of the aperture 102a.

[0081] Next, the coordinate system calculation unit 222A of the second example will be explained using Figures 8 and 9.

[0082] Figure 8 is a block diagram showing the configuration of the coordinate system calculation unit in the second example. Figure 9 is a diagram illustrating the method for calculating the measurement coordinate system using the coordinate system calculation unit in the second example.

[0083] The coordinate system calculation unit 222A includes a detection unit 311, an extraction unit 312, and a calculation unit 313.

[0084] The detection unit 311 uses the spatial three-dimensional model 2011, which is the measurement result from the distance measuring sensor 210 acquired by the acquisition unit 221 as shown in Figure 9(a), and the storage three-dimensional model 2012, shown in Figure 9(b), to detect the shelf area 2014 corresponding to the shelf 102, as shown in Figure 9(c). The storage three-dimensional model 2012 is a three-dimensional model of the shelf 102 when no luggage 103 is stored in it, and is a three-dimensional model that was previously generated using the measurement result from the distance measuring sensor 210 for the shelf 102 when no luggage 103 is stored in it. The storage three-dimensional model 2012 is generated by the model generation unit 223, which will be described later, and is stored in the storage unit 225. The storage three-dimensional model 2012 may also include position information 2013 indicating the positions of the four corners of the opening 102a of the shelf 102.

[0085] As shown in Figure 9(d), the extraction unit 312 uses positional information 2013 in the stored three-dimensional model 2012 to extract four opening endpoints 2016, which are the positions of the four corners of the opening 2015 in the shelf area 2014. The shape of the opening 2015 defined by the four opening endpoints 2016 is an example of a partial shape that serves as the basis for calculating the measurement coordinate system.

[0086] As shown in Figure 9(e), the calculation unit 313 calculates a rotation matrix 2017 and a translation vector 2018 that indicate the positional relationship between the distance measuring sensor 210 and the shelf 102, based on the shapes of the four opening endpoints 2016 as seen from the distance measuring sensor 210. The calculation unit 313 uses the rotation matrix 2017 and the translation vector 2018 to transform the sensor coordinate system 2004 of the distance measuring sensor 210 and calculates the measurement coordinate system 2000. Specifically, if the rotation matrix 2017 is R and the translation vector 2018 is T, the calculation unit 313 can transform the three-dimensional point x in the sensor coordinate system 2004 into the three-dimensional point X in the measurement coordinate system 2000 using equation 1 shown below. This allows the calculation unit 313 to calculate the measurement coordinate system 2000.

[0087] X=Rx+T...Formula 1

[0088] Next, the coordinate system calculation unit 222B of the third example will be explained using Figures 10 and 11.

[0089] Figure 10 is a block diagram showing the configuration of the coordinate system calculation unit in the third example. Figure 11 is a diagram illustrating the method for calculating the measurement coordinate system by the coordinate system calculation unit in the third example.

[0090] The coordinate system calculation unit 222B includes a detection unit 321, an extraction unit 322, and a calculation unit 323. In the third example, a marker 104 is placed at a specific position on the shelf 102 (for example, on the top surface), and the coordinate system calculation unit 222B determines the measurement coordinate system 2000 based on the position of the marker 104. In other words, the measurement coordinate system 2000 in this case is a coordinate system based on the position of the marker 104 installed on the shelf 102.

[0091] The marker 104 may have, for example, a check pattern. However, the marker 104 is not limited to a check pattern; it can be any alignment mark (positioning mark) having a predetermined shape.

[0092] The detection unit 321 detects a marker region 2024 corresponding to a marker 104 installed on the shelf 102, as shown in Figure 11(c), from the image 2021, which is the measurement result from the distance measuring sensor 210 acquired by the acquisition unit 221, as shown in Figure 11(a).

[0093] The extraction unit 322 extracts the pattern contour 2025, which is the outline of the check pattern, from the marker region 2024 on the image 2021, as shown in Figure 11(d).

[0094] The calculation unit 323 calculates a rotation matrix 2026 and a translation vector 2027 that indicate the positional relationship between the distance measuring sensor 210 and the marker 104, based on the shape of the extracted pattern contour 2025. The calculation unit 323 uses the rotation matrix 2026 and the translation vector 2027, along with the positional relationship between the stored three-dimensional model 2022 and the marker 2023 shown in Figure 11(b), to calculate the three-dimensional positional relationship between the distance measuring sensor 210 and the shelf 102, and then uses the calculated three-dimensional positional relationship to transform the sensor coordinate system 2004 to calculate the measurement coordinate system 2000. The positional relationship between the stored three-dimensional model 2022 and the marker 2023 may be measured in advance, or it may be generated in advance based on the design data of the shelf 102 on which the marker 104 is placed.

[0095] Returning to Figure 2, the model generation unit 223 will be explained.

[0096] The model generation unit 223 generates a storage 3D model, which is a 3D model of the shelf 102 that does not contain the package 103. The model generation unit 223 obtains measurement results of the shelf 102 that does not contain the package 103 using the distance measuring sensor 210 and generates the storage 3D model. The specific processing by the model generation unit 223 will be described later. The generated storage 3D model is stored in the storage unit 225.

[0097] Here, the model generation unit 223 will be explained in detail using Figures 12 and 13.

[0098] Figure 12 is a block diagram showing an example of the configuration of the model generation unit. Figure 13 is a flowchart of the process by which the model generation unit calculates the volume of the storage space.

[0099] The model generation unit 223 includes a detection unit 401, a generation unit 402, and a volume calculation unit 403.

[0100] The detection unit 401 detects the shelf area corresponding to the shelf 102 from the three-dimensional spatial model measured by the distance measuring sensor 210 (S101). If the three-dimensional measurement system 200 is equipped with multiple distance measuring sensors 210, the detection unit 401 performs the processing of step S101 for each of the multiple distance measuring sensors 210. As a result, the detection unit 401 detects multiple shelf areas corresponding to each of the multiple distance measuring sensors 210.

[0101] The generation unit 402 integrates multiple shelf areas and generates a storage three-dimensional model (S102) when the three-dimensional measurement system 200 is equipped with multiple distance measuring sensors 210. Specifically, the generation unit 402 uses ICP (Iterative Closest Point) to integrate multiple shelf areas. Alternatively, three-dimensional point cloud alignment may be performed, or the relative positional relationship of multiple distance measuring sensors 210 may be calculated in advance, and multiple shelf areas may be integrated based on the calculated relative positional relationship. The relative positional relationship may be calculated using SfM (Structure from Motion) by combining multiple images acquired by each of the multiple distance measuring sensors 210 into a multi-view image. The multiple distance measuring sensors 210 may be installed based on a design drawing in which the relative positional relationship is determined.

[0102] Alternatively, instead of using multiple distance measuring sensors 210, a single distance measuring sensor 210 may be moved, and multiple measurement results obtained from multiple positions may be used to generate a three-dimensional storage model of the shelf 102 by integrating the multiple shelf areas obtained from each of the multiple measurement results.

[0103] The storage 3D model may be generated based on the 3D CAD data used during the design of the shelf 102, or based on the dimensional measurement data of the shelf 102 or the equipment specification data published by the manufacturer, without using the results measured by the distance measuring sensor 210.

[0104] Furthermore, if the three-dimensional measurement system 200 does not have multiple distance measuring sensors 210 but only one distance measuring sensor 210 and uses one measurement result measured from one position, the model generation unit 223 does not need to have the generation unit 402. In other words, the model generation unit 223 does not need to perform step S102.

[0105] The volume calculation unit 403 calculates the volume of the storage space 101 of the shelf 102 using the three-dimensional storage model (S103).

[0106] Returning to Figure 2, let's explain the filling rate calculation unit 224.

[0107] The filling rate calculation unit 224 calculates the filling rate of the luggage 103 relative to the storage space 101 of the shelf 102. The filling rate calculation unit 224 may, for example, use a spatial three-dimensional model, image, and measurement coordinate system 2000 acquired by the distance measuring sensor 210 to calculate the ratio of the volume of the luggage 103 to the volume of the storage space 101 as the filling rate.

[0108] Here, the filling rate calculation unit 224 will be explained in detail using Figures 14 and 15.

[0109] Figure 14 is a block diagram showing an example of the configuration of the filling rate calculation unit. Figure 15 is a diagram illustrating an example of how the filling rate is calculated by the filling rate calculation unit. Figure 15 shows an example where the distance measuring sensor 210 is directly facing the opening 102a of the shelf 102. The distance measuring sensor 210 is positioned on the Z-axis minus side where the opening 102a of the shelf 102 is formed, and measures the storage space 101 of the shelf 102 through the opening 102a of the shelf 102. In other words, the distance measuring sensor 210 is positioned on the upper side of the shelf 102 in the vertical direction. This example is an example where the measurement coordinate system 2000 is measured by the coordinate system calculation unit 222 of the first example. In other words, in this case, the sensor coordinate system 2004 and the measurement coordinate system 2000 coincide.

[0110] The filling rate calculation unit 224 includes an extraction unit 501, an estimation unit 502, and a calculation unit 503.

[0111] The extraction unit 501 uses the spatial three-dimensional model 2011 and the stored three-dimensional model to extract the luggage region 2033, which is the portion of the spatial three-dimensional model corresponding to the luggage 103. Specifically, the extraction unit 501 converts the data structure of the spatial three-dimensional model 2011, which is the measurement result from the distance measuring sensor 210 acquired by the acquisition unit 221 as shown in Figure 15(a), into voxel data, thereby generating the voxel data 2031 shown in Figure 15(b). Using the generated voxel data 2031 and the stored three-dimensional model 2032, which is the voxelized stored three-dimensional model as shown in Figure 15(c), the extraction unit 501 subtracts the stored three-dimensional model 2032 from the voxel data 2031 to extract the luggage region 2033, which is the region of the voxel data 2031 where the luggage 103 was measured, as shown in Figure 15(d). The luggage area 2033 is an example of an object area, which corresponds to the object being measured.

[0112] The estimation unit 502 uses the extracted luggage region 2033 to estimate a luggage model 2034, which is a three-dimensional model of the luggage 103 within the storage space 101. Specifically, the estimation unit 502 uses the luggage region 2033 to interpolate the luggage region 2033 to the area where the luggage 103 is hidden from the distance sensor 210 in the Z-axis direction, which is the direction in which the distance sensor 210 and the shelf 102 are aligned, i.e., to the Z-positive direction. For example, the estimation unit 502 determines for each of the multiple voxels constituting the luggage region 2033 whether the voxel is located in the Z-negative direction more than the furthest voxel that is located furthest in the Z-positive direction among the multiple voxels. If the voxel is located in the Z-negative direction more than the furthest voxel, the estimation unit 502 interpolates the voxel to a position in the same Z-axis direction as the furthest voxel if there are no other voxels located further in the Z-positive direction than that voxel. As a result, the estimation unit 502 estimates a luggage model 2034 as shown in Figure 15(e).

[0113] The calculation unit 503 uses the storage three-dimensional model and the luggage model 2034 to calculate the first filling rate of luggage 103 in the storage space 101. Specifically, the calculation unit 503 counts the number of voxels that make up the luggage model 2034 and calculates the volume of luggage 103 by multiplying the counted number by a predetermined voxel size. The calculation unit 503 calculates the ratio of the calculated volume of luggage 103 to the volume of the storage space 101 of the shelf 102 calculated by the model generation unit 223 as the first filling rate.

[0114] The distance measuring sensor 210 does not need to be directly facing the opening 102a of the shelf 102. Figure 16 is a diagram illustrating another example of the method for calculating the filling rate by the filling rate calculation unit. Figure 16 shows an example where the distance measuring sensor 210 is positioned at an angle to the opening 102a of the shelf 102. This example is one in which the measurement coordinate system 2000 is measured by the coordinate system calculation unit 222A of the second example or the coordinate system calculation unit 222B of the third example. In other words, in this case, the sensor coordinate system 2004 and the measurement coordinate system 2000 are different.

[0115] In the example shown in Figure 16, the coordinate system used is the measurement coordinate system 2000. The estimation unit 502 uses the luggage region 2033 to interpolate the luggage region 2033 in the Z-axis direction of the measurement coordinate system 2000, which is the direction in which the distance sensor 210 and the shelf 102 are aligned, in the region where the luggage 103 is hidden from the distance sensor 210, that is, in the positive Z-axis direction.

[0116] Other processing performed by the filling rate calculation unit 224 is the same as in the case of Figure 15, so its explanation is omitted.

[0117] The pairs of spatial three-dimensional models and images used in the calculation of the measurement coordinate system by the coordinate system calculation unit 222 and the calculation of the filling rate by the filling rate calculation unit 224 may be results measured at the same time by the distance measuring sensor 210, or they may be results measured at different times.

[0118] The distance measuring sensor 210 and the information processing device 220 may be connected to each other via a communication network. The communication network may be a public communication network such as the Internet, or a dedicated communication network. As a result, the spatial three-dimensional model and image obtained by the distance measuring sensor 210 are transmitted from the distance measuring sensor 210 to the information processing device 220 via the communication network.

[0119] Furthermore, the information processing device 220 may acquire the spatial three-dimensional model and images from the distance measuring sensor 210 without using a communication network. For example, the spatial three-dimensional model and images may be temporarily stored in an external storage device such as a hard disk drive (HDD) or solid-state drive (SSD) from the distance measuring sensor 210, and the information processing device 220 may acquire the spatial three-dimensional model and images from the external storage device. The external storage device may also be a cloud server.

[0120] The information processing device 220 includes, for example, a computer system comprising a control program, a processing circuit such as a processor or logic circuit that executes the control program, and a recording device such as an internal memory or an accessible external memory that stores the control program. The functions of each processing unit of the information processing device 220 may be implemented by software or by hardware.

[0121] Next, the operation of the information processing device 220 will be described.

[0122] Figure 17 is a flowchart of the filling rate measurement method performed by the information processing device.

[0123] The information processing device 220 acquires a three-dimensional spatial model from the distance measuring sensor 210 (S111). At this time, the information processing device 220 may also acquire an image of the object to be measured from the distance measuring sensor 210.

[0124] The information processing device 220 retrieves the stored three-dimensional model stored in the memory unit 225 (S112).

[0125] The information processing device 220 calculates a measurement coordinate system based on the shape of the opening 102a of the shelf 102 (S113). Step S113 is a process performed by the coordinate system calculation unit 222.

[0126] The information processing device 220 uses the voxel data 2031 of the spatial three-dimensional model 2011 and the stored three-dimensional model 2032 of the stored three-dimensional model to extract the luggage area 2033 corresponding to luggage 103 from the voxel data 2031 (S114). Step S114 is a process performed by the extraction unit 501 of the filling rate calculation unit 224.

[0127] The information processing device 220 estimates a luggage model 2034, which is a three-dimensional model of luggage 103 within the storage space 101, using the extracted luggage area 2033 (S115). Step S115 is a process performed by the estimation unit 502 of the filling rate calculation unit 224.

[0128] The information processing device 220 uses the storage three-dimensional model and the luggage model 2034 to calculate the first filling rate of luggage 103 in the storage space 101 (S116). Step S116 is the processing performed by the calculation unit 503 of the filling rate calculation unit 224.

[0129] Figure 18 is a flowchart of the process (S113) for calculating the measurement coordinate system by the coordinate system calculation unit in the first example.

[0130] The coordinate system calculation unit 222 sequentially acquires images 2001, which are measurement results from the distance measuring sensor 210 acquired by the acquisition unit 221, in real time, and superimposes an adjustment marker 2002 on each sequentially acquired image 2001 (S121). Step S121 is a process performed by the auxiliary unit 301 of the coordinate system calculation unit 222.

[0131] The coordinate system calculation unit 222 acquires the position and orientation of the distance measuring sensor 210 (S122). Step S121 is a process performed by the auxiliary unit 301 of the coordinate system calculation unit 222.

[0132] The coordinate system calculation unit 222 determines the sensor coordinate system 2004 of the distance measuring sensor 210 using the position and orientation of the distance measuring sensor 210 when the four adjustment markers 2002 are aligned with the four corners of the aperture 102a, and calculates the measurement coordinate system 2000 using the determined sensor coordinate system 2004 (S123). Step S123 is a process performed by the calculation unit 302 of the coordinate system calculation unit 222.

[0133] Figure 19 is a flowchart of the process (S113) for calculating the measurement coordinate system by the coordinate system calculation unit in the second example.

[0134] The coordinate system calculation unit 222A uses the spatial three-dimensional model 2011, which is the measurement result from the distance measuring sensor 210 acquired by the acquisition unit 221, and the storage three-dimensional model 2012 to detect the shelf area 2014 corresponding to the shelf 102 (S121A). Step S121A is a process performed by the detection unit 311 of the coordinate system calculation unit 222A.

[0135] The coordinate system calculation unit 222A uses position information 2013 in the stored three-dimensional model 2012 to extract the four opening endpoints 2016, which are the positions of the four corners of the opening 2015 in the shelf area 2014 (S122A). Step S122A is a process performed by the extraction unit 312 of the coordinate system calculation unit 222A.

[0136] The coordinate system calculation unit 222A calculates a rotation matrix 2017 and a translation vector 2018 that show the positional relationship between the distance measuring sensor 210 and the shelf 102, based on the shapes of the four aperture endpoints 2016 as seen from the distance measuring sensor 210. Then, the coordinate system calculation unit 222A calculates the measurement coordinate system 2000 by transforming the sensor coordinate system 2004 of the distance measuring sensor 210 using the rotation matrix 2017 and the translation vector 2018 (S123A). Step S123A is a process performed by the calculation unit 313 of the coordinate system calculation unit 222A.

[0137] Figure 20 is a flowchart of the process (S113) for calculating the measurement coordinate system by the coordinate system calculation unit in the third example.

[0138] The coordinate system calculation unit 222B detects the marker region 2024 from the image 2021, which is the measurement result from the distance measuring sensor 210 acquired by the acquisition unit 221 (S121B). Step S121B is a process performed by the detection unit 321 of the coordinate system calculation unit 222B.

[0139] The coordinate system calculation unit 222B extracts the pattern contour 2025 from the marker region 2024 on the image 2021 (S122B). Step S122B is a process performed by the extraction unit 322 of the coordinate system calculation unit 222B.

[0140] The coordinate system calculation unit 222B calculates a rotation matrix 2026 and a translation vector 2027 that indicate the positional relationship between the distance measuring sensor 210 and the marker 104, based on the shape of the extracted pattern contour 2025. Then, the coordinate system calculation unit 222B uses the rotation matrix 2026 and translation vector 2027, along with the positional relationship between the stored three-dimensional model 2022 and the marker 2023, to calculate the three-dimensional positional relationship between the distance measuring sensor 210 and the shelf 102. By transforming the sensor coordinate system 2004 using the calculated three-dimensional positional relationship, the measurement coordinate system 2000 is calculated (S123B). Step S123B is a process performed by the calculation unit 323 of the coordinate system calculation unit 222B.

[0141] The filling rate calculated by the information processing device 220 may be output from the information processing device 220. The filling rate may be displayed by a display device (not shown) provided by the information processing device 220, or it may be transmitted to an external device different from the information processing device 220. For example, the calculated filling rate may be output to a cargo handling system and used for controlling the cargo handling system.

[0142] According to the filling rate measurement method of this embodiment, the luggage model 2034 of luggage 103 is estimated using the luggage region 2033 extracted using a three-dimensional spatial model of shelf 102 with luggage 103 stored in it and a three-dimensional storage model of shelf 102 without luggage 103 stored in it. As a result, the first filling rate of luggage 103 in the storage space 101 can be easily calculated simply by measuring shelf 102 with luggage 103 stored in it.

[0143] Furthermore, in the filling rate measurement method, the luggage model 2034 is estimated based on a three-dimensional coordinate system that uses the shape of a part of the shelf 102 as a reference. This reduces the processing load required to estimate the luggage model 2034.

[0144] Furthermore, in the filling rate measurement method, the luggage model 2034 is estimated based on a three-dimensional coordinate system that uses only a portion of the shape of the shelf 102 as a reference. The shape of only a portion of the first storage area, which is easy to extract from the image, can be used to calculate the measurement coordinate system. Therefore, the processing speed of luggage model estimation can be improved, and the accuracy of the measurement coordinate system can be improved.

[0145] Furthermore, in the filling rate measurement method, the three-dimensional coordinate system is a three-dimensional orthogonal coordinate system with a Z axis. In the estimation, the luggage model 2034 is estimated by interpolating the Z-axis positive direction side, which is opposite to the Z-axis negative direction of the luggage area 2033. Therefore, the processing load for estimating the luggage model 2034 can be effectively reduced.

[0146] Furthermore, in the filling rate measurement method, the three-dimensional coordinate system is a coordinate system based on the shape of the opening 102a of the shelf 102. Therefore, a coordinate system based on the shape of the shelf 102 opening 102a can be easily calculated, and the cargo model 2034 can be estimated based on the calculated coordinate system.

[0147] Furthermore, in the filling rate measurement method, the three-dimensional coordinate system is a coordinate system based on the marker 104 installed on the shelf 102. Therefore, the coordinate system based on the marker 104 can be easily calculated, and the luggage model 2034 can be estimated based on the calculated coordinate system.

[0148] Furthermore, in the filling rate measurement method, the distance measuring sensor 210B has at least two cameras for generating a three-dimensional spatial model. The distance measuring sensor 210, including the distance measuring sensor 210B, is fixed on the upper side of the first storage unit.

[0149] Thus, when the distance measuring sensor 210 is fixed to the upper side of the first storage unit, and the first storage unit is movable, such as a trolley described later, the objects within the measurement range of the distance measuring sensor 210 are limited to the ground or the base (bottom surface) of the trolley, and there are no other movable objects besides the trolley, making it easy to isolate the object to be measured from the background based on the measurement results. Note that the measurement range is the shooting range of the camera if the distance measuring sensor 210 has a camera. On the other hand, when the distance measuring sensor 210 is fixed to a side other than the upper side, other moving objects besides the first storage unit are more likely to enter the measurement range, making it difficult to isolate the object to be measured from the background.

[0150] (Variation 1) In the information processing device 220 according to the above embodiment, the ratio of the volume of the luggage 103 stored in the storage space 101 to the volume of the storage space 101 is calculated as the filling rate, but the device is not limited to this.

[0151] Figure 21 is a diagram illustrating the method for calculating the filling rate.

[0152] In Figures 21(a) and (b), the storage space 101 of the shelf 102 has a volume that is just enough to store 16 packages 103. As shown in Figure 21(a), when 8 packages 103 are placed without any gaps, 8 more packages 103 can be stored in the remaining storage space 101. On the other hand, as shown in Figure 21(b), when packages are placed with gaps, in order to store 8 more packages 103 in the remaining space of the storage space 101, it is necessary to move the packages 103 that are already stored. If packages 103 are stored in the remaining space of the storage space 101 without moving the packages 103 that are already stored, only 6 packages 103 can be stored.

[0153] Thus, although the amount of luggage 103 that can be stored in the remaining space of the storage space 101 is different in the case of Figure 21(a) and the case of Figure 21(b), the filling rate is calculated to be the same 50% in both cases. For this reason, it is conceivable to calculate the filling rate that takes into account the space that can actually be stored, according to the shape of the remaining space of the storage space 101.

[0154] Figure 22 is a block diagram showing an example of the configuration of the calculation unit of the filling rate calculation unit according to Modified Example 1. Figure 23 is a flowchart of the filling rate calculation process of the calculation unit of the filling rate calculation unit according to Modified Example 1.

[0155] As shown in Figure 22, the calculation unit 503 includes a luggage volume calculation unit 601, a region division unit 602, a planned luggage measurement unit 603, a region estimation unit 604, and a calculation unit 605.

[0156] The luggage volume calculation unit 601 calculates the luggage volume, which is the volume of luggage 103, from the luggage model 2034 (S131). The luggage volume calculation unit 601 calculates the volume of luggage 103 stored in the storage space 101 in the same manner as in the embodiment.

[0157] Next, the region division unit 602 divides the storage space 101 of the three-dimensional spatial model 2011 into an occupied region 2041 occupied by the luggage 103 and an empty region 2042 not occupied by the luggage 103 (S132).

[0158] Next, the planned luggage measurement unit 603 calculates the volume of one piece of luggage to be stored (S133). If there are multiple types of luggage with different shapes and sizes as shown in Figure 21(c), the planned luggage measurement unit 603 calculates the volume of one piece of luggage for each type. For example, the planned luggage measurement unit 603 calculates the volume of luggage 103a, the volume of luggage 103b, and the volume of luggage 103c, respectively.

[0159] Next, the area estimation unit 604 estimates the arrangement that allows the most packages 103 to be stored to be placed in the empty area 2042, and estimates the number of packages 103 to be stored in that arrangement. In other words, the area estimation unit 604 estimates the maximum number of packages 103 that can be stored in the empty area 2042. The area estimation unit 604 calculates the available storage volume in the empty area 2042 by multiplying the volume of one package by the number of packages that can be stored (S134).

[0160] The area estimation unit 604 may estimate how many items of each type can be stored if there are multiple types of items, or it may estimate how many items of each type can be stored in a mixed manner. When storing items of multiple types in a mixed manner, the area estimation unit 604 calculates the storable volume of the empty area 2042 as the sum of the volumes obtained by multiplying the volume of one item of each type by the number of items of that type that can be stored. For example, if the area estimation unit 604 estimates that n1 items of item 103a, n2 items of item 103b, and n3 items of item 103c can be stored, it calculates the storable volume of the empty area 2042 as the sum of the first volume obtained by multiplying the volume of item 103a by n1, the second volume obtained by multiplying the volume of item 103b by n2, and the third volume obtained by multiplying the volume of item 103c by n3. Note that n1, n2, and n3 are all integers of 0 or more.

[0161] The calculation unit 605 calculates the filling rate by applying the volume of stored luggage and the available storage volume to the following formula 2 (S135).

[0162] Filling rate (%) = (Volume of stored goods) / (Volume of stored goods + Available storage volume) × 100 ... Equation 2

[0163] Thus, the filling rate calculation unit 224 may calculate the filling rate as the ratio of the volume of the luggage 103 stored in the storage space 101 to the volume of the space in the storage space 101 that can store the luggage 103.

[0164] This makes it possible to calculate a first filling rate that allows for an appropriate determination of how much luggage 103 can be stored in the available space of the storage space 101.

[0165] (Modification 2) In the information processing device 220 according to the above embodiment, the filling rate of goods 103 for the storage space 101 of one shelf 102 is calculated, but the filling rate of goods 103 for the storage space 101 of two or more shelves 102 may also be calculated.

[0166] Figure 24 shows an example of storing two or more shelves in a storage space such as the back of a truck. Figure 25 is a table showing the relationship between the shelves stored in the storage space of the truck bed and their occupancy rate.

[0167] As shown in Figure 24, a cargo bed 106 having a storage space 105 stores multiple cage trolleys 112. The cargo bed 106 may be, for example, a van-type cargo bed of a truck. The cargo bed 106 is an example of a second storage unit. The second storage unit is not limited to the cargo bed 106, but may be a container or a warehouse.

[0168] Storage space 105 is an example of a second storage space. Storage space 105 has a volume large enough to store multiple cage trolleys 112. In modified example 2, storage space 105 can store six cage trolleys 112. Since storage space 105 can store multiple cage trolleys 112, storage space 105 is larger than storage space 111.

[0169] The cage trolley 112 has a storage space 111 capable of storing multiple packages 103. The cage trolley 112 is an example of a first storage section. The storage space 111 is an example of a first storage space. The storage space 105 may also store the shelf 102 described in the embodiment.

[0170] The multiple packages 103 are not stored directly on the loading platform 106, but are stored on multiple cage trolleys 112. Then, the cage trolleys 112 containing the multiple packages 103 are stored on the loading platform 106.

[0171] In this case, the configuration of the calculation unit 503 of the filling rate calculation unit 224 will be described.

[0172] Figure 26 is a block diagram showing an example of the configuration of the calculation unit of the filling rate calculation unit according to Modified Example 2. Figure 27 is a flowchart of the filling rate calculation process of the calculation unit of the filling rate calculation unit according to Modified Example 2.

[0173] As shown in Figure 26, the calculation unit 503 according to the modified example 2 has an acquisition unit 701, a counting unit 702, and a calculation unit 703.

[0174] The acquisition unit 701 acquires the number of cage trolleys 112 that can be stored in the loading platform 106 (S141). In the case of the modified example 2, the maximum number of cage trolleys 112 that can be stored in the loading platform 106 is 6, so 6 is acquired.

[0175] The counting unit 702 counts the number of cage trolleys 112 to be stored in the loading platform 106 (S142). When the cage trolleys 112 shown in Figure 24 are stored in the loading platform 106, the counting unit 702 counts 3 cage trolleys 112.

[0176] The calculation unit 703 calculates a second filling rate, which is the filling rate of one or more cage trolleys 112 relative to the loading platform 106 (S143). Specifically, the calculation unit 703 may calculate the second filling rate as the ratio of the number of cage trolleys 112 stored in the loading platform 106 to the maximum number of cage trolleys 112 that can be stored in the loading platform 106. For example, the calculation unit 703 calculates 50% as the second filling rate because a maximum of 6 cage trolleys 112 can be stored in the loading platform 106, and 3 of them are stored in the loading platform 106.

[0177] The calculation unit 703 may calculate the loading rate of the cargo 103 for each of the one or more cage trolleys 112 stored in the loading platform 106, and use the calculated loading rate to calculate the loading rate of the cargo 103 for the second storage space. Specifically, the calculation unit 703 may calculate the average loading rate of the cargo 103 for the cage trolleys 112 as the loading rate of the cargo 103 for the second storage space. In this case, if there is remaining space in the storage space 105 of the loading platform 106 that can accommodate cage trolleys 112, the calculation unit 703 may calculate the average by setting the loading rate of the number of cage trolleys 112 that can be stored in the remaining space to 0%.

[0178] For example, if the three cage trolleys 112 shown in Figure 25 have filling rates of 70%, 30%, and 20%, and the cargo bed 106 can accommodate a maximum of six cage trolleys 112, then the 20% obtained by averaging the filling rates of the six cage trolleys 112 (70%, 30%, 20%, 0%, 0%, 0%) may be used as the filling rate of the cargo 103 for the second storage space.

[0179] Therefore, the second filling rate can be appropriately calculated when one or more cage trolleys 112 are stored in the storage space 105.

[0180] (Variation 3) Next, I will explain the third variation.

[0181] Figure 28 is a diagram illustrating the configuration of the cage trolley according to Modification 3.

[0182] Figure 28(a) shows the car carriage 112 with the opening / closing section 113 in the closed position. Figure 28(b) shows the car carriage 112 with the opening / closing section 113 in the open position.

[0183] The cage trolley 112 according to Modification 3 has an opening / closing section 113 that opens and closes the opening 112a. The opening / closing section 113 is a grid-like or mesh-like cover having a plurality of through holes 113a. Therefore, even when the opening / closing section 113 of the cage trolley 112 is in the closed state, the distance measuring sensor 210 can measure the three-dimensional shape inside the storage space 111 of the cage trolley 112 through the plurality of through holes 113a and the opening 112a.

[0184] This is because the electromagnetic waves emitted by the distance measuring sensor 210 pass through multiple through holes 113a and openings 112a. In the case of the distance measuring sensor 210A as well, the infrared pattern emitted by the distance measuring sensor 210A passes through multiple through holes 113a and openings 112a, so even when the opening / closing part 113 of the cage trolley 112 is in the closed state, the three-dimensional shape inside the storage space 111 of the cage trolley 112 can be measured through the multiple through holes 113a and openings 112a. Similarly, in the case of the distance measuring sensor 210B, the two cameras 211B and 212B can photograph the inside of the storage space 111 through the multiple through holes 113a and openings 112a, so the three-dimensional shape inside the storage space 111 of the cage trolley 112 can be measured.

[0185] Therefore, the information processing device 220 can determine whether or not luggage 103 is stored in the storage space 111. However, it is difficult to determine the correct filling rate unless the method for calculating the filling rate is switched to a different method depending on whether the opening / closing part 113 is closed, open, or not present. For this reason, the filling rate calculation unit 224 according to the modified example 3 calculates the filling rate using the first method when the opening / closing part 113 is open, and calculates the filling rate using the second method when the opening / closing part 113 is closed.

[0186] Figure 29 is a block diagram showing an example of the configuration of the filling rate calculation unit according to Modification Example 3. Figure 30 is a flowchart of the filling rate calculation process of the filling rate calculation unit according to Modification Example 3.

[0187] As shown in Figure 29, the filling rate calculation unit 224 according to the modified example 3 includes a detection unit 801, a switching unit 802, a first filling rate calculation unit 803, and a second filling rate calculation unit 804.

[0188] The detection unit 801 detects the open / closed state of the opening / closing section 113 using a three-dimensional spatial model (S151). Specifically, the detection unit 801 uses a three-dimensional spatial model to detect that the opening / closing section 113 is in the closed state if three-dimensional point clouds exist in the front-to-back direction of the opening 112a region of the cage trolley 112 (i.e., in the direction of alignment between the distance measuring sensor 210 and the cage trolley 112) both inside and outside the storage space 111. The detection unit 801 detects that the opening / closing section 113 is in the open state if three-dimensional point clouds exist only inside the storage space 111.

[0189] The switching unit 802 determines whether the opening / closing unit 113 is in the open or closed state (S152), and switches the next process according to the determination result.

[0190] If the switching unit 802 determines that the opening / closing unit 113 is in the open state (open state in S152), the first filling rate calculation unit 803 calculates the filling rate using the first method (S153). Specifically, the first filling rate calculation unit 803 calculates the filling rate of the cage trolley 112 by performing the same processing as the filling rate calculation unit 224 in the embodiment.

[0191] The second filling rate calculation unit 804 calculates the filling rate using the second method (S154) when the switching unit 802 determines that the opening / closing unit 113 is in the closed state (closed state in S152). Details of the second method will be explained with reference to Figure 31.

[0192] Figure 31 illustrates an example of a second method for calculating the filling rate.

[0193] Let's consider the case where a three-dimensional spatial model 2051 is obtained, as shown in Figure 31(a).

[0194] Figure 31(b) is an enlarged view of region R2 in the three-dimensional spatial model 2051. As shown in Figure 31(b), the second filling rate calculation unit 804 divides region R2 into a second part where the opening / closing part 113 is detected and a first part where the luggage 103 is detected.

[0195] The first part is the region containing the three-dimensional point cloud on the far side of the opening 112a. The first part is also the portion where the distance measuring sensor 210 faces the luggage 103 in the direction from the distance measuring sensor 210 toward the luggage 103. In other words, the first part is the portion facing the through hole 113a in the closed opening / closing section 113 in the direction from the distance measuring sensor 210 toward the luggage 103. The opening / closing section 113 may have a configuration with one through hole 113a. The direction from the distance measuring sensor 210 toward the luggage 103 may, for example, be aligned horizontally.

[0196] The second part is the region containing the three-dimensional point cloud on the front side in the front-rear direction of the opening 112a of the cage trolley 112. The second part is also the portion in the direction from the distance measuring sensor 210 toward the luggage 103 in which the distance measuring sensor 210 does not face the luggage 103. In other words, the second part is the portion hidden by the closed opening / closing section 113 in the direction from the distance measuring sensor 210 toward the luggage 103.

[0197] The second filling rate calculation unit 804 generates the voxel data 2052 shown in Figure 31(c) by voxing the first part and the second part, respectively. In the voxel data 2052, the white areas without hatching are the areas where the second part has been voxed, and the areas with dot hatching are the areas where the first part has been voxed.

[0198] The second filling rate calculation unit 804 then estimates whether or not the package 103 exists behind the opening / closing section 113 in the white area corresponding to the area of ​​the opening / closing section 113. Specifically, the second filling rate calculation unit 804 assigns a score based on the probability of the package being present to the 26 voxels adjacent to the dot hatching voxel where the package 103 exists in the voxelized area. Then, as shown in Figure 31(d), it assigns the added score to the voxels shown in the white area adjacent to the multiple voxels where the package 103 exists. The second filling rate calculation unit 804 performs this for all voxels where the package 103 exists, and determines that the package 103 exists in the voxels shown in the white area if the sum of the scores is equal to or greater than an arbitrary threshold. The second filling rate calculation unit 804, for example, if an arbitrary threshold is set to 0.1, determines that packages 103 are present in all areas, and can calculate a package model 2053 in which the shape of the area hidden by the opening / closing unit 113 is estimated, as shown in Figure 31(e).

[0199] In this way, the information processing device 220 estimates the shape of the second part that the distance measuring sensor does not face the object to be measured, based on the shape of the first part that the distance measuring sensor faces the luggage 103. Therefore, even if there is a second part, the three-dimensional model of the object can be appropriately estimated.

[0200] Furthermore, if it is a rule that the luggage 103 be placed without any gaps inside the cage trolley 112, the second filling rate calculation unit 804 may extract the contour R3 of the region where one or more luggage 103 are placed, as shown in Figure 32, and determine that the region inside the extracted contour R3 is the region where luggage 103 exists. The second filling rate calculation unit 804 may then estimate the region of the opening / closing section 113 inside the contour R3 using a three-dimensional point cloud of the region of the multiple through holes 113a of the opening / closing section 113.

[0201] In the filling rate measurement method according to Modification 3, the cage trolley 112 further has a plurality of through holes 113a and an opening / closing part 113 that opens and closes the opening 112a. The filling rate measurement method further determines whether the opening / closing part 113 is in an open state or a closed state, and if the opening / closing part 113 is in an open state, the cargo model 2034 is estimated by extraction and estimation, similar to the filling rate calculation unit 224 of the embodiment. If the opening / closing part 113 is in a closed state, the filling rate calculation unit 224 estimates the second part hidden by the opening / closing part 113 based on a plurality of first parts corresponding to the plurality of through holes 113a of the opening / closing part 113 from the voxel data 2031 based on the spatial three-dimensional model 2011, and estimates the cargo model 2034 using the plurality of first parts, the estimated second part, and the storage three-dimensional model 2032.

[0202] According to this, even when storing luggage 103 in a cage trolley 112 equipped with an opening / closing section 113 that opens and closes an opening 112a, the method for estimating the luggage model 2034 is switched between the first method and the second method depending on the open / closed state of the opening / closing section 113, thereby enabling the appropriate estimation of the three-dimensional model of the object.

[0203] Furthermore, in the filling rate measurement method according to Modification 3, the direction from the distance measuring sensor 210 toward the cargo 103 is, for example, aligned horizontally. Therefore, there is no need to adjust the position of the distance measuring sensor 210 so that measurements can be taken from a direction without the opening / closing section 113 having the through hole 113a, thus providing greater flexibility in the installation of the distance measuring sensor 210. Thus, even if the position of the distance measuring sensor 210 has not been fully adjusted, measurement results for estimating a three-dimensional model of the object using the distance measuring sensor 210 can be obtained.

[0204] (Modification 4) Figure 33 is a diagram illustrating the method for generating a three-dimensional spatial model related to Modification 4.

[0205] As shown in Figure 33, even when generating a three-dimensional spatial model, the three-dimensional measurement system 200 may integrate the measurement results of multiple distance measuring sensors 210, similar to the processing of the model generation unit 223. In this case, the three-dimensional measurement system 200 can identify the positions and orientations of the multiple distance measuring sensors 210 by pre-calibrating them, and then generate a three-dimensional spatial model including a three-dimensional point cloud with low occlusion by integrating the obtained measurement results based on the identified positions and orientations of the multiple distance measuring sensors 210.

[0206] (Variation 5) Figure 34 is a diagram illustrating the method for generating a three-dimensional spatial model related to Modification 5.

[0207] As shown in Figure 34, even when generating a three-dimensional spatial model, the three-dimensional measurement system 200 may move the cart 112 and at least one of the distance measuring sensors 210 so as to cross the measurement area R1 of one distance measuring sensor 210, and integrate multiple measurement results obtained by the distance measuring sensor 210 at multiple timings during the movement. In this case, the relative position and orientation between the cart 112 and the distance measuring sensor 210 can be calculated, and by integrating the multiple measurement results using the relative position and orientation, a three-dimensional spatial model including a three-dimensional point cloud with low occlusion can be generated.

[0208] (others) The methods for measuring the filling rate related to this disclosure have been described above based on the embodiments described above, but this disclosure is not limited to the embodiments described above.

[0209] For example, in the above embodiment, it was explained that each processing unit of the information processing device, etc., is realized by a CPU and a control program. For example, the components of the processing unit may each consist of one or more electronic circuits. Each of the one or more electronic circuits may be a general-purpose circuit or a dedicated circuit. The one or more electronic circuits may include, for example, a semiconductor device, an IC (Integrated Circuit), or an LSI (Large Scale Integration). The IC or LSI may be integrated on a single chip or on multiple chips. Here, we refer to them as IC or LSI, but the name may change depending on the degree of integration, and they may be called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Also, FPGAs (Field Programmable Gate Arrays) that are programmed after the LSI is manufactured can be used for the same purpose.

[0210] Furthermore, the general or specific embodiments of this disclosure may be implemented as a system, apparatus, method, integrated circuit, or computer program. Alternatively, they may be implemented as a computer-readable non-temporary recording medium such as an optical disk, HDD (Hard Disk Drive), or semiconductor memory on which the computer program is stored. They may also be implemented as any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

[0211] Furthermore, this disclosure also includes forms that can be obtained by applying various modifications to each embodiment that a person skilled in the art can conceive, as well as forms that can be realized by arbitrarily combining the components and functions of the embodiments without departing from the spirit of this disclosure. [Industrial applicability]

[0212] This disclosure is useful as a filling rate measurement method, information processing device, program, etc., that can calculate the filling rate of an object to be measured. [Explanation of Symbols]

[0213] 101, 105, 111 storage space 102 shelves 102a, 112a opening 103, 103a~103c Luggage 104 markers 106 Cargo bed 112 cage trolley 113 Opening / Closing Section 113a Through hole 200 Three-Dimensional Measurement Systems 210, 210A, 210B Distance Measuring Sensors 211 Laser irradiation section 211A Infrared pattern irradiation unit 211B, 212B Camera 212 Laser light receiving section 212A Infrared Camera 220 Information Processing Devices 221, 701 Acquisition Department 222, 222A, 222B Coordinate system calculation section 223 Model Generation Unit 224 Filling rate calculation section 225 Storage section 301 Auxiliary section 302, 313, 323, 503, 605, 703 Calculation Unit 311, 321, 401 Detection unit 312, 322, 501 Extraction part 402 Generator 403 Volume Calculation Unit 502 Estimation Department 601 Cargo Volume Calculation Unit 602 Area division part 603 Scheduled Cargo Measurement Department 604 Area estimation part 702 Counting Department 801 Detection Unit 802 Switching section 803 1st filling rate calculation section 804 2nd filling rate calculation section 2000 Measurement Coordinate System Images from 2001 and 2021 2002 Adjustment Marker 2003 Superimposed image 2004 Sensor Coordinate System 2011, 2051 Spatial Three-Dimensional Model 2012, 2022, 2032 Stored 3D Models 2013 Location information 2014 Shelf area 2015 Aperture 2016 Opening end point 2017, 2026 rotation matrices 2018, 2027 Translation Vectors 2023 Marker 2024 Marker Area 2025 Pattern Contour 2031, 2052 Voxel Data 2033 Luggage Area 2034, 2053 Luggage Models 2041 Occupied area 2042 empty area P1 one point R1 Measurement area R2 area R3 contour

Claims

1. For each of the one or more first storage units stored in the second storage space of the second storage unit, which has a second storage space capable of storing multiple first storage units, a first filling rate of the object to be measured in the first storage space is obtained. The number of the first storage units that can be additionally stored in the second storage space is obtained, Based on the acquired first filling rate and the number of additionally storable first storage units, the second filling rate of the object to be measured in the second storage space is calculated. method.

2. The second filling rate is calculated by treating the additionally storable first storage unit as the first storage unit with a filling rate of 0%. The method according to claim 1.

3. The first storage section has an opening / closing section that is positioned to cover the opening, In calculating the first filling rate, if the opening / closing part is in the open state, the first filling rate is calculated using the first method, and if the opening / closing part is in the closed state, the first filling rate is calculated using the second method. The method according to claim 1.

4. The first storage section has an opening and a plurality of through holes, and an opening / closing section arranged to cover the opening. The first filling rate is calculated based on a three-dimensional spatial model generated by measuring through the plurality of through holes using a distance measuring sensor facing the first storage unit. The spatial three-dimensional model is generated based on the shape of the second part of the object to be measured, which is estimated based on the shape of the first part of the object to be measured, measured by the distance measuring sensor through the through-hole. The method according to claim 1.

5. A device for calculating the second filling rate of an object to be measured in a second storage space, which has a second storage space capable of accommodating a plurality of first storage units, comprising a memory and a processor, wherein the processor uses the memory, For each of the one or more first storage units stored in the second storage space, the first filling rate of the object to be measured in the first storage space is obtained. The number of the first storage units that can be additionally stored in the second storage space is obtained, Based on the acquired first filling rate and the number of additionally storable first storage units, the second filling rate of the object to be measured in the second storage space is calculated. Device.

Images