Filling Rate Measurement Method, Information Processing Apparatus, and Program

Abstract

This filling rate measurement method comprises: acquiring a space three-dimensional model by measuring a first storage section from a first direction side using a range finding sensor, the first storage section having a first storage space in which a measurement object is stored and an opening on the first direction side (S111); acquiring a storage three-dimensional model being a three-dimensional model of the first storage section in which no measurement object is stored (S112); extracting an object portion being a portion corresponding to the measurement object out of the storage three-dimensional model using the acquired space three-dimensional model and the acquired storage three-dimensional model (S114); estimating an object three-dimensional model using the extracted object portion, the object three-dimensional model being a three-dimensional model of the measurement object in the first storage space (S115); and calculating a first filling rate of the measurement object relative to the first storage space using the storage three-dimensional model and the object three-dimensional model (S116).

Description

Technical Field

[0001] The present disclosure relates to a filling rate measurement method, an information processing apparatus, and a program.

Background Art

[0002] Patent Document 1 discloses a three-dimensional shape measurement apparatus that acquires a three-dimensional shape using a three-dimensional laser scanner.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Sufficient consideration has not been given to application examples of the measured three-dimensional shape. For example, sufficient consideration has not been given to the calculation of the filling rate indicating how much of the measurement object is stored in a predetermined storage space.

[0005] The present disclosure provides a filling rate measurement method and the like that can calculate the filling rate of a measurement object.

Means for Solving the Problems

[0006] The filling rate measurement method according to one aspect of the present disclosure has a first storage space in which a measurement object is stored, and a first storage unit having an opening is measured through the opening by a distance measurement sensor facing the first storage unit to obtain a spatial three-dimensional model. A storage three-dimensional model, which is a three-dimensional model of the first storage unit in which the measurement object is not stored, is obtained. Using the obtained spatial three-dimensional model and the storage three-dimensional model, an object part, which is a part corresponding to the measurement object in the spatial three-dimensional model, is extracted. Using the extracted object part, an object three-dimensional model, which is a three-dimensional model of the measurement object in the first storage space, is estimated. Using the storage three-dimensional model and the object three-dimensional model, a first filling rate of the measurement object with respect to the first storage space is calculated.

[0007] The information processing apparatus according to one aspect of the present disclosure includes a processor and a memory. The processor uses the memory to obtain a spatial three-dimensional model obtained by measuring, from the first direction side through the opening, a first storage unit having a first storage space in which a measurement object is stored and an opening formed therein, by a distance measurement sensor facing the first storage unit. A storage three-dimensional model, which is a three-dimensional model of the first storage unit in which the measurement object is not stored, is obtained. Using the obtained spatial three-dimensional model and the storage three-dimensional model, an object part, which is a part corresponding to the measurement object in the storage three-dimensional model, is extracted. Using the extracted object part, an object three-dimensional model, which is a three-dimensional model of the measurement object in the first storage space, is estimated. Using the storage three-dimensional model and the object three-dimensional model, a first filling rate of the measurement object with respect to the first storage space is calculated.

[0008] Note that the present disclosure may be implemented as a program that causes a computer to execute the steps included in the filling rate measurement method. Further, the present disclosure may be implemented as a non-transitory recording medium such as a CD-ROM that can be read by a computer on which the program is recorded. Further, the present disclosure may be implemented as information, data, or a signal indicating the program. And those programs, information, data, and signals may be distributed via a communication network such as the Internet.

Advantages of the Invention

[0009] According to the present disclosure, it is possible to provide a filling rate measurement method or the like that can calculate the filling rate of a measurement object.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

DETAILED DESCRIPTION OF THE INVENTION

[0011] (BACKGROUND OF THE DISCLOSURE) In the logistics and distribution fields, it is required to measure the filling rate of measurement objects such as luggage in storage spaces and improve the utilization efficiency of the storage spaces. Also, in the logistics and distribution fields, since measurement objects are stored in many storage units such as containers, it is required to measure more filling rates in a short time. However, methods for easily measuring the filling rate have not been sufficiently studied.

[0012] Therefore, in the present disclosure, by applying a technique for generating a three-dimensional model for a storage unit in which a measurement object is stored, a filling rate measurement method or the like for easily calculating the filling rates of more storage units in a short time is provided.

[0013] The filling rate measurement method according to one aspect of the present disclosure has a first storage space in which an object to be measured is stored, and a first storage unit having an opening is measured through the opening by a distance measurement sensor facing the first storage unit to obtain a spatial three-dimensional model, obtains a storage three-dimensional model that is a three-dimensional model of the first storage unit in which the object to be measured is not stored, uses the obtained spatial three-dimensional model and the storage three-dimensional model to extract an object portion that is a portion corresponding to the object to be measured in the spatial three-dimensional model, uses the extracted object portion to estimate an object three-dimensional model that is a three-dimensional model of the object to be measured in the first storage space, and uses the storage three-dimensional model and the object three-dimensional model to calculate a first filling rate of the object to be measured with respect to the first storage space.

[0014] According to this, the object three-dimensional model of the object to be measured is estimated using the spatial three-dimensional model obtained by measuring the first storage unit in the state where the object to be measured is stored and the object portion extracted using the storage three-dimensional model of the first storage unit in which the object to be measured is not stored. Thereby, it is possible to easily calculate the first filling rate of the object to be measured with respect to the first storage space only by measuring the first storage unit in the state where the object to be measured is stored.

[0015] Further, in the estimation, the object three-dimensional model may be estimated based on a first three-dimensional coordinate system based on a part of the shape of the first storage unit.

[0016] Therefore, the processing amount of estimating the object three-dimensional model can be reduced.

[0017] Furthermore, the first three-dimensional coordinate system may be calculated based only on the part of the shape of the first storage unit.

[0018] Therefore, only the shape of a part of the first storage unit that is easy to extract on the image can be used for calculating the first three-dimensional coordinate system. Thus, the processing speed of estimating the object three-dimensional model can be improved, and the calculation accuracy of the first three-dimensional coordinate system can be improved.

[0019] Further, a part of the shape may be the shape of the opening.

[0020] Therefore, a coordinate system based on the shape of the opening can be easily calculated, and a three-dimensional model of the object can be estimated based on the calculated coordinate system.

[0021] Further, in the estimation, the three-dimensional model of the object may be estimated based on a first three-dimensional coordinate system with reference to the position of the marker installed in the first storage unit.

[0022] Therefore, a coordinate system based on the marker can be easily calculated, and a three-dimensional model of the object can be estimated based on the calculated coordinate system.

[0023] Further, in the estimation, in the direction from the distance measurement sensor toward the measurement object, the shape of a second part that does not face the measurement object may be estimated based on the shape of a first part where the distance measurement sensor faces the measurement object, thereby estimating the three-dimensional model of the object.

[0024] Therefore, even when there is a shape of a second part where the distance measurement sensor does not face the measurement object, the three-dimensional model of the object can be estimated.

[0025] Further, the first storage unit further has a through hole and an opening / closing part that is arranged to cover the opening in a closed state. The first part is a part that faces the through hole in the opening / closing part in the closed state in the said direction, and the second part is a part that is hidden by the opening / closing part in the closed state in the said direction. The filling rate measurement method further determines whether the opening / closing part is in an open state or a closed state. When the opening / closing part is in the open state, the three-dimensional model of the object is estimated by performing the extraction and the estimation. When the opening / closing part is in the closed state, the second part is estimated based on the first part, and the three-dimensional model of the object may be estimated using the first part, the estimated second part, and the stored three-dimensional model.

[0026] According to this, even when measuring an object to be measured is stored in the first storage unit provided with the opening / closing part for opening and closing the opening, since the estimation method of the three-dimensional model of the object is switched according to the open / closed state of the opening / closing part, the three-dimensional model of the object can be appropriately estimated.

[0027] Also, the said direction may be along the horizontal direction.

[0028] For this reason, since it is not necessary to adjust the position of the distance measuring sensor so that measurement can be performed from a direction without the opening / closing part having the through hole, the degree of freedom in installing the distance measuring sensor is high. Therefore, even if the position of the distance measuring sensor cannot be adjusted, it is possible to obtain a measurement result for estimating the three-dimensional model of the object by the distance measuring sensor.

[0029] Further, in the said calculation, the ratio of the volume of the measurement object stored in the first storage space to the volume of the space in the first storage space where the measurement object can be stored may be calculated as the first filling rate.

[0030] For this reason, it is possible to calculate the first filling rate for appropriately determining how much of the measurement object can be stored in the empty space of the first storage space.

[0031] Further, the first storage unit and the additional first storage unit are stored in a second storage space of the second storage unit, and the filling rate measurement method may further calculate a second filling rate of the first storage unit and the additional first storage unit with respect to the second storage space.

[0032] Therefore, the second filling rate when one or more first storage units are stored in the second storage space can be appropriately calculated.

[0033] Further, the storage three-dimensional model may be a three-dimensional model measured by the distance measurement sensor and an additional distance measurement sensor.

[0034] Therefore, a storage three-dimensional model with less occlusion can be generated.

[0035] Further, the distance measurement sensor has at least two cameras for generating the spatial three-dimensional model and is fixed above the first storage unit.

[0036] In this way, when the distance measurement sensor is fixed above the first storage unit, what is photographed by the two cameras of the distance measurement sensor is limited to the ground or the pedestal (bottom surface) of the first storage unit, etc., and there is nothing that can move other than the first storage unit. Therefore, it is easy to separate the measurement object from the background.

[0037] An information processing apparatus according to an aspect of the present disclosure includes a processor and a memory. The processor has a first storage space in which a measurement object is stored, using the memory, and a first storage unit having an opening formed therein is measured from the first direction side through the opening by a distance measurement sensor facing the first storage unit to obtain a spatial three-dimensional model. A storage three-dimensional model, which is a three-dimensional model of the first storage unit in which the measurement object is not stored, is obtained. Using the obtained spatial three-dimensional model and the storage three-dimensional model, an object part, which is a part corresponding to the measurement object in the storage three-dimensional model, is extracted. Using the extracted object part, an object three-dimensional model, which is a three-dimensional model of the measurement object in the first storage space, is estimated. Using the storage three-dimensional model and the object three-dimensional model, a first filling rate of the measurement object with respect to the first storage space is calculated.

[0038] According to this, the object three-dimensional model of the measurement object is estimated using the spatial three-dimensional model obtained by measuring the first storage unit in a state where the measurement object is stored and the object part extracted using the storage three-dimensional model of the first storage unit in which the measurement object is not stored. Thereby, it is possible to easily calculate the first filling rate of the measurement object with respect to the first storage space only by measuring the first storage unit in a state where the measurement object is stored.

[0039] Note that the present disclosure may be realized as a program that causes a computer to execute steps included in the above filling rate measurement method. Further, the present disclosure may be realized as a non-temporary recording medium such as a CD-ROM readable by a computer on which the program is recorded. Further, the present disclosure may be realized as information, data, or a signal indicating the program. And those programs, information, data, and signals may be distributed via a communication network such as the Internet.

[0040] Hereinafter, each embodiment of the filling rate measurement method and the like according to the present disclosure will be described in detail with reference to the drawings. Note that each of the embodiments described below shows a specific example of the present disclosure. Therefore, the numerical values, shapes, materials, components, arrangements and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure.

[0041] Also, each figure is a schematic diagram and is not necessarily drawn precisely. Further, in each figure, the same reference numerals are given to substantially the same configurations, and redundant descriptions may be omitted or simplified.

[0042] (Embodiment) With reference to FIG. 1, an overview of the filling rate measurement method according to the embodiment will be described.

[0043] FIG. 1 is a diagram for explaining an overview of the filling rate measurement method according to the embodiment.

[0044] In the filling rate measurement method, as shown in FIG. 1, the luggage 103 stored in the shelf 102 having the storage space 101 is measured using the distance measurement sensor 210. Then, using the obtained measurement result, the filling rate of the luggage 103 with respect to the storage space 101 is calculated. An opening 102a for taking the luggage 103 in and out of the storage space 101 is formed in the shelf 102. The distance measurement sensor 210 is arranged at a position facing the opening 102a of the shelf 102 in a direction to measure the shelf 102 including the opening 102a, and measures the measurement region R1 including the inside of the storage space 101 through the opening 102a.

[0045] Incidentally, as shown in FIG. 1 for example, the shelf 102 has a box-like shape. The shelf may not have a box-like shape as long as it has a configuration including a placement surface on which the load 103 is placed and a storage space 101 above the placement surface where the load 103 is stored. The shelf 102 is an example of a first storage unit. The storage space 101 is an example of a first storage space. Although the storage space 101 is the internal space of the shelf 102, it is not limited to this, and it may be the space in a warehouse where measurement objects such as the load 103 are stored. The load 103 is an example of a measurement object. The measurement object is not limited to the load 103 and may be a commodity. That is, the measurement object may be any object as long as it is a portable object.

[0046] FIG. 2 is a block diagram showing a characteristic configuration of the three-dimensional measurement system according to the embodiment. FIG. 3 is a diagram for explaining a first example of the configuration of the distance measurement sensor. FIG. 4 is a diagram for explaining a second example of the configuration of the distance measurement sensor. FIG. 5 is a diagram for explaining a third example of the configuration of the distance measurement sensor.

[0047] As shown in FIG. 2, the three-dimensional measurement system 200 includes a distance measurement sensor 210 and an information processing device 220. The three-dimensional measurement system 200 may include a plurality of distance measurement sensors 210 or may include one distance measurement sensor 210.

[0048] The distance measuring sensor 210 measures a three-dimensional space including the storage space 101 of the shelf 102 through the opening 102a of the shelf 102, thereby obtaining a measurement result including the shelf 102 and the storage space 101 of the shelf 102. Specifically, the distance measuring sensor 210 generates a spatial three-dimensional model represented by a set of three-dimensional points indicating the three-dimensional positions of a plurality of measurement points on the shelf 102 or the package 103 (hereinafter referred to as the measurement target) (the surface of the measurement target). The set of three-dimensional points is referred to as a three-dimensional point cloud. The three-dimensional position indicated by each three-dimensional point in the three-dimensional point cloud is represented by, for example, three-dimensional coordinates of three-value information composed of an X component, a Y component, and a Z component in a three-dimensional coordinate space composed of XYZ axes. Note that the three-dimensional model may include not only three-dimensional coordinates but also color information indicating the color of each point or shape information representing the surface shape of each point and its periphery. The color information may be represented, for example, in an RGB color space or in another color space such as HSV, HLS, or YUV.

[0049] A specific example of the distance measuring sensor 210 will be described with reference to FIGS. 3 to 5.

[0050] As shown in FIG. 3, the distance measurement sensor 210 of the first example generates a spatial three-dimensional model by emitting electromagnetic waves and acquiring reflected waves obtained by reflecting the emitted electromagnetic waves from the measurement target. Specifically, the distance measurement sensor 210 measures the time taken for the emitted electromagnetic waves to be reflected by the measurement target and return to the distance measurement sensor 210 after being emitted, and uses the measured time and the wavelength of the electromagnetic waves used for the measurement to calculate the distance between the distance measurement sensor 210 and the point P1 on the surface of the measurement target. The distance measurement sensor 210 emits electromagnetic waves in a plurality of predetermined radial directions from a reference point of the distance measurement sensor 210. For example, the distance measurement sensor 210 may emit electromagnetic waves at a first angular interval around the horizontal direction and emit electromagnetic waves at a second angular interval around the vertical direction. Therefore, the distance measurement sensor 210 can calculate the three-dimensional coordinates of a plurality of points on the measurement target by detecting the distances between the distance measurement sensor 210 and the measurement target in a plurality of directions around the distance measurement sensor 210. Thus, the distance measurement sensor 210 can calculate position information indicating a plurality of three-dimensional positions on the measurement target and generate a spatial three-dimensional model having the position information. The position information may be a three-dimensional point cloud including a plurality of three-dimensional points indicating a plurality of three-dimensional positions.

[0051] The distance measurement sensor 210 of the first example is a three-dimensional laser measuring instrument having a laser irradiation unit 211 that irradiates laser light as electromagnetic waves and a laser light receiving unit 212 that receives reflected light obtained by reflecting the irradiated laser light from the measurement target. The distance measurement sensor 210 scans the measurement target with laser light by rotating or swinging a unit including the laser irradiation unit 211 and the laser light receiving unit 212 around different two axes, or by installing a movable mirror (MEMS (Micro Electro Mechanical Systems) mirror) that swings around two axes on the path of the laser for irradiation or reception. Thereby, the distance measurement sensor 210 can generate a high-precision and high-density three-dimensional model of the measurement target.

[0052] The distance measurement sensor 210 exemplifies a three-dimensional laser measuring device that measures the distance to a measurement target by irradiating laser light. However, the present invention is not limited to this, and a millimeter-wave radar measuring device that measures the distance to a measurement target by emitting millimeter waves may also be used.

[0053] Further, the distance measurement sensor 210 may generate a three-dimensional model having color information. The first color information is color information generated using an image captured by the distance measurement sensor 210, and is color information indicating the color of each of a plurality of first three-dimensional points included in the first three-dimensional point cloud.

[0054] Specifically, the distance measurement sensor 210 may incorporate a camera that captures a measurement target around the distance measurement sensor 210. The camera incorporated in the distance measurement sensor 210 generates an image by capturing an area including the irradiation range of the laser light irradiated by the distance measurement sensor 210. Further, the imaging range captured by the camera is associated in advance with the irradiation range. Specifically, the plurality of directions in which the laser light is irradiated by the distance measurement sensor 210 and each pixel in the image captured by the camera are associated in advance, and the distance measurement sensor 210 sets the pixel value of the image associated with the direction of the three-dimensional point as color information indicating the color of each of the plurality of three-dimensional points included in the three-dimensional point cloud.

[0055] As shown in FIG. 4, the distance measurement sensor 210A in the second example is a distance measurement sensor using the structured light method. The distance measurement sensor 210A includes an infrared pattern irradiation unit 211A and an infrared camera 212A. The infrared pattern irradiation unit 211A projects a predetermined infrared pattern 213A onto the surface of the measurement target. The infrared camera 212A acquires an infrared image by photographing the measurement target onto which the infrared pattern 213A is projected. The distance measurement sensor 210A searches for the infrared pattern 213A included in the obtained infrared image, and based on a triangle formed by connecting three positions: the position of a point P1 on the infrared pattern on the measurement target in real space, the position of the infrared pattern irradiation unit 211A, and the position of the infrared camera 212A, calculates the distance from the infrared pattern irradiation unit 211A or the infrared camera 212A to the position of the point P1 on the measurement target. Thereby, the distance measurement sensor 210A can acquire the three-dimensional points of the measurement points on the measurement target.

[0056] Note that the distance measurement sensor 210A can acquire a high-density three-dimensional model by moving the unit of the distance measurement sensor 210A having the infrared pattern irradiation unit 211A and the infrared camera 212A, or by making the infrared pattern irradiated by the infrared pattern irradiation unit 211A a fine texture.

[0057] Also, the distance measurement sensor 210A may generate a three-dimensional model having color information by using the visible light region of the color information that can be acquired by the infrared camera 212A, and associating the obtained visible light region with the three-dimensional points in consideration of the position or orientation of the infrared pattern irradiation unit 211A or the infrared camera 212A. Further, the distance measurement sensor 210A may be configured to further include a visible light camera for adding color information.

[0058] The distance measurement sensor 210B in the third example is a distance measurement sensor that measures three-dimensional points by stereo camera measurement, as shown in FIG. 5. The distance measurement sensor 210B is a stereo camera having two cameras 211B and 212B. The distance measurement sensor 210B acquires a stereo image having parallax by photographing a measurement target at synchronized timing with the two cameras 211B and 212B. The distance measurement sensor 210B performs a matching process of feature points between the two obtained stereo images (two images) and acquires alignment information between the two images with pixel accuracy or sub-pixel accuracy. The distance measurement sensor 210B calculates the distance from either of the two cameras 211B and 212B to the matching position (that is, point P1) on the measurement target based on a triangle formed by connecting the matching position of a point P1 on the measurement target in the real space and the positions of the two cameras 211B and 212B respectively. Thereby, the distance measurement sensor 210B can acquire the three-dimensional points of the measurement points on the measurement target.

[0059] Note that the distance measurement sensor 210B can obtain a high-precision three-dimensional model by moving the unit of the distance measurement sensor 210B having the two cameras 211B and 212B, or by increasing the number of cameras mounted on the distance measurement sensor 210B to three or more, photographing the same measurement target, and performing a matching process.

[0060] Also, by using the cameras 211B and 212B included in the distance measurement sensor 210B as visible light cameras, it is easy to add color information to the obtained three-dimensional model.

[0061] In the present embodiment, the information processing apparatus 220 is described as an example including the distance measurement sensor 210 of the first example, but a configuration including the distance measurement sensor 210A of the second example or the distance measurement sensor 210B of the third example instead of the distance measurement sensor 210 of the first example may also be used.

[0062] Note that the two cameras 211B and 212B can capture monochrome images including visible light images or infrared images. In this case, the matching process between the two images in the three-dimensional measurement system 200 may be performed using, for example, SLAM (Simultaneous Localization And Mapping) or SfM (Structure from Motion). Further, the point cloud density of the measurement space model may be increased by MVS (Multi View Stereo) using the information indicating the positions and postures of the cameras 211B and 212B obtained by performing this process.

[0063] Returning to FIG. 2, the configuration of the information processing apparatus 220 will be described.

[0064] The information processing apparatus 220 includes an acquisition unit 221, a coordinate system calculation unit 222, a model generation unit 223, a filling rate calculation unit 224, and a storage unit 225.

[0065] The acquisition unit 221 acquires the spatial three-dimensional model and images generated by the distance measurement sensor 210. Specifically, the acquisition unit 221 may acquire the spatial three-dimensional model and images from the distance measurement sensor 210. The spatial three-dimensional model and images acquired by the acquisition unit 221 may be stored in the storage unit 225.

[0066] The coordinate system calculation unit 222 calculates the positional relationship between the distance measurement sensor 210 and the shelf 102 using the spatial three-dimensional model and images. Thereby, the coordinate system calculation unit 222 calculates a measurement coordinate system based on a part of the shape of the shelf 102. The coordinate system calculation unit 222 may calculate a measurement coordinate system based only on a part of the shape of the shelf 102. Specifically, the coordinate system calculation unit 222 calculates the measurement coordinate system based on the shape of the opening 102a of the shelf 102 as a part of the shape serving as a reference for calculating the measurement coordinate system. Note that the shape of the opening 102a serving as a reference for calculating the measurement coordinate system may be the corner of the shape of the opening 102a or the side of the shape of the opening 102a when the shape of the opening 102a is rectangular as shown in the embodiment.

[0067] The measurement coordinate system is a three-dimensional orthogonal coordinate system and is an example of the first three-dimensional coordinate system. By calculating the measurement coordinate system, the relative position and orientation of the distance measurement sensor 210 with respect to the shelf 102 can be specified. That is, thereby, the sensor coordinate system of the distance measurement sensor 210 can be aligned with the measurement coordinate system, and calibration between the shelf 102 and the distance measurement sensor 210 can be performed. Note that the sensor coordinate system is a three-dimensional orthogonal coordinate system.

[0068] In the present embodiment, the rectangular parallelepiped-shaped shelf 102 has an opening 102a on one surface of the shelf 102, but is not limited thereto. The shelf may have a configuration in which openings are provided on a plurality of surfaces of the rectangular parallelepiped shape, such as having openings on two surfaces, a front surface and a rear surface, or having openings on two surfaces, a front surface and an upper surface. When the shelf has a plurality of openings, a predetermined reference position may be set for one of the plurality of openings. The predetermined reference position may be set in a three-dimensional point or a space where there is no voxel of the storage three-dimensional model, which is a three-dimensional model of the shelf 102.

[0069] Here, the coordinate system calculation unit 222 of the first example will be described with reference to FIGS. 6 and 7.

[0070] FIG. 6 is a block diagram showing the configuration of the coordinate system calculation unit of the first example. FIG. 7 is a diagram for explaining a method of calculating a measurement coordinate system by the coordinate system calculation unit of the first example.

[0071] The coordinate system calculation unit 222 calculates a measurement coordinate system. The measurement coordinate system is a three-dimensional coordinate system serving as a reference for the spatial three-dimensional model. For example, the distance measurement sensor 210 is installed at the origin of the measurement coordinate system and is installed in a direction facing the opening 102a of the shelf 102. At this time, in the measurement coordinate system, the upward direction of the distance measurement sensor 210 may be set as the X axis, the rightward direction may be set as the Y axis, and the forward direction may be set as the Z axis. The coordinate system calculation unit 222 includes an auxiliary unit 301 and a calculation unit 302.

[0072] As shown in Fig. 7(a), the auxiliary unit 301 sequentially and in real time acquires the image 2001 which is the measurement result of the distance measuring sensor 210 acquired by the acquisition unit 221, and superimposes the adjustment marker 2002 for each sequentially acquired image 2001. The auxiliary unit 301 sequentially outputs the superimposed image 2003 with the adjustment marker 2002 superimposed on the image 2001 to a display device (not shown). The display device sequentially displays the superimposed image 2003 output by the information processing device 220. Note that the auxiliary unit 301 and the display device may be integrally provided on the distance measuring sensor 210.

[0073] The adjustment marker 2002 is a marker for assisting the user to move the distance measuring sensor 210 so that the position and orientation of the distance measuring sensor 210 with respect to the shelf 102 become specific positions and orientations. The user can arrange the distance measuring sensor 210 with respect to the shelf 102 in a specific position and orientation by changing the position and orientation of the distance measuring sensor 210 while looking at the superimposed image 2003 displayed on the display device so that the adjustment marker 2002 overlaps with a predetermined reference position of the shelf 102. The predetermined reference position of the shelf 102 is, for example, the positions of the four corners of the rectangular opening 102a of the shelf 102.

[0074] When the distance measuring sensor 210 is arranged with respect to the shelf 102 in a specific position and orientation, a superimposed image 2003 with four adjustment markers 2002 superimposed on four positions corresponding to the positions of the four corners of the opening 102a of the shelf 102 is generated. For example, the user can align the four adjustment markers 2002 with the positions of the four corners of the opening 102a as shown in Fig. 7(b) by moving the distance measuring sensor 210 so that the adjustment marker 2002 moves in the direction of the arrow shown in Fig. 7(a).

[0075] Note that although the auxiliary unit 301 superimposes the adjustment marker 2002 on the image 2001, the adjustment marker may be superimposed on a spatial three-dimensional model, and the spatial three-dimensional model with the adjustment marker superimposed thereon may be displayed on the display device.

[0076] As shown in Fig. 7(c), the calculation unit 302 calculates a rotation matrix 2005 and a translation vector 2006 indicating the positional relationship between the distance measurement sensor 210 and the shelf 102 when the four adjustment markers 2002 are aligned with the positions of the four corners of the opening 102a. The calculation unit 302 calculates a measurement coordinate system 2000 with an arbitrary corner (one of the four corners) of the opening 102a as the origin by converting the sensor coordinate system 2004 of the distance measurement sensor 210 using the calculated rotation matrix 2005 and translation vector 2006. When the four adjustment markers 2002 are aligned with the positions of the four corners of the opening 102a, the user may input to an input device (not shown). The information processing device 220 may determine the time when the four adjustment markers 2002 are aligned with the positions of the four corners of the opening 102a by acquiring the time when such input is received from the input device. Further, the information processing device 220 may determine whether or not the four adjustment markers 2002 are aligned with the positions of the four corners of the opening 102a by analyzing the image 2001.

[0077] Next, the coordinate system calculation unit 222A of the second example will be described with reference to Figs. 8 and 9.

[0078] Fig. 8 is a block diagram showing the configuration of the coordinate system calculation unit of the second example. Fig. 9 is a diagram for explaining the method of calculating the measurement coordinate system by the coordinate system calculation unit of the second example.

[0079] The coordinate system calculation unit 222A includes a detection unit 311, an extraction unit 312, and a calculation unit 313.

[0080] The detection unit 311 detects a shelf area 2014 corresponding to the shelf 102, as shown in FIG. 9(c), using the spatial three-dimensional model 2011 which is the measurement result of the distance measurement sensor 210 acquired by the acquisition unit 221 as shown in FIG. 9(a) and the stored three-dimensional model 2012 as shown in FIG. 9(b). Note that the stored three-dimensional model 2012 is a three-dimensional model of the shelf 102 in which the load 103 is not stored, and is a three-dimensional model generated in advance using the measurement result of the distance measurement sensor 210 for the shelf 102 when the load 103 is not stored. The stored three-dimensional model 2012 is generated by a model generation unit 223 described later and stored in the storage unit 225. The stored three-dimensional model 2012 may include position information 2013 indicating the positions of the four corners of the opening 102a of the shelf 102.

[0081] As shown in FIG. 9(d), the extraction unit 312 extracts four opening endpoints 2016 which are the positions of the four corners of the opening 2015 in the shelf area 2014, using the position information 2013 in the stored three-dimensional model 2012. The shape of the opening 2015 defined by the four opening endpoints 2016 is an example of a part of the shape serving as a reference for calculating the measurement coordinate system.

[0082] As shown in FIG. 9(e), the calculation unit 313 calculates a rotation matrix 2017 and a translation vector 2018 indicating the positional relationship between the distance measurement sensor 210 and the shelf 102, based on the shape of the four opening endpoints 2016 as seen from the distance measurement sensor 210. The calculation unit 313 calculates the measurement coordinate system 2000 by converting the sensor coordinate system 2004 of the distance measurement sensor 210 using the rotation matrix 2017 and the translation vector 2018. Specifically, when the rotation matrix 2017 is R and the translation vector 2018 is T, the three-dimensional point x in the sensor coordinate system 2004 can be converted into the three-dimensional point X in the measurement coordinate system 2000 by the following formula 1 shown below. Thereby, the calculation unit 313 can calculate the measurement coordinate system 2000.

[0083] X = Rx + T ··· Formula 1

[0084] Next, the coordinate system calculation unit 222B of the third example will be described with reference to FIGS. 10 and 11.

[0085] FIG. 10 is a block diagram showing the configuration of the coordinate system calculation unit of the third example. FIG. 11 is a diagram for explaining a method of calculating a measurement coordinate system by the coordinate system calculation unit of the third example.

[0086] The coordinate system calculation unit 222B includes a detection unit 321, an extraction unit 322, and a calculation unit 323. In the third example, a marker 104 is arranged at a specific position (for example, the upper surface position) of the shelf 102, and the coordinate system calculation unit 222B specifies the measurement coordinate system 2000 based on the position of the marker 104. That is, the measurement coordinate system 2000 in this case is a coordinate system based on the position of the marker 104 installed on the shelf 102.

[0087] Note that the marker 104 has, for example, a check pattern. The marker 104 is not limited to the check pattern as long as it is an alignment mark (positioning mark) having a predetermined shape.

[0088] The detection unit 321 detects a marker region 2024 corresponding to the marker 104 installed on the shelf 102 from the image 2021, which is the measurement result by the distance measurement sensor 210 acquired by the acquisition unit 221 as shown in FIG. 11(a), as shown in FIG. 11(c).

[0089] The extraction unit 322 extracts a pattern contour 2025, which is the contour of the check pattern, from the marker region 2024 on the image 2021 as shown in FIG. 11(d).

[0090] The calculation unit 323 calculates a rotation matrix 2026 and a translation vector 2027 indicating the positional relationship between the distance measurement sensor 210 and the marker 104 based on the shape of the extracted pattern contour 2025. The calculation unit 323 uses the rotation matrix 2026 and the translation vector 2027 and the positional relationship between the stored three-dimensional model 2022 and the marker 2023 shown in FIG. 11(b) to calculate the three-dimensional positional relationship between the distance measurement sensor 210 and the shelf 102, and calculates the measurement coordinate system 2000 by converting the sensor coordinate system 2004 using the calculated three-dimensional positional relationship. Note that the positional relationship between the stored three-dimensional model 2022 and the marker 2023 may be measured in advance, or may be generated in advance based on the design data of the shelf 102 where the marker 104 is arranged.

[0091] Returning to FIG. 2, the model generation unit 223 will be described.

[0092] The model generation unit 223 generates a stored three-dimensional model that is a three-dimensional model of the shelf 102 in which the luggage 103 is not stored. The model generation unit 223 acquires the measurement result of the shelf 102 in which the luggage 103 is not stored by the distance measurement sensor 210, and generates a stored three-dimensional model. Specific processing by the model generation unit 223 will be described later. The generated stored three-dimensional model is stored in the storage unit 225.

[0093] Here, the model generation unit 223 will be specifically described with reference to FIGS. 12 and 13.

[0094] FIG. 12 is a block diagram showing an example of the configuration of the model generation unit. FIG. 13 is a flowchart of the process for calculating the volume of the storage space by the model generation unit.

[0095] The model generation unit 223 includes a detection unit 401, a generation unit 402, and a volume calculation unit 403.

[0096] The detection unit 401 detects a shelf area corresponding to the shelf 102 from the three-dimensional space model measured by the distance measurement sensor 210 (S101). When the three-dimensional measurement system 200 includes a plurality of distance measurement sensors 210, the detection unit 401 performs the process of step S101 for each of the plurality of distance measurement sensors 210. Thereby, the detection unit 401 detects a plurality of shelf areas respectively corresponding to the plurality of distance measurement sensors 210.

[0097] When the three-dimensional measurement system 200 includes a plurality of distance measurement sensors 210, the generation unit 402 integrates the plurality of shelf areas and generates a stored three-dimensional model (S102). Specifically, in order to integrate the plurality of shelf areas, the generation unit 402 may perform alignment of the three-dimensional point cloud by ICP (Iterative Closest Point), or may calculate in advance the relative positional relationship between the plurality of distance measurement sensors 210, and integrate the plurality of shelf areas based on the calculated relative positional relationship. The calculation of the relative positional relationship may be performed by using SfM (Structure from Motion) with a plurality of images respectively acquired by the plurality of distance measurement sensors 210 as multi-viewpoint images. The plurality of distance measurement sensors 210 may be installed based on a design drawing in which the relative positional relationship is determined.

[0098] Instead of using a plurality of distance measurement sensors 210, a stored three-dimensional model of the shelf 102 may be generated by moving one distance measurement sensor 210 and integrating a plurality of shelf areas obtained from the plurality of measurement results measured from a plurality of positions.

[0099] Note that the stored three-dimensional model may be generated based on the 3D CAD data at the time of design of the shelf 102 without using the results measured by the distance measurement sensor 210, or may be generated based on the dimension measurement data of the shelf 102 or the equipment specification data publicly available from the manufacturer.

[0100] In addition, when the three-dimensional measurement system 200 does not include a plurality of distance measurement sensors 210 but only one distance measurement sensor 210 and uses only one measurement result measured from one position, the model generation unit 223 may not have the generation unit 402. That is, the model generation unit 223 may not perform step S102.

[0101] The volume calculation unit 403 calculates the volume of the storage space 101 of the shelf 102 using the stored three-dimensional model (S103).

[0102] Returning to FIG. 2, the filling rate calculation unit 224 will be described.

[0103] The filling rate calculation unit 224 calculates the filling rate of the load 103 with respect to the storage space 101 of the shelf 102. For example, the filling rate calculation unit 224 may calculate the ratio of the volume of the load 103 to the volume of the storage space 101 as the filling rate using the spatial three-dimensional model, the image, and the measurement coordinate system 2000 acquired by the distance measurement sensor 210.

[0104] Here, the filling rate calculation unit 224 will be specifically described with reference to FIGS. 14 and 15.

[0105] FIG. 14 is a block diagram showing an example of the configuration of the filling rate calculation unit. FIG. 15 is a diagram for explaining an example of the method for calculating the filling rate by the filling rate calculation unit. Note that FIG. 15 shows an example in the case where the distance measurement sensor 210 faces the opening 102a of the shelf 102. The distance measurement sensor 210 is disposed on the side of the negative Z-axis direction where the opening 102a of the shelf 102 is formed, and measures the storage space 101 of the shelf 102 through the opening 102a of the shelf 102. That is, the distance measurement sensor 210 is disposed above the shelf 102 in the vertical direction. This example is an example in the case where the measurement coordinate system 2000 is measured by the coordinate system calculation unit 222 of the first example. That is, in this case, the sensor coordinate system 2004 and the measurement coordinate system 2000 coincide with each other.

[0106] The filling rate calculation unit 224 includes an extraction unit 501, an estimation unit 502, and a calculation unit 503.

[0107] The extraction unit 501 extracts a luggage area 2033, which is a part corresponding to the luggage 103 in the spatial three-dimensional model, using the spatial three-dimensional model 2011 and the stored three-dimensional model. Specifically, the extraction unit 501 converts the data structure of the spatial three-dimensional model 2011, which is the measurement result by the distance measurement sensor 210 acquired by the acquisition unit 221 as shown in Fig. 15(a), into voxel data to generate the voxel data 2031 shown in Fig. 15(b). The extraction unit 501 subtracts the stored three-dimensional model 2032, which is the voxelized stored three-dimensional model shown in Fig. 15(c), from the generated voxel data 2031 to extract the luggage area 2033, which is the area where the luggage 103 in the voxel data 2031 is measured, as shown in Fig. 15(d). The luggage area 2033 is an example of an object part that is a part corresponding to the object to be measured.

[0108] The estimation unit 502 estimates a luggage model 2034, which is a three-dimensional model of the luggage 103 in the storage space 101, using the extracted luggage area 2033. Specifically, the estimation unit 502 uses the luggage area 2033 to interpolate the area where the luggage 103 is hidden from the distance measurement sensor 210 in the Z-axis direction, which is the arrangement direction of the distance measurement sensor 210 and the shelf 102, that is, interpolates the luggage area 2033 to the positive Z-axis direction side. For example, for each of the plurality of voxels constituting the luggage area 2033, the estimation unit 502 determines whether the voxel is arranged on the minus Z-axis direction side of the farthest voxel that is arranged on the most positive Z-axis direction side among the plurality of voxels. When the voxel is arranged on the minus Z-axis direction side of the farthest voxel, and when there is no voxel arranged on the positive Z-axis direction side of the voxel, the estimation unit 502 interpolates the voxel to the position in the same Z-axis direction as the farthest voxel. Thereby, the estimation unit 502 estimates a luggage model 2034 as shown in Fig. 15(e).

[0109] The calculation unit 503 calculates the first filling rate of the package 103 with respect to the storage space 101 using the stored three-dimensional model and the package model 2034. Specifically, the calculation unit 503 counts the number of voxels constituting the package model 2034 and multiplies the counted number by a predetermined voxel size to calculate the volume of the package 103. The calculation unit 503 calculates, as the first filling rate, the ratio of the calculated volume of the package 103 to the volume of the storage space 101 of the shelf 102 calculated by the model generation unit 223.

[0110] The distance measurement sensor 210 does not necessarily face the opening 102a of the shelf 102 directly. FIG. 16 is a diagram for explaining another example of the method for calculating the filling rate by the filling rate calculation unit. FIG. 16 shows an example in which the distance measurement sensor 210 is disposed at an angle with respect to the opening 102a of the shelf 102. This example is an example in which the measurement coordinate system 2000 is measured by the coordinate system calculation unit 222A of the second example or the coordinate system calculation unit 222B of the third example. That is, in this case, the sensor coordinate system 2004 is different from the measurement coordinate system 2000.

[0111] The coordinate system used in the example of FIG. 16 is the measurement coordinate system 2000. The estimation unit 502 uses the package region 2033 to interpolate the region where the package 103 is hidden from the distance measurement sensor 210 in the Z-axis direction of the measurement coordinate system 2000, which is the arrangement direction of the distance measurement sensor 210 and the shelf 102, that is, the package region 2033 on the positive Z-axis direction side.

[0112] Since other processing by the filling rate calculation unit 224 is the same as that in the case of FIG. 15, the description thereof is omitted.

[0113] Note that the pair of the spatial three-dimensional model and the image used for the calculation of the measurement coordinate system by the coordinate system calculation unit 222 and the calculation of the filling rate by the filling rate calculation unit 224 may be the results measured by the distance measurement sensor 210 at the same time, or may be the results measured at different times.

[0114] The distance measurement sensor 210 and the information processing device 220 may be communicably connected to each other via a communication network. The communication network may be a public communication network such as the Internet or a dedicated communication network. Thereby, the three-dimensional spatial model and the image obtained by the distance measurement sensor 210 are transmitted from the distance measurement sensor 210 to the information processing device 220 via the communication network.

[0115] Further, the information processing device 220 may acquire the three-dimensional spatial model and the image from the distance measurement sensor 210 without going through the communication network. For example, the three-dimensional spatial model and the image are temporarily stored from the distance measurement sensor 210 in an external storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and the information processing device 220 may acquire the three-dimensional spatial model and the image from the external storage device. Further, the external storage device may be a cloud server.

[0116] The information processing device 220 includes at least a computer system including, for example, a control program, a processing circuit such as a processor or a logic circuit that executes the control program, and a recording device such as an internal memory or an accessible external memory that stores the control program. The functions of each processing unit of the information processing device 220 may be realized by software or by hardware.

[0117] Next, the operation of the information processing device 220 will be described.

[0118] FIG. 17 is a flowchart of a filling rate measurement method performed by the information processing device.

[0119] The information processing device 220 acquires a three-dimensional spatial model from the distance measurement sensor 210 (S111). At this time, the information processing device 220 may further acquire an image of the measurement target from the distance measurement sensor 210.

[0120] The information processing device 220 acquires the stored three-dimensional model stored in the storage unit 225 (S112).

[0121] The information processing apparatus 220 calculates a measurement coordinate system based on the shape of the opening 102a of the shelf 102 (S113). Step S113 is a process performed by the coordinate system calculation unit 222.

[0122] The information processing apparatus 220 extracts a luggage area 2033 corresponding to the luggage 103 in the voxel data 2031 using the voxel data 2031 of the spatial three-dimensional model 2011 and the stored three-dimensional model 2032 of the stored three-dimensional model (S114). Step S114 is a process performed by the extraction unit 501 of the filling rate calculation unit 224.

[0123] The information processing apparatus 220 estimates a luggage model 2034, which is a three-dimensional model of the luggage 103 in the storage space 101, using the extracted luggage area 2033 (S115). Step S115 is a process performed by the estimation unit 502 of the filling rate calculation unit 224.

[0124] The information processing apparatus 220 calculates a first filling rate of the luggage 103 with respect to the storage space 101 using the stored three-dimensional model and the luggage model 2034 (S116). Step S116 is a process performed by the calculation unit 503 of the filling rate calculation unit 224.

[0125] FIG. 18 is a flowchart of a process (S113) for calculating a measurement coordinate system by the coordinate system calculation unit of the first example.

[0126] The coordinate system calculation unit 222 sequentially acquires in real time an image 2001, which is a measurement result by the distance measurement sensor 210, acquired by the acquisition unit 221, and superimposes an adjustment marker 2002 on each sequentially acquired image 2001 (S121). Step S121 is a process performed by the auxiliary unit 301 of the coordinate system calculation unit 222.

[0127] The coordinate system calculation unit 222 acquires the position and orientation of the distance measurement sensor 210 (S122). Step S121 is a process performed by the auxiliary unit 301 of the coordinate system calculation unit 222.

[0128] The coordinate system calculation unit 222 specifies the sensor coordinate system 2004 of the distance measurement sensor 210 by using the position and orientation of the distance measurement sensor 210 when the four adjustment markers 2002 are aligned with the positions of the four corners of the opening 102a, and calculates the measurement coordinate system 2000 by using the specified sensor coordinate system 2004 (S123). Step S123 is a process by the calculation unit 302 of the coordinate system calculation unit 222.

[0129] FIG. 19 is a flowchart of a process (S113) for calculating a measurement coordinate system by the coordinate system calculation unit of the second example.

[0130] The coordinate system calculation unit 222A detects a shelf area 2014 corresponding to the shelf 102 by using the spatial three-dimensional model 2011, which is the measurement result by the distance measurement sensor 210 acquired by the acquisition unit 221, and the stored three-dimensional model 2012 (S121A). Step S121A is a process by the detection unit 311 of the coordinate system calculation unit 222A.

[0131] The coordinate system calculation unit 222A extracts four opening endpoints 2016, which are the positions of the four corners of the opening 2015 in the shelf area 2014, by using the position information 2013 in the stored three-dimensional model 2012 (S122A). Step S122A is a process by the extraction unit 312 of the coordinate system calculation unit 222A.

[0132] The coordinate system calculation unit 222A calculates a rotation matrix 2017 and a translation vector 2018 indicating the positional relationship between the distance measurement sensor 210 and the shelf 102 based on the shape of the four opening endpoints 2016 as seen from the distance measurement sensor 210. Then, the coordinate system calculation unit 222A calculates the measurement coordinate system 2000 by converting the sensor coordinate system 2004 of the distance measurement sensor 210 by using the rotation matrix 2017 and the translation vector 2018 (S123A). Step S123A is a process by the calculation unit 313 of the coordinate system calculation unit 222A.

[0133] FIG. 20 is a flowchart of a process (S113) for calculating a measurement coordinate system by the coordinate system calculation unit of the third example.

[0134] The coordinate system calculation unit 222B detects a marker area 2024 from the image 2021 which is the measurement result by the distance measurement sensor 210 acquired by the acquisition unit 221 (S121B). Step S121B is a process by the detection unit 321 of the coordinate system calculation unit 222B.

[0135] The coordinate system calculation unit 222B extracts a pattern contour 2025 from the marker area 2024 on the image 2021 (S122B). Step S122B is a process by the extraction unit 322 of the coordinate system calculation unit 222B.

[0136] The coordinate system calculation unit 222B calculates a rotation matrix 2026 and a translation vector 2027 indicating the positional relationship between the distance measurement sensor 210 and the marker 104 based on the shape of the extracted pattern contour 2025. Then, the coordinate system calculation unit 222B uses the rotation matrix 2026 and the translation vector 2027 and the positional relationship between the stored three-dimensional model 2022 and the marker 2023 to calculate the three-dimensional positional relationship between the distance measurement sensor 210 and the shelf 102, and calculates the measurement coordinate system 2000 by converting the sensor coordinate system 2004 using the calculated three-dimensional positional relationship (S123B). Step S123B is a process by the calculation unit 323 of the coordinate system calculation unit 222B.

[0137] Note that the filling rate calculated by the information processing device 220 may be output from the information processing device 220. The filling rate may be displayed by a display device (not shown) provided in the information processing device 220, or may be transmitted to an external device different from the information processing device 220. For example, the calculated filling rate may be output to the luggage transportation system and used for the control of the luggage transportation system.

[0138] According to the filling rate measurement method according to this embodiment, the luggage model 2034 of the luggage 103 is estimated using the luggage area 2033 extracted using the three-dimensional space model measured for the shelf 102 with the luggage 103 stored therein and the three-dimensional storage model of the shelf 102 without the luggage 103 stored therein. Thereby, by simply measuring the shelf 102 with the luggage 103 stored therein, the first filling rate of the luggage 103 with respect to the storage space 101 can be easily calculated.

[0139] Also, in the filling rate measurement method, the luggage model 2034 is estimated based on a three-dimensional coordinate system based on the shape of a part of the shelf 102. Therefore, the processing amount for estimating the luggage model 2034 can be reduced.

[0140] Also, in the filling rate measurement method, the luggage model 2034 is estimated based on a three-dimensional coordinate system based on only the shape of a part of the first storage part that is easy to extract on the image. The shape of only a part of the first storage part can be used for calculating the measurement coordinate system. Therefore, the processing speed for estimating the luggage model can be improved, and the calculation accuracy of the measurement coordinate system can be improved.

[0141] Also, in the filling rate measurement method, the three-dimensional coordinate system is a three-dimensional orthogonal coordinate system having a Z-axis, and in the estimation, the luggage model 2034 is estimated by interpolating the positive Z-axis direction side opposite to the negative Z-axis direction of the luggage area 2033. Therefore, the processing amount for estimating the luggage model 2034 can be effectively reduced.

[0142] Also, in the filling rate measurement method, the three-dimensional coordinate system is a coordinate system based on the shape of the opening 102a of the shelf 102. Therefore, a coordinate system based on the shape of the opening 102a of the shelf 102 can be easily calculated, and the luggage model 2034 can be estimated based on the calculated coordinate system.

[0143] Also, in the filling rate measurement method, the three-dimensional coordinate system is a coordinate system based on the marker 104 installed on the shelf 102. Therefore, the coordinate system based on the marker 104 can be easily calculated, and the luggage model 2034 can be estimated based on the calculated coordinate system.

[0144] Also, in the filling rate measurement method, the distance measurement sensor 210B has at least two cameras for generating a three-dimensional space model. Such a distance measurement sensor 210 including the distance measurement sensor 210B is fixed above the first storage unit.

[0145] Thus, when the distance measurement sensor 210 is fixed above the first storage unit, if the first storage unit can move like a basket cart described later, the objects existing within the measurement range of the distance measurement sensor 210 are limited to the ground or the pedestal (bottom surface) of the basket cart, etc., and there are no objects that can move other than the basket cart. Therefore, it is easy to separate the measurement object from the background from the measurement result. Note that the measurement range is the shooting range of the camera when the distance measurement sensor 210 has a camera. On the other hand, when the distance measurement sensor 210 is fixed other than the upper side, it becomes easier for objects other than the first storage unit to enter the measurement range, so it becomes difficult to separate the measurement object from the background.

[0146] (Modification 1) In the information processing apparatus 220 according to the above-described embodiment, the ratio of the volume of the luggage 103 stored in the storage space 101 to the volume of the storage space 101 is calculated as the filling rate, but it is not limited to this.

[0147] FIG. 21 is a diagram for explaining a method of calculating the filling rate.

[0148] In FIGS. 21(a) and 21(b), the storage space 101 of the shelf 102 has a volume that can exactly store 16 pieces of luggage 103. As shown in FIG. 21(a), when 8 pieces of luggage 103 are arranged without gaps, 8 more pieces of luggage 103 can be stored in the remaining empty storage space 101. On the other hand, as shown in FIG. 21(b), when the luggage is arranged with gaps, if we try to store 8 pieces of luggage 103 in the remaining space of the storage space 101, it is necessary to move the already stored luggage 103. If we store the luggage 103 in the remaining space of the storage space 101 without moving the already stored luggage 103, only 6 pieces of luggage 103 can be stored.

[0149] Thus, although the luggage 103 that can be stored in the remaining space of the storage space 101 is different between the case of FIG. 21(a) and the case of FIG. 21(b), in both cases, the filling rate is calculated to be the same 50%. Therefore, it is conceivable to calculate the filling rate considering the space that can be substantially stored according to the shape of the remaining space of the storage space 101.

[0150] FIG. 22 is a block diagram showing an example of the configuration of the calculation unit of the filling rate calculation unit according to Modification 1. FIG. 23 is a flowchart of the filling rate calculation process of the calculation unit of the filling rate calculation unit according to Modification 1.

[0151] As shown in FIG. 22, the calculation unit 503 includes a luggage volume calculation unit 601, a region division unit 602, a planned luggage measurement unit 603, a region estimation unit 604, and a calculation unit 605.

[0152] The luggage volume calculation unit 601 calculates the luggage volume, which is the volume of the luggage 103, from the luggage model 2034 (S131). The luggage volume calculation unit 601 calculates the volume of the luggage 103 stored in the storage space 101 in the same manner as in the embodiment.

[0153] Next, the region division unit 602 divides the storage space 101 of the space three-dimensional model 2011 into an occupied region 2041 occupied by the luggage 103 and an empty region 2042 not occupied by the luggage 103 (S132).

[0154] Next, the scheduled luggage measurement unit 603 calculates the volume of one piece of luggage to be stored (S133). When there are multiple types of the luggage to be stored and their shapes and sizes are as shown in Fig. 21(c), the scheduled luggage measurement unit 603 calculates the volume of one piece of luggage for each type. For example, the scheduled luggage measurement unit 603 calculates the volume of the luggage 103a, the volume of the luggage 103b, and the volume of the luggage 103c, respectively.

[0155] Next, the area estimation unit 604 estimates the way of placing the luggage 103 to be stored that can store the most luggage in the empty area 2042, and estimates the number of the luggage 103 to be stored in that case. That is, the area estimation unit 604 estimates the maximum number of the luggage 103 to be stored that can be stored in the empty area 2042. The area estimation unit 604 calculates the storable volume in the empty area 2042 by multiplying the volume of one piece of luggage by the number of storable luggage (S134).

[0156] Note that when there are multiple types of luggage, the area estimation unit 604 may estimate how many pieces of luggage of each type can be stored, or may estimate how many pieces of luggage can be stored in a mixture of multiple types. When the area estimation unit 604 stores luggage in a mixture of multiple types, the area estimation unit 604 calculates the integrated value of the volumes obtained by multiplying the volume of one piece of luggage of each type by the number of storable pieces of that type of luggage as the storable volume in the empty area 2042. For example, when the area estimation unit 604 estimates that n1 pieces of the luggage 103a, n2 pieces of the luggage 103b, and n3 pieces of the luggage 103c can be stored, the area estimation unit 604 calculates the integrated value of the first volume obtained by multiplying the volume of the luggage 103a by n1, the second volume obtained by multiplying the volume of the luggage 103b by n2, and the third volume obtained by multiplying the volume of the luggage 103c by n3 as the storable volume in the empty area 2042. Note that n1, n2, and n3 are integers of 0 or more, respectively.

[0157] The calculation unit 605 calculates the filling rate by applying the volume of the stored luggage and the storable volume to the following formula 2 (S135).

[0158] Filling rate (%) = (Volume of stored luggage) / (Volume of stored luggage + Storable volume) × 100 ··· Equation 2

[0159] In this way, the filling rate calculation unit 224 may calculate, as the filling rate, the ratio of the volume of the luggage 103 stored in the storage space 101 to the volume of the space in the storage space 101 where the luggage 103 can be stored.

[0160] Thereby, it is possible to calculate the first filling rate for appropriately determining how much luggage 103 can be stored in the empty space of the storage space 101.

[0161] (Modification 2) In the information processing apparatus 220 according to the above embodiment, although the filling rate of the luggage 103 with respect to the storage space 101 of one shelf 102 is calculated, the filling rate of the luggage 103 with respect to the storage spaces 101 of two or more shelves 102 may be calculated.

[0162] FIG. 24 is a diagram showing an example when two or more shelves are stored in a storage space such as a truck bed. FIG. 25 is a table showing the relationship between the shelves stored in the storage space of the truck bed and their filling rates.

[0163] As shown in FIG. 24, a plurality of cage carts 112 are stored in the truck bed 106 having the storage space 105. The truck bed 106 may be, for example, a van body type truck bed. The truck bed 106 is an example of a second storage unit. The second storage unit is not limited to the truck bed 106 and may be a container or a warehouse.

[0164] The storage space 105 is an example of a second storage space. The storage space 105 has a volume large enough to store a plurality of cage carts 112. In Modification 2, the storage space 105 can store six cage carts 112. Since the storage space 105 can store a plurality of cage carts 112, the storage space 105 is larger than the storage space 111.

[0165] The basket carriage 112 has a storage space 111 capable of storing a plurality of packages 103. The basket carriage 112 is an example of a first storage unit. The storage space 111 is an example of a first storage space. Note that the shelf 102 described in the embodiment may be stored in the storage space 105.

[0166] The plurality of packages 103 are not directly stored on the loading platform 106, but are stored in a plurality of basket carriages 112. Then, the basket carriages 112 storing the plurality of packages 103 are stored on the loading platform 106.

[0167] In this case, the configuration of the calculation unit 503 of the filling rate calculation unit 224 will be described.

[0168] FIG. 26 is a block diagram showing an example of the configuration of the calculation unit of the filling rate calculation unit according to Modification 2. FIG. 27 is a flowchart of the filling rate calculation process of the calculation unit of the filling rate calculation unit according to Modification 2.

[0169] As shown in FIG. 26, the calculation unit 503 according to Modification 2 includes an acquisition unit 701, a counting unit 702, and a calculation unit 703.

[0170] The acquisition unit 701 acquires the number of basket carriages 112 that can be stored on the loading platform 106 (S141). In the case of Modification 2, since the maximum number of basket carriages 112 that can be stored on the loading platform 106 is 6, 6 is acquired.

[0171] The counting unit 702 counts the number of basket carriages 112 to be stored on the loading platform 106 (S142). When the basket carriage 112 shown in FIG. 24 is stored on the loading platform 106, the counting unit 702 counts three as the number of basket carriages 112.

[0172] The calculation unit 703 calculates a second filling rate, which is the filling rate of one or more cage trucks 112 with respect to the loading platform 106 (S143). Specifically, the calculation unit 703 may calculate, as the second filling rate, the ratio of the number of cage trucks 112 stored in the loading platform 106 to the maximum number of cage trucks 112 that can be stored in the loading platform 106. For example, if a maximum of 6 cage trucks 112 can be stored in the loading platform 106 and 3 of them are stored in the loading platform 106, the calculation unit 703 calculates 50% as the second filling rate.

[0173] In addition, for each of the one or more cage trucks 112 to be stored in the loading platform 106, the calculation unit 703 may calculate the filling rate of the goods 103 with respect to the cage truck 112, and use the calculated filling rate to calculate the filling rate of the goods 103 with respect to the second storage space. Specifically, the calculation unit 703 may calculate the average of the filling rates of the goods 103 with respect to the cage trucks 112 as the filling rate of the goods 103 with respect to the second storage space. In this case, when there is remaining space in the storage space 105 of the loading platform 106 where the cage truck 112 can be stored, the calculation unit 703 may calculate the average by setting the filling rates of the cage trucks 112 that can be stored in the remaining space where the cage truck 112 can be stored to 0%.

[0174] For example, when the filling rates of the three cage trucks 112 shown in FIG. 25 are 70%, 30%, and 20% respectively, and a maximum of 6 cage trucks 112 can be stored in the loading platform 106, the filling rates of the 6 cage trucks 112 are set to 70%, 30%, 20%, 0%, 0%, and 0% respectively, and the average obtained by calculating the average may be calculated as the filling rate of the goods 103 with respect to the second storage space.

[0175] Therefore, when one or more cage trucks 112 are stored in the storage space 105, the second filling rate can be appropriately calculated.

[0176] (Modification Example 3) Next, Modification Example 3 will be described.

[0177] FIG. 28 is a diagram for explaining the configuration of the cage truck according to Modification Example 3.

[0178] FIG. 28(a) is a view showing the car body 112 with the opening / closing part 113 in the closed state. FIG. 28(b) is a view showing the car body 112 with the opening / closing part 113 in the open state.

[0179] The car body 112 according to Modification 3 has an opening / closing part 113 for opening and closing the opening 112a. The opening / closing part 113 is a lattice-shaped or mesh-shaped cover having a plurality of through holes 113a. Therefore, even when the opening / closing part 113 of the car body 112 is in the closed state, the distance measuring sensor 210 can measure the three-dimensional shape inside the storage space 111 of the car body 112 through the plurality of through holes 113a and the opening 112a.

[0180] This is because the electromagnetic wave emitted by the distance measuring sensor 210 passes through the plurality of through holes 113a and the opening 112a. In the case of the distance measuring sensor 210A as well, since the infrared pattern irradiated by the distance measuring sensor 210A passes through the plurality of through holes 113a and the opening 112a, even when the opening / closing part 113 of the car body 112 is in the closed state, the three-dimensional shape inside the storage space 111 of the car body 112 can be measured through the plurality of through holes 113a and the opening 112a. Also, in the case of the distance measuring sensor 210B, since the two cameras 211B and 212B can photograph the inside of the storage space 111 through the plurality of through holes 113a and the opening 112a, the three-dimensional shape inside the storage space 111 of the car body 112 can be measured.

[0181] Therefore, the information processing device 220 can determine whether or not the luggage 103 is stored in the storage space 111. However, if the method for calculating the filling rate is not switched to a different method depending on whether the opening / closing part 113 is in the closed state, the open state, or there is no opening / closing part 113, it is difficult to obtain the correct filling rate. For this reason, the filling rate calculation unit 224 according to Modification 3 calculates the filling rate by the first method when the opening / closing part 113 is in the open state, and calculates the filling rate by the second method when the opening / closing part 113 is in the closed state.

[0182] FIG. 29 is a block diagram showing an example of the configuration of the filling rate calculation unit according to Modification 3. FIG. 30 is a flowchart of the filling rate calculation process of the filling rate calculation unit according to Modification 3.

[0183] As shown in FIG. 29, the filling rate calculation unit 224 according to Modification 3 includes a detection unit 801, a switching unit 802, a first filling rate calculation unit 803, and a second filling rate calculation unit 804.

[0184] The detection unit 801 detects the open / closed state of the opening / closing unit 113 using the spatial three-dimensional model (S151). Specifically, the detection unit 801 uses the spatial three-dimensional model to detect that the opening / closing unit 113 is in the closed state when three-dimensional point clouds exist at the respective positions inside and outside the storage space 111 in the front-rear direction (i.e., the direction in which the distance measurement sensor 210 and the cage carriage 112 are arranged) of the region of the opening 112a of the cage carriage 112. The detection unit 801 detects that the opening / closing unit 113 is in the open state when three-dimensional point clouds exist only inside the storage space 111.

[0185] The switching unit 802 determines whether the opening / closing unit 113 is in the open state or the closed state (S152), and switches the next process according to the determination result.

[0186] When the switching unit 802 determines that the opening / closing unit 113 is in the open state (open state in S152), the first filling rate calculation unit 803 calculates the filling rate by the first method (S153). Specifically, the first filling rate calculation unit 803 calculates the filling rate of the cage carriage 112 by performing the same process as the process by the filling rate calculation unit 224 in the embodiment.

[0187] When the switching unit 802 determines that the opening / closing unit 113 is in the closed state (closed state in S152), the second filling rate calculation unit 804 calculates the filling rate by the second method (S154). The details of the second method will be described with reference to FIG. 31.

[0188] FIG. 31 is a diagram for explaining an example of the second method for calculating the filling rate.

[0189] Consider the case where the spatial three-dimensional model 2051 is acquired as shown in Fig. 31(a).

[0190] Fig. 31(b) is an enlarged view of the region R2 in the spatial three-dimensional model 2051. As shown in Fig. 31(b), the second filling rate calculation unit 804 divides the region R2 into a second part where the opening / closing part 113 is detected and a first part where the package 103 is detected.

[0191] The first part is a region including a three-dimensional point group on the back side of the region of the opening 112a. Also, the first part is the part where the distance measuring sensor 210 faces the package 103 in the direction from the distance measuring sensor 210 to the package 103. That is, the first part is the part facing the through hole 113a in the closed opening / closing part 113 in the direction from the distance measuring sensor 210 to the package 103. Note that the opening / closing part 113 may have a configuration with one through hole 113a. Also, the direction from the distance measuring sensor 210 to the package 103 may be along the horizontal direction, for example.

[0192] The second part is a region including a three-dimensional point group on the front side in the front-rear direction of the region of the opening 112a of the basket carriage 112. Also, the second part is the part where the distance measuring sensor 210 does not face the package 103 in the direction from the distance measuring sensor 210 to the package 103. That is, the second part is the part hidden by the closed opening / closing part 113 in the direction from the distance measuring sensor 210 to the package 103.

[0193] The second filling rate calculation unit 804 generates the voxel data 2052 shown in Fig. 31(c) by voxelizing the first part and the second part respectively. In the voxel data 2052, the white region without hatching is the region where the second part is voxelized, and the region with dot hatching is the region where the first part is voxelized.

[0194] Then, the second filling rate calculation unit 804 estimates whether or not the load 103 exists behind the opening / closing part 113 for the white area corresponding to the area of the opening / closing part 113. Specifically, the second filling rate calculation unit 804 assigns a score based on the probability that a load exists in 26 voxels adjacent to the hatched voxels of the dots where the load 103 exists in the voxelized area. Then, as shown in FIG. 31(d), a score added to the voxels indicated by the white area adjacent to the plurality of voxels where the load 103 exists is assigned. The second filling rate calculation unit 804 performs this for all voxels where the load 103 exists, and determines that the load 103 exists in the voxels indicated by the white area where the total score value is equal to or greater than an arbitrary threshold value. For example, when the second filling rate calculation unit 804 sets an arbitrary threshold value to 0.1, since it determines that the load 103 exists in all areas, it can calculate the load model 2053 whose shape of the area hidden by the opening / closing part 113 is estimated as shown in FIG. 31(e).

[0195] In this way, based on the shape of the first part where the distance measurement sensor 210 faces the load 103, the information processing apparatus 220 estimates the shape of the second part where the distance measurement sensor does not face the measurement object, so that even when there is a second part, the object three-dimensional model can be appropriately estimated.

[0196] In addition, when it is ruled that the load 103 is to be arranged in the basket cart 112 without gaps, as shown in FIG. 32, the second filling rate calculation unit 804 may extract the contour R3 of the area where one or more loads 103 are arranged, and determine that the area where the load 103 exists is inside the extracted contour R3. Then, the second filling rate calculation unit 804 may estimate the area of the opening / closing part 113 inside the contour R3 using the three-dimensional point cloud in the area of the plurality of through holes 113a of the opening / closing part 113.

[0197] In the filling rate measurement method according to Modification 3, the cage carriage 112 further has a plurality of through holes 113a and an opening / closing part 113 that opens and closes the opening 112a. In the filling rate measurement method, it is further determined whether the opening / closing part 113 is in an open state or a closed state. When the opening / closing part 113 is in the open state, similar to the filling rate calculation unit 224 of the embodiment, the luggage model 2034 is estimated by performing extraction and estimation. When the opening / closing part 113 is in the closed state, the filling rate calculation unit 224 estimates a second part hidden by the opening / closing part 113 based on a plurality of first parts corresponding to the plurality of through holes 113a of the opening / closing part 113 among the voxel data 2031 based on the spatial three-dimensional model 2011, and estimates the luggage model 2034 using the plurality of first parts, the estimated second part, and the stored three-dimensional model 2032.

[0198] According to this, even when the luggage 103 is stored in the cage carriage 112 provided with the opening / closing part 113 that opens and closes the opening 112a, since the estimation method of the luggage model 2034 is switched between the first method and the second method according to the open / closed state of the opening / closing part 113, the object three-dimensional model can be appropriately estimated.

[0199] Also, in the filling rate measurement method according to Modification 3, the direction from the distance measurement sensor 210 toward the luggage 103 is along, for example, the horizontal direction. For this reason, since it is not necessary to adjust the position of the distance measurement sensor 210 so that measurement can be performed from a direction without the opening / closing part 113 having the through holes 113a, the degree of freedom in installing the distance measurement sensor 210 is high. Therefore, even if the position of the distance measurement sensor 210 cannot be adjusted, measurement results for estimating the object three-dimensional model by the distance measurement sensor 210 can be obtained.

[0200] (Modification 4) FIG. 33 is a diagram for explaining a method of generating a spatial three-dimensional model according to Modification 4.

[0201] As shown in FIG. 33, even when generating a spatial three-dimensional model, the three-dimensional measurement system 200 may integrate the measurement results of a plurality of distance measurement sensors 210 in the same manner as the processing of the model generation unit 223. In this case, the three-dimensional measurement system 200 identifies the positions and postures of the plurality of distance measurement sensors 210 by performing calibration in advance, and based on the identified positions and postures of the plurality of distance measurement sensors 210, integrates the obtained plurality of measurement results to generate a spatial three-dimensional model including a three-dimensional point cloud with less occlusion.

[0202] (Modification 5) FIG. 34 is a diagram for explaining a method of generating a spatial three-dimensional model according to Modification 5.

[0203] As shown in FIG. 34, even when generating a spatial three-dimensional model, the three-dimensional measurement system 200 moves at least one of the cage carts 112 and one distance measurement sensor 210 so as to cross the measurement region R1 of one distance measurement sensor 210, and may integrate the plurality of measurement results obtained by the distance measurement sensor 210 at a plurality of timings during the movement. In this case, the relative position and posture between the cage cart 112 and one distance measurement sensor 210 are calculated, and using the relative position and posture, a spatial three-dimensional model including a three-dimensional point cloud with less occlusion can be generated by integrating the plurality of measurement results.

[0204] (Others) As described above, the filling rate measurement method and the like according to the present disclosure have been described based on the above-described embodiments. However, the present disclosure is not limited to the above-described embodiments.

[0205] For example, in the above embodiment, it was described that each processing unit included in an information processing apparatus or the like is realized by a CPU and a control program. For example, the components of the processing unit may each be constituted by one or more electronic circuits. Each of the one or more electronic circuits may be a general-purpose circuit or a dedicated circuit. The one or more electronic circuits may include, for example, a semiconductor device, an IC (Integrated Circuit), or an LSI (Large Scale Integration). The IC or LSI may be integrated on one chip or may be integrated on a plurality of chips. Here, although it is called an IC or LSI, the name may change depending on the degree of integration, and it may be called a system LSI, a VLSI (Very Large Scale Integration), or a ULSI (Ultra Large Scale Integration). Also, an FPGA (Field Programmable Gate Array) programmed after the manufacture of the LSI can be used for the same purpose.

[0206] Further, the general or specific aspect of the present disclosure may be realized by a system, an apparatus, a method, an integrated circuit, or a computer program. Alternatively, it may be realized by a computer-readable non-transitory recording medium such as an optical disk, an HDD (Hard Disk Drive), or a semiconductor memory in which the computer program is stored. Also, it may be realized by an arbitrary combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0207] In addition, forms obtained by applying various modifications conceivable by those skilled in the art to each embodiment, and forms realized by arbitrarily combining the components and functions in the embodiment without departing from the gist of the present disclosure are also included in the present disclosure.

Industrial Applicability

[0208] The present disclosure is useful as a filling rate measurement method, an information processing apparatus, a program, etc. that can calculate the filling rate of a measurement target.

Description of Symbols

[0209] 101, 105, 111 Storage Spaces 102 Shelf 102a, 112a Openings 103, 103a - 103c Goods 104 Marker 106 Loading Platform 112 Basket Trolley 113 Opening / Closing Part 113a Through - Hole 200 Three - Dimensional Measurement System 210, 210A, 210B Distance Measurement Sensors 211 Laser Irradiation Unit 211A Infrared Pattern Irradiation Unit 211B, 212B Cameras 212 Laser Light Receiving Unit 212A Infrared Camera 220 Information Processing Device 221, 701 Acquisition Units 222, 222A, 222B Coordinate System Calculation Units 223 Model Generation Unit 224 Filling Rate Calculation Unit 225 Storage Unit 301 Auxiliary Unit 302, 313, 323, 503, 605, 703 Calculation Units 311, 321, 401 Detection Units 312, 322, 501 Extraction Units 402 Generation Unit 403 Volume Calculation Unit 502 Estimation Unit 601 Goods Volume Calculation Unit 602 Region Division Unit 603 Scheduled Goods Measurement Unit 604 Region Estimation Unit 702 Counting Unit 801 Detection Unit 802 Switching Unit 803 First Filling Rate Calculation Unit 804 Second Filling Rate Calculation Unit 2000 Measurement Coordinate System 2001 and 2021 Images 2002 Adjustment Marker 2003 Overlay Image 2004 Sensor Coordinate System 2011 and 2051 Spatial 3D Model 2012, 2022, and 2032 Stored 3D Model 2013 Location Information 2014 Shelf Area 2015 Opening 2016 Opening Endpoint 2017 and 2026 Rotation Matrix 2018 and 2027 Translation Vector 2023 Marker 2024 Marker Area 2025 Pattern Outline 2031 and 2052 Voxel Data 2033 Luggage Area 2034 and 2053 Luggage Model 2041 Occupied Area 2042 Empty Area P1 One Point R1 Measurement Area R2 Area R3 Outline

Claims

1. A first storage space for storing an object to be measured, and a first storage portion having an opening, obtains a three-dimensional space model obtained by measuring through the opening by a distance measuring sensor facing the first storage portion, obtains a storage three-dimensional model that is a three-dimensional model of the first storage portion in which the object to be measured is not stored, uses the obtained three-dimensional space model and the storage three-dimensional model to extract an object portion that is a portion corresponding to the object to be measured in the three-dimensional space model, calculates a first three-dimensional coordinate system based only on a part of the shape of the first storage portion, uses the extracted object portion and the calculated first three-dimensional coordinate system to estimate an object three-dimensional model that is a three-dimensional model of the object to be measured in the first storage space, uses the storage three-dimensional model and the object three-dimensional model to calculate a first filling rate of the object to be measured with respect to the first storage space Filling rate measurement method.

2. In the calculation of the first three-dimensional coordinate system, a part of the first storage portion serves as the origin of the first three-dimensional coordinate system The filling rate measurement method according to claim 1.

3. In the calculation of the first three-dimensional coordinate system, the first three-dimensional coordinate system is calculated by converting the sensor coordinate system of the distance measuring sensor The filling rate measurement method according to claim 1 or 2.

4. The shape of the part is the shape of the opening The filling rate measurement method according to any one of claims 1 to 3.

5. In the estimation, in the direction from the distance measuring sensor toward the object to be measured, based on the shape of the first portion where the distance measuring sensor faces the object to be measured, the shape of the second portion that does not face the object to be measured is estimated, thereby estimating the object three-dimensional model The filling rate measurement method according to any one of claims 1 to 3.

6. The first storage portion further has a through hole and an opening / closing portion arranged to cover the opening in a closed state, The first portion is a portion facing the through hole in the closed opening / closing portion in the direction, The second portion is a portion hidden by the closed opening / closing portion in the direction, The filling rate measurement method further determines whether the opening / closing portion is in an open state or a closed state, When the opening / closing portion is in the open state, the object three-dimensional model is estimated by performing the extraction and the estimation, When the opening / closing part is in the closed state, based on the first part, the second part is estimated, and using the first part, the estimated second part, and the stored three-dimensional model, the object three-dimensional model is estimated. The filling rate measurement method according to claim 5.

7. The direction is along the horizontal direction. The filling rate measurement method according to claim 6.

8. In the calculation, among the first storage spaces, the ratio of the volume of the measurement object stored in the first storage space to the volume of the space capable of storing the measurement object in the first storage space is calculated as the first filling rate. The filling rate measurement method according to any one of claims 1 to 7.

9. The first storage part and the additional first storage part are stored in a second storage space of the second storage part. The filling rate measurement method further includes calculating a second filling rate of the first storage part and the additional first storage part with respect to the second storage space. The filling rate measurement method according to any one of claims 1 to 8.

10. The stored three-dimensional model is a three-dimensional model measured by the distance measurement sensor and an additional distance measurement sensor. The filling rate measurement method according to any one of claims 1 to 9.

11. The distance measurement sensor has at least two cameras for generating the spatial three-dimensional model and is fixed above the first storage part. The filling rate measurement method according to any one of claims 1 to 10.

12. a processor; a memory, and includes The processor uses the memory to obtain a spatial three-dimensional model obtained by measuring, through the opening, an opening by a distance measurement sensor facing the first storage part having a first storage space in which a measurement object is stored and having an opening formed therein; obtain a stored three-dimensional model that is a three-dimensional model of the first storage part in which the measurement object is not stored; using the obtained spatial three-dimensional model and the stored three-dimensional model, extract an object part that is a part corresponding to the measurement object in the spatial three-dimensional model; calculate a first three-dimensional coordinate system based only on a part of the shape of the first storage part; using the extracted object part and the calculated first three-dimensional coordinate system, estimate an object three-dimensional model that is a three-dimensional model of the measurement object in the first storage space; using the stored three-dimensional model and the object three-dimensional model, calculate a first filling rate of the measurement object with respect to the first storage space. Information processing apparatus.

13. In the calculation of the first three-dimensional coordinate system, a part of the first storage unit serves as the origin of the first three-dimensional coordinate system The information processing apparatus according to claim 12.

14. In the calculation of the first three-dimensional coordinate system, the first three-dimensional coordinate system is calculated by converting the sensor coordinate system of the distance measuring sensor The information processing apparatus according to claim 12 or 13.

15. A program for causing a computer to execute a filling rate measurement method, wherein the filling rate measurement method is acquiring a spatial three-dimensional model obtained by measuring, through the opening, a first storage unit that has a first storage space in which an object to be measured is stored and in which an opening is formed, by a distance measuring sensor facing the first storage unit; acquiring a storage three-dimensional model that is a three-dimensional model of the first storage unit in which the object to be measured is not stored; extracting an object part that is a part corresponding to the object to be measured in the spatial three-dimensional model, using the acquired spatial three-dimensional model and the storage three-dimensional model; calculating a first three-dimensional coordinate system based only on the shape of a part of the first storage unit; estimating an object three-dimensional model that is a three-dimensional model of the object to be measured in the first storage space, using the extracted object part and the calculated first three-dimensional coordinate system; calculating a first filling rate of the object to be measured with respect to the first storage space, using the storage three-dimensional model and the object three-dimensional model Program.

16. In the calculation of the first three-dimensional coordinate system, a part of the first storage unit serves as the origin of the first three-dimensional coordinate system The program according to claim 15.

17. In the calculation of the first three-dimensional coordinate system, the first three-dimensional coordinate system is calculated by converting the sensor coordinate system of the distance measuring sensor The program according to claim 15 or 16.

Images