Material mountain inventory management device, inventory management method, and program

Abstract

To provide an inventory management apparatus, an inventory management method, and a program capable of linking and managing material information and shape information of a pile of materials without imposing a manual burden.SOLUTION: In an inventory management system, an inventory management apparatus 100 is provided with: a storage unit 140 for storing material information, which is information about a pile of materials; an acquisition unit 151 for acquiring shape information indicating a three-dimensional surface shape of a target area of a yard containing the pile of materials; an extraction unit 153 for extracting the shape information indicating the three-dimensional surface shape of the pile of materials from the shape information indicating the three-dimensional surface shape of the target area acquired by the acquisition unit 151; and a linking unit 154 for, when an area indicated by position data contained in the material information stored in the storage unit 140 and the area specified by the shape information indicating the three-dimensional surface shape of the pile of materials extracted by the extraction unit 153 overlap even partially, linking the material information stored in the storage unit 140 to the shape information extracted by the extraction unit 153.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a stock management device for a material pile, a stock management method, and a program.

Background Art

[0002] Inventory management of a material pile in which materials such as cement are stacked in a yard is performed. In the inventory management of the material pile, material information such as the brand of the material pile, the position on the yard, the height, and the inventory quantity is acquired based on the input by an operator or the position information of a heavy machine that stacks the materials, and is managed by a computer. In recent years, a technique for measuring the three-dimensional surface shape of a material pile with high accuracy has been developed and used for the inventory management of the material pile. For example, Patent Document 1 discloses a method for measuring a material pile that acquires distance data of a material pile using a three-dimensional distance sensor and acquires the three-dimensional surface shape of the material pile based on the distance data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the measurement method of Patent Document 1, the shape information indicating the three-dimensional surface shape of the material pile cannot be easily associated with the material information managed by a computer, and it is necessary to manually associate the two one by one or manage them separately without associating the two, which requires a human cost for the management of the material pile. Such a problem exists not only when managing a material pile stacked with cement but also when managing the inventory of a material pile stacked with other materials.

[0005] The present invention has been made based on such a background, and an object thereof is to provide an inventory management apparatus, an inventory management method, and a program capable of associating and managing material information and shape information of a material pile without imposing a burden on human hands.

Means for Solving the Problems

[0006] In order to achieve the above object, an inventory management apparatus according to a first aspect of the present invention on the yard Indicating an area set on a horizontal plane so as to surround at least a part of the piled-up material heap including position data Said a storage unit that stores material information, which is information about a material pile; an acquisition unit that acquires shape information indicating a three-dimensional surface shape of a target area of the yard including the material pile; an extraction unit that extracts shape information indicating a three-dimensional surface shape of the material pile from the shape information indicating the three-dimensional surface shape of the target area acquired by the acquisition unit; when a region indicated by position data included in the material information stored in the storage unit and a region specified by the shape information indicating the three-dimensional surface shape of the material pile extracted by the extraction unit overlap even partially, Set on a horizontal plane so as to surround at least a part of the material heap an association unit that associates the material information stored in the storage unit with the shape information extracted by the extraction unit; On a horizontal plane and is provided with.

[0007] Each shape information includes point cloud data composed of a large number of points arranged on the surfaces of the target area and the material pile, respectively. The extraction unit may extract, as point cloud data indicating the three-dimensional surface shape of the material pile, point cloud data composed of points having a height equal to or higher than an extraction threshold value from the point cloud data indicating the three-dimensional surface shape of the target area.

[0008] The extraction unit may remove the material pile as noise when the volume of the region covered by the point cloud data indicating the three-dimensional surface shape of the extracted material pile is equal to or less than a noise threshold value.

[0009] ​When at least a part of the rectangular area indicated by the position data included in the material information stored in the storage unit overlaps with the rectangular area specified by the point cloud data indicating the three-dimensional surface shape of the material heap extracted by the extraction unit, the material information stored in the storage unit and the shape information extracted by the extraction unit may be associated with each other.

[0010] When at least a part of a plurality of material heaps overlap to form an integrated material heap, the material information includes shape data of each material heap generated based on the positions of the vertices of each material heap and the angle of repose of the material deposited on each material heap. The inventory management device further includes a classification unit that generates shape information indicating the three-dimensional surface shape of each material heap based on the shape data of each material heap included in the material information stored in the storage unit and the shape information indicating the three-dimensional surface shape of the integrated material heap extracted by the extraction unit. The association unit may associate the material information stored in the storage unit with the shape information indicating the three-dimensional surface shape of each material heap generated by the classification unit.

[0011] The inventory management device further includes an estimation unit that estimates shape information indicating the three-dimensional surface shape of each material heap from the shape information indicating the three-dimensional surface shape of the integrated material heap in which at least a part of a plurality of material heaps overlap, extracted by the extraction unit. The association unit may associate the material information stored in the storage unit with the shape information indicating the three-dimensional surface shape of each material heap estimated by the estimation unit.

[0012] In order to achieve the above object, an inventory management device according to a second aspect of the present invention includes A storage unit that stores material information, which is information about the material heap including position data indicating an area where the material heap is piled up in the yard An acquisition unit that acquires shape information indicating a three-dimensional surface shape of a target area of the yard including the material heap An extraction unit that extracts shape information indicating a three-dimensional surface shape of the material heap from the shape information indicating the three-dimensional surface shape of the target area acquired by the acquisition unit An estimation unit that estimates shape information indicating a three-dimensional surface shape of each material heap from the shape information indicating a three-dimensional surface shape of a lump of material heaps in which at least a part of a plurality of material heaps overlap, which is extracted by the extraction unit When an area indicated by the position data included in the material information stored in the storage unit and an area specified by the shape information indicating the three-dimensional surface shape of each material heap estimated by the estimation unit overlap even partially, an association unit that associates the material information stored in the storage unit with the shape information indicating the three-dimensional surface shape of each material heap estimated by the estimation unit Comprising Each shape information includes point cloud data composed of a large number of points arranged on the surface of the target area and the material heap, respectively. The estimation unit extracts, from point cloud data indicating the three-dimensional surface shape of a lump of material mountains where at least a part of a plurality of material mountains overlap, point cloud data composed of points with a height equal to or greater than a division threshold value as the point cloud data of the central region of each material mountain, and assigns the point cloud data of the surrounding region around the central region among the point cloud data indicating the three-dimensional surface shape of the lump of material mountains as the point cloud data of each material mountain according to the distance from the central region of each material mountain. Yes

[0013] To achieve the above object, the inventory management method according to the 3 viewpoint of the present invention is an inventory management method executed by an inventory management device, a step of an acquisition unit acquiring shape information indicating the three-dimensional surface shape of a target region of a yard including a material mountain; a step of an extraction unit extracting shape information indicating the three-dimensional surface shape of the material mountain from the shape information indicating the three-dimensional surface shape of the target region acquired by the acquisition unit; a linking unit Said when a region indicated by position data included in material information, which is information about the material mountain, Set on a horizontal plane so as to surround at least a part of the material heap piled up in the yard and 、 a region specified by the shape information indicating the three-dimensional surface shape of the material mountain extracted by the extraction unit On a horizontal plane overlap even partially, linking the material information and the shape information extracted by the extraction unit; and includes.

[0014] To achieve the above object, the program according to the 4 viewpoint of the present invention is a computer acquisition means for acquiring shape information indicating the three-dimensional surface shape of a target region of a yard including a material mountain, extraction means for extracting shape information indicating the three-dimensional surface shape of the material mountain from the shape information indicating the three-dimensional surface shape of the target region acquired by the acquisition means, Said when a region indicated by position data included in material information, which is information about the material mountain, Set on a horizontal plane so as to surround at least a part of the material heap piled up in the yard and、 When there is even partial overlap between a region specified by shape information indicating the three-dimensional surface shape of the material pile extracted by the extraction means On a horizontal plane and the association means for associating the material information with the shape information extracted by the extraction means functions as such. [Effect of the Invention]

[0015] According to the present invention, it is possible to provide an inventory management device, an inventory management method, and a program capable of associating and managing material information and shape information of a material pile without imposing a burden on human hands. [Brief Description of the Drawings]

[0016]

FIG. 1

FIG. 2

FIG. 3

FIG. 4

FIG. 5

FIG. 6

FIG. 7

FIG. 8

FIG. 9

FIG. 10

FIG. 11

FIG. 12

FIG. 13

FIG. 14

FIG. 15

FIG. 16

FIG. 17

FIG. 18

Embodiments for Carrying Out the Invention

[0017] Hereinafter, an inventory management apparatus, an inventory management method, and a program according to embodiments of the present invention will be described in detail with reference to the drawings. In each drawing, the same or equivalent parts are denoted by the same reference numerals. Also, in each embodiment, a rectangular coordinate system is used in which the direction in which the rails installed in the yard extend is the X-axis direction, the direction arranged on the horizontal plane and extending perpendicular to the X-axis direction is the Y-axis direction, and the height direction is the Z-axis direction.

[0018] (Embodiment 1) With reference to FIGS. 1 to 8, an inventory management apparatus, an inventory management method, and a program according to Embodiment 1 will be described. In Embodiment 1, a case of managing the inventory of two stacks of materials piled up in the yard so as not to overlap each other will be described as an example.

[0019] FIG. 1 is a schematic diagram showing the configuration of an inventory management system 1 according to Embodiment 1. The inventory management system 1 is a system for managing various information regarding a stack of cement materials piled up in the yard. The inventory management system 1 includes a measuring device 2 that measures the three-dimensional surface shape of the target area of the yard, and an inventory management device 100 that extracts shape information indicating the three-dimensional surface shape of the stack of materials from the three-dimensional surface shape of the target area measured by the measuring device 2 and stores it in association with the material information of the stack of materials. The measuring device 2 and the inventory management device 100 are communicably connected by a wired or wireless communication circuit.

[0020] The target area of the yard is an area whose three-dimensional surface shape is measured by the measuring device 2, and is set to all or part of the yard so as to include all stacks of materials to be managed. The three-dimensional surface shape of the target area of the yard including the stack of materials is shown in a simplified manner in FIG. 1, but is expressed as shown in FIG. 2, for example. The shape information is information indicating the three-dimensional surface shape of the target area of the yard or the stack of materials, and is expressed, for example, in the form of point cloud data that is a set of a large number of points. The point cloud data is expressed by the coordinate values of points distributed on the surface of the target area of the yard including the stack of materials. On the other hand, the material information includes all information regarding the stack of materials other than the shape information, for example, information such as the name of the stack of materials, the brand of cement, and the inventory quantity.

[0021] Returning to FIG. 1, the measuring device 2 is supported above the yard by a crane 3 and includes, for example, a 2D-LiDAR (Light Detection and Ranging). The 2D-LiDAR includes a light source that radially irradiates a pulsed laser onto a measurement target, and an optical sensor that detects the light reflected by the target object. In the 2D-LiDAR, the distance to the measurement target is obtained from the time required from the emission of the pulsed laser to the detection of the reflected light, and the distance is converted into coordinate values of points indicating the two-dimensional surface shape of the measurement target in the XYZ coordinate system set for the yard, and point cloud data, which is a set of a large number of points indicating the two-dimensional surface shape of the measurement target, is generated.

[0022] The crane 3 is a heavy machine that stacks the incoming cement on the yard. The crane 3 drops the cement from above the yard toward the yard, and includes a long boom 4 that supports the measuring device 2 above the yard, a columnar member (not shown) that extends in the vertical direction and supports the proximal end side of the boom 4, and a traveling mechanism 5 that is installed on the yard, supports the columnar member, and travels on a rail 6 that extends in a direction intersecting the boom 4.

[0023] Hereinafter, a procedure for measuring the three-dimensional surface shape of a target area of the yard using the measuring device 2 will be described. First, the measuring device 2 is made to acquire point cloud data indicating the two-dimensional surface shape of the target area of the yard on a straight line a. When the measurement of the surface shape on the straight line a is completed, the crane 3 is moved in the longitudinal direction of the rail 6 (from the left side to the right side in FIG. 1). Next, the measuring device 2 is made to acquire point cloud data indicating the two-dimensional surface shape of the target area of the yard on a straight line b. The straight line b is set parallel to the straight line a at a position different from the straight line a. By repeating the above steps for straight lines c, d,..., the two-dimensional surface shapes of the target area of the yard on the straight lines a, b, c, d,... are obtained. When the point cloud data indicating the two-dimensional surface shapes of the material piles on the straight lines a, b, c, d,... are arranged in order, point cloud data indicating the three-dimensional surface shape of the target area of the yard is obtained.

[0024] FIG. 3 is a block diagram showing the hardware configuration of the inventory management device 100 according to Embodiment 1. The inventory management device 100 is, for example, a general-purpose computer. The inventory management device 100 includes an operation unit 110, a display unit 120, a communication unit 130, a storage unit 140, and a control unit 150. Each unit of the inventory management device 100 is communicably connected to each other via an internal bus (not shown).

[0025] The operation unit 110 receives a user's instruction and supplies an operation signal corresponding to the received operation to the control unit 150. The operation unit 110 includes, for example, a keyboard and a mouse.

[0026] The display unit 120 displays various images for the user operating the inventory management device 100 based on the image data supplied from the control unit 150.

[0027] The communication unit 130 is an interface that can be connected to a communication network such as the Internet line, for example.

[0028] The storage unit 140 includes, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, and a hard disk drive. The storage unit 140 stores programs and various data to be executed by the control unit 150 and functions as a work memory for the control unit 150 to execute processing. Further, the storage unit 140 includes a material information storage unit 141 and a shape information storage unit 142.

[0029] FIG. 4 shows an example of the data table of the material information storage unit 141 according to Embodiment 1. The material information storage unit 141 is created for each of the material piles A, B,..., and stores material information A, B,... which is information regarding the material piles A, B,.... Each material information includes, for example, information regarding name (pile name), brand of cement, position, height, and inventory quantity. The pile name is a unique name assigned to each material pile and includes identification information. The brand is, for example, the specific gravity or components of the cement that constitutes the material pile. The name and brand of the material pile are input to the operation unit 110 of the inventory management device 100 by the operator, for example, when loading cement by the boom 4.

[0030] The position of the material pile is represented by a rectangular area set so as to surround at least a part of the material pile. The rectangular area is specified by four vertices (X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2). The height of the material pile is the height from the yard at the vertex of the material pile. The inventory quantity is, for example, the weight of the material pile. The position and height of the material pile are estimated from the position information of the boom 4 of the crane 3, and the inventory quantity of the material pile is estimated from the load measured by a belt conveyor (not shown) installed on the crane 3.

[0031] Returning to FIG. 3, the shape information storage unit 142 stores shape information indicating the three-dimensional surface shape of the material pile. The shape information is point cloud data composed of a large number of points indicating the three-dimensional surface shape of the material pile. The shape information is stored in association with a label which is identification information individually assigned to each measured material pile, and the pile name included in the material information associated with the shape information.

[0032] The control unit 150 includes a processor and controls each part of the inventory management device 100. The processor is, for example, a CPU (Central Processing Unit). The control unit 150 executes the inventory management process of FIG. 7 and the extraction process of FIG. 8 by executing the program stored in the storage unit 140. Functionally, the control unit 150 includes an acquisition unit 151, a correction unit 152, an extraction unit 153, an association unit 154, and an output unit 155.

[0033] The acquisition unit 151 acquires point cloud data indicating the three-dimensional surface shape in the target area of the yard from the measuring device 2. Further, the acquisition unit 151 stores the point cloud data indicating the three-dimensional surface shape in the target area of the yard in the storage unit 140, and acquires it from the storage unit 140 at the timing when an instruction from the user is received.

[0034] The correction unit 152 executes correction processing on the point cloud data indicating the three-dimensional surface shape in the target area of the yard acquired by the acquisition unit 151. In the correction processing of the point cloud data, coordinate values that protrude by a threshold or more compared to the surrounding coordinate values among the coordinate values of the points included in the point cloud data may be replaced with the average value of the surrounding coordinate values. Further, when data included in the point cloud data is missing, the missing data may be interpolated based on the coordinate values of surrounding points. For data interpolation, for example, linear interpolation may be used. In addition, the coordinate values of each point included in the point cloud data may be smoothed. For data smoothing, for example, a moving average method may be used.

[0035] The extraction unit 153 extracts point cloud data indicating the three-dimensional surface shape of the material pile from the point cloud data indicating the three-dimensional surface shape of the target area of the yard that has been subjected to correction processing by the correction unit 152. Hereinafter, with reference to FIG. 5, the procedure for extracting point cloud data indicating the three-dimensional surface shape of the material pile from the point cloud data indicating the three-dimensional surface shape of the target area of the yard will be described.

[0036] First, as shown in FIG. 5(a), from the point cloud data indicating the three-dimensional surface shape in the target area of the yard that has been subjected to correction processing, a mass (candidate for the material pile) indicated by point cloud data, which is a set of points located at positions where the height from the reference plane set in the yard is equal to or higher than the extraction threshold, is extracted. The reference plane is set on the horizontal plane, and if the surface of the yard is a horizontal plane, it is set to coincide with the surface of the yard. Among the many points extracted as being at positions where the height from the reference plane is equal to or higher than the extraction threshold, points whose distance between adjacent points is equal to or less than the threshold may be extracted as the points constituting the mass. Labels L1 to L5 may be assigned to each of the extracted masses for identification.

[0037] Next, as shown in Fig. 5(b), a material pile is extracted from each mass to which labels L1 to L5 are assigned. Specifically, the volume of each mass to which labels L1 to L5 are assigned is calculated, and a mass whose volume is equal to or less than the noise threshold is removed as noise. The volume of a mass is the volume of the region covered by the point cloud data included in the mass. The masses remaining after the removal of noise are the material piles. Labels L1 and L2 may be assigned again to each of the extracted material piles for identification.

[0038] Returning to Fig. 3, the association unit 154 associates the material information of the material pile stored in the material information storage unit 141 with the point cloud data indicating the three-dimensional surface shape of the material pile extracted by the extraction unit 153, and stores the point cloud data of the material pile extracted by the extraction unit 153 in the shape information storage unit 142 in association with the material information. Specifically, the region indicated by the position data included in the material information of the material pile stored in the material information storage unit 141 is compared with the region indicated by the point cloud data indicating the three-dimensional surface shape of the material pile, and the material information and the point cloud data are associated when the two regions overlap even partially.

[0039] More specifically, the position data included in the material information is represented, for example, by a rectangular region set in the material pile as shown in Fig. 6. The rectangular region of the material information is set on the XY plane and is specified by four vertices (X1, Y1), (X1, Y2), (X2, Y1), and (X2, Y2) represented by XY coordinate values. Here, X1 < X2 and Y1 < Y2. To compare the rectangular region of the material information with the point cloud data indicating the three-dimensional surface shape of the material pile, a rectangular region may be set for the point cloud data indicating the three-dimensional surface shape of the material pile and the two rectangular regions may be compared. Specifically, first, the minimum X coordinate value Xmin, the maximum X coordinate value Xmax, the minimum Y coordinate value Ymin, and the maximum Y coordinate value Ymax are extracted from a large number of points included in the point cloud data indicating the three-dimensional surface shape of the material pile, and a rectangular region with the four coordinate points (Xmin, Ymin), (Xmin, Ymax), (Xmax, Ymin), and (Xmax, Ymax) as vertices is set.

[0040] Next, compare the rectangular region of the material information with the rectangular region of the point cloud data. If the coordinate values X1, X2, Y1, Y2 and the coordinate values Xmin, Xmax, Ymin, Ymax satisfy all of the following equations (1) to (4), it may be determined that the rectangular region of the material information overlaps with the rectangular region of the point cloud data. X1 < Xmax …(1) X2 > Xmin …(2) Y1 < Ymax …(3) Y2 > Ymin …(4)

[0041] For example, in the specific example of FIG. 6, since the rectangular region of the material information of the material pile A and the rectangular region of the point cloud data of the material pile of the label L1 satisfy the above equations (1) to (4), the point cloud data of the material pile of the label L1 may be associated with the material information of the material pile A. Similarly, the point cloud data of the material pile of the label L2 may be associated with the material information of the material pile B.

[0042] Note that the point cloud data of the material pile may not correspond to any of the material information stored in the material information storage unit 141. For example, this is the case when a material pile stacked by a heavy machine other than the crane 3, for example, a bulldozer, is extracted. In such a case, the point cloud data itself may be deleted by the operation of the operator, or new material information corresponding to the point cloud data may be input and associated with the input material information.

[0043] Returning to FIG. 3, the output unit 155 outputs the point cloud data associated with the material information by the association unit 154. The output unit 155 may create, for example, a display screen in which a data table created based on the material information of each material pile and a three-dimensional view showing the three-dimensional surface shape of each material pile created based on the point cloud data of each material pile are associated, and display it on the display unit 120. The above is the hardware configuration of the inventory management device 100 according to the first embodiment.

[0044] (Inventory Management Process) Next, with reference to the flowchart of FIG. 7, the inventory management process executed by the inventory management device 100 according to Embodiment 1 will be described. The inventory management process is a process of extracting shape information indicating the three-dimensional surface shape of the material pile from the three-dimensional surface shape of the target area of the yard and associating and storing it with the material information of the material pile. Hereinafter, it is assumed that the inventory management device 100 has acquired point cloud data composed of a plurality of points indicating the three-dimensional surface shape of the target area of the yard.

[0045] The acquisition unit 151 acquires point cloud data indicating the three-dimensional surface shape of the target area of the yard from the storage unit 140 (step S1).

[0046] The correction unit 152 executes a correction process on the point cloud data acquired in the process of step S1 (step S2). The correction process for the point cloud data includes, for example, replacement of burst data included in the point cloud data, interpolation of missing data, and smoothing of the point cloud data.

[0047] Next, the extraction unit 153 executes an extraction process of extracting point cloud data indicating the three-dimensional surface shape of the material pile from the point cloud data indicating the three-dimensional surface shape of the target area of the yard that has been corrected in the process of step S2 (step S3). Hereinafter, with reference to the flowchart of FIG. 8, the flow of the extraction process executed by the extraction unit 153 will be described.

[0048] (Extraction process) First, the extraction unit 153 extracts a mass represented by point cloud data at a position where the height from the reference plane of the yard is equal to or higher than the extraction threshold from the point cloud data indicating the three-dimensional surface shape of the target area of the yard that has been corrected in the process of step S2 (step S11).

[0049] Next, the extraction unit 153 assigns a label to each mass extracted in the process of step S11 (step S12). For example, as shown in FIG. 5(a), labels L1 to L5 are assigned to each mass.

[0050] Next, the extraction unit 153 extracts the material piles from the clusters labeled in the process of step S12 (step S13). Specifically, the volume of the clusters labeled in the process of step S12 is calculated, and the clusters with a volume equal to or less than the noise threshold are deleted. The remaining clusters are extracted as material piles.

[0051] Next, the extraction unit 153 assigns labels to the material piles extracted in the process of step S13 (step S14), and returns the process. For example, as shown in FIG. 5(b), labels L1 and L2 are assigned to each material pile. The above is the flow of the extraction process according to the first embodiment.

[0052] Returning to FIG. 7, the association unit 154 compares the material information of the material piles stored in the material information storage unit 141 with the point cloud data indicating the three-dimensional surface shape of the material piles extracted by the extraction unit 153. When the rectangular region indicated by the position data overlaps with the rectangular region indicated by the point cloud data, the material information and the point cloud data are associated (step S4), and the point cloud data associated with the material information is stored in the shape information storage unit 142 (step S5).

[0053] Next, the output unit 155 causes the display unit 120 to display a display image in which the data table created based on the material information and the three-dimensional diagram indicating the three-dimensional surface shape of the material piles created based on the point cloud data are associated (step S6), and ends the process. The above is the flow of the inventory management process according to the first embodiment.

[0054] As described above, the inventory management apparatus 100 according to the first embodiment includes an extraction unit 153 that extracts point cloud data indicating the three-dimensional surface shape of a material pile from point cloud data indicating the three-dimensional surface shape of a target area in a yard, and a region indicated by position data included in the material information and a region specified by the point cloud data indicating the three-dimensional surface shape of the material pile extracted by the extraction unit 153. An association unit 154 that associates the material information and the point cloud data when they overlap even partially. Therefore, the material information and the point cloud data can be associated and managed without imposing a burden on human hands.

[0055] (Embodiment 2) With reference to FIGS. 9 to 12, an inventory management apparatus 100, an inventory management method, and a program according to Embodiment 2 will be described. In Embodiment 1, the shape information of each material pile was extracted when the material piles did not overlap with each other. However, in Embodiment 2, the shape information of each material pile is extracted even when there is a single material pile in which the material piles overlap with each other. Hereinafter, the description will focus on the differences between the two.

[0056] Hereinafter, in Embodiment 2, as shown in FIG. 9(a), a case where two material piles are stacked so as to overlap each other in the point cloud data extracted by the extraction unit 153 will be described as an example. To divide a single material pile composed of two material piles into two material piles, it is convenient to roughly estimate the shapes of the two material piles. Therefore, the inventory management apparatus 100 according to Embodiment 2 has the following configuration.

[0057] FIG. 10 is a block diagram showing the configuration of the inventory management apparatus 100 according to Embodiment 2. The material information storage unit 141 stores, as part of the material information, shape data indicating the shapes of two material piles estimated by the operator. As shown in FIG. 9(b), the shape data is data obtained by estimating the two-dimensional surface shape of each material pile created for each plane (for example, the XZ plane) arranged in the same direction (for example, the Y-axis direction) and extending in the vertical direction. The shape data is estimated based on the angle of repose of each cement and the vertex position of each material pile on the assumption that, for example, a conical material pile formed by cement falling from the crane 3 and spreading downward from the vertex does not collapse even if another material pile is formed so as to overlap the previously formed material pile.

[0058] Functionally, the control unit 150 further includes a division unit 156. The division unit 156 generates point cloud data indicating the three-dimensional surface shape of each material pile based on the shape data of each material pile included in the material information stored in the material information storage unit 141 and the point cloud data indicating the three-dimensional surface shape of a single material pile extracted by the extraction unit 153.

[0059] Specifically, first, the point cloud data indicating the three-dimensional surface shape of a lump of material mountains extracted by the extraction unit 153 is arranged in the same direction (for example, the Y-axis direction), and cut for each plane extending in the vertical direction (for example, the XZ plane), thereby generating point cloud data indicating the two-dimensional surface shape of the lump of material mountains. Specifically, for the point cloud data indicating the three-dimensional surface shape of the lump of material mountains, a plurality of XZ planes arranged in the Y-axis direction are set, and while updating the Y coordinate value at which the XZ plane is set, point cloud data indicating the two-dimensional surface shape of the lump of material mountains may be generated.

[0060] Next, as shown in Fig. 11(a), the shape data indicating the two-dimensional surface shape of each material mountain included in the material information is superimposed on the point cloud data indicating the two-dimensional surface shape of the lump of material mountains. For the point cloud data of the lump of material mountains that overlaps with the shape data of each material mountain, it may be classified into the material mountain corresponding to the shape data. For the point cloud data that does not overlap with the shape data of each material mountain, it may be classified into the material mountain corresponding to the closest shape data. For the point cloud data indicating the boundary between the material mountains, it may be set using the shape data of each material mountain. For the part where the boundary between the material mountains cannot be set using the shape data, specifically, the valley part sandwiched between the shape data of material mountains A and B, it may be simply divided by a straight line extending in the vertical direction. In this way, the point cloud data is classified as shown in Fig. 11(b).

[0061] By repeating the above steps for each XZ plane, point cloud data indicating the three-dimensional surface shape of each material mountain is generated from the point cloud data indicating the three-dimensional surface shape of the lump of material mountains.

[0062] (Inventory management process) Referring to the flowchart of FIG. 12, the inventory management process executed by the inventory management apparatus 100 according to Embodiment 2 will be described. First, the control unit 150 sequentially executes the processes of steps S1 to S3. After the execution of the process of step S3 is completed, the sorting unit 156 sorts the point cloud data of a lump of material mountains A and B that are overlapped and integrated into one based on the shape data included in the material information stored in the material information storage unit 141 for each of the material mountains A and B (step S3A).

[0063] Specifically, as shown in FIG. 11(a), the shape data included in the material information is superimposed on the point cloud data indicating the two-dimensional surface shape of a lump of material mountains cut for each XZ plane arranged in the Y-axis direction, and as shown in FIG. 11(b), the point cloud data indicating the two-dimensional surface shape of a lump of material mountains is sorted for each of the material mountains A and B. By performing the above steps for each Y coordinate value at which the XZ plane is set, point cloud data indicating the two-dimensional surface shape of each of the material mountains A and B is obtained for each Y coordinate value. Then, by arranging the point cloud data indicating the two-dimensional surface shape of each of the material mountains A and B in order in the Y-axis direction, point cloud data indicating the three-dimensional surface shape of each of the material mountains A and B is generated.

[0064] Next, the control unit 150 sequentially executes the processes of steps S4 to S6 for each of the material mountains extracted in the extraction process of step S3 and the point cloud data indicating the three-dimensional surface shape of each of the material mountains sorted by the process of step S3A, and ends the process. The above is the flow of the inventory management process according to Embodiment 2.

[0065] As described above, the inventory management apparatus 100 according to Embodiment 2 further includes a sorting unit 156 that generates point cloud data indicating the three-dimensional surface shape of each material mountain based on the shape data of each material mountain included in the material information and the point cloud data indicating the three-dimensional surface shape of a lump of material mountains. Therefore, even when the material mountains overlap each other, the point cloud data of the material mountains can be extracted and the extracted point cloud data of the material mountains can be associated with the material information.

[0066] (Embodiment 3) With reference to FIGS. 13 to 18, an inventory management apparatus 100, an inventory management method, and a program according to Embodiment 3 will be described. In Embodiment 3, different from the case of Embodiment 2, even when a plurality of separately stacked material piles collapse and integrate due to natural phenomena such as rain or wind, or when an operator intentionally stacks them on top of each other, the material information of each individual material pile is associated with the shape information without using the shape data of the material information.

[0067] Hereinafter, in Embodiment 3, as shown in FIG. 13(a), a case where two material piles are extracted as one integrated material pile will be described as an example. In such a case, even if points equal to or more than the extraction threshold are extracted from the point cloud data indicating the three-dimensional surface shape of the integrated material pile, as shown in FIG. 13(b), the point cloud data included in the overlapping portion of the two material piles is also extracted at the same time, so that each material pile cannot be separated and extracted. Therefore, the inventory management apparatus 100 according to Embodiment 3 has the following configuration.

[0068] FIG. 14 is a block diagram showing the configuration of the inventory management apparatus 100 according to Embodiment 3. Functionally, the control unit 150 further includes an estimation unit 157. The estimation unit 157 estimates the point cloud data indicating the three-dimensional surface shape of each material pile from the point cloud data indicating the three-dimensional surface shape of the integrated material pile extracted by the extraction unit 153.

[0069] Specifically, from the point cloud data indicating the three-dimensional surface shape of an integrated material pile in which a plurality of material piles are integrated, the point cloud data in the central region of each material pile is assigned to each material pile. Next, the point cloud data in the peripheral region around the central region among the point cloud data indicating the three-dimensional surface shape of the integrated material pile is assigned as the point cloud data of each material pile according to the distance from the central region of each material pile. The central region is a region including the apex of the material pile among the integrated material piles, and the peripheral region is a region around the central region among the integrated material piles.

[0070] Next, with reference to FIG. 15, a procedure for dividing the point cloud data of a lump of material extracted by the extraction unit 153 into each lump of material will be described. First, as shown in FIG. 15(a), point cloud data composed of a plurality of points with a height equal to or greater than the division threshold is extracted from the point cloud data indicating the three-dimensional surface shape of a lump of material. The division threshold is a value larger than the extraction threshold and is set to such an extent that the vertices of each lump of material can be extracted from a lump of material. Labels L1 and L2 for identifying each lump of material are attached to each lump of material extracted here.

[0071] Next, the peripheral region around the central region is added to the central region of each extracted lump of material. Specifically, as shown in FIG. 15(b), while updating the X coordinate value, the peripheral region in a lump of material is scanned in the Y-axis direction, and the label of the lump of material detected immediately before or after the peripheral region among the labels L1 and L2 is assigned to each point in the peripheral region. Also, as shown in FIG. 15(c), while updating the Y coordinate value, the peripheral region in a lump of material is scanned in the X-axis direction, and the label of the lump of material detected immediately before or after the peripheral region among the labels L1 and L2 is assigned to each point in the peripheral region. At this time, a peripheral region sandwiched between different lumps of material is divided by the center line of the peripheral region and assigned to each lump of material. By repeating the above processing a plurality of times and deleting the remaining unassigned peripheral regions as noise, point cloud data indicating the three-dimensional surface shape of each lump of material as shown in FIG. 16 can be obtained.

[0072] The association unit 154 associates the material information of the lump of material stored in the material information storage unit 141 with the point cloud data indicating the three-dimensional surface shape of each lump of material estimated by the estimation unit 157. The above is the configuration of the inventory management device according to the third embodiment.

[0073] (Inventory Management Process) Referring to the flowchart of FIG. 17, the inventory management process executed by the inventory management apparatus 100 according to Embodiment 3 will be described. First, the control unit 150 sequentially executes the processes of steps S1 to S3 in FIG. 17. After the execution of the process of step S3 in FIG. 17 is completed, the estimation unit 157 executes an estimation process of estimating the point cloud data indicating the three-dimensional surface shape of each material pile from the point cloud data indicating the three-dimensional surface shape of a group of material piles extracted by the extraction unit 153 (step S3B). Hereinafter, referring to the flowchart of FIG. 18, the estimation process executed by the inventory management apparatus 100 according to Embodiment 3 will be described.

[0074] (Estimation Process) The estimation unit 157 extracts the central region of each material pile from a group of material piles (step S21). The central region of each material pile is represented by point cloud data composed of points whose height is equal to or greater than the division threshold value.

[0075] Next, the estimation unit 157 assigns the peripheral region around the central region of the material pile extracted in the process of step S21 to the region of the material pile extracted in the process of step S21 (step S22). The point cloud data of the peripheral region of the material pile is scanned in the X-axis direction while updating the Y coordinate value, and scanned in the Y-axis direction while updating the X coordinate value, and the label (L1 or L2) of the material pile detected immediately before or after the peripheral region is assigned, so that each material pile is classified.

[0076] Next, the estimation unit 157 deletes the unassigned peripheral region (noise) that was not assigned to any material pile in the process of step S22 (step S23), and returns the process. The above is the flow of the estimation process.

[0077] Returning to FIG. 17, the control unit 150 sequentially executes the processes of steps S4 to S6 for each material pile extracted in the extraction process of step S3 and the point cloud data indicating the three-dimensional surface shape of the material pile obtained in the extraction process of step S3B, and ends the process. The above is the flow of the inventory management process according to Embodiment 3.

[0078] As described above, the inventory management device 100 according to the third embodiment further includes an estimation unit 157 that estimates point cloud data indicating the three-dimensional surface shape of each material pile from the point cloud data extracted by the extraction unit 153 and indicating the three-dimensional surface shape of a lump of material piles. Therefore, even when a plurality of material piles are extracted as a lump of material piles, the point cloud data of each material pile can be extracted and the extracted point cloud data of each material pile can be associated with the material information.

[0079] The present invention is not limited to the above-described embodiments, and the following modifications are also possible.

[0080] (Modification example) In the above embodiment, the 2D-LiDAR is used as the measuring device 2, and the point cloud data indicating the three-dimensional surface shape of the measurement object in the yard is acquired by moving the 2D-LiDAR in the X-axis direction. However, the present invention is not limited to this. To measure the three-dimensional surface shape of the measurement object in the yard, for example, a 3D-LiDAR, a stereo camera, a TOF (Time Of Flight) camera, or an area camera may be used.

[0081] In the above embodiment, the measuring device 2 is attached to the crane 3. However, the present invention is not limited to this. As long as the measuring device 2 can be arranged above the material pile, the measuring device may be supported by a heavy machine other than the crane 3. Further, the measuring device 2 may be supported by a drone, and the position of the drone may be acquired by GPS to acquire the point cloud data indicating the three-dimensional surface shape of the measurement object in the yard.

[0082] In the above embodiment, the shape information indicating the three-dimensional surface shape of the object is expressed by point cloud data. However, the present invention is not limited to this. For example, the point cloud data may be converted into mesh data or surface data and used.

[0083] In the above-described embodiment, the position data included in the material information was represented by a rectangular region set in the material pile, but the present invention is not limited to this. The position data included in the material information may be represented by, for example, a circular, elliptical, triangular, or other polygonal region. Regardless of the shape of the position data included in the material information, when a part of the region indicated by the position data included in the material information overlaps with the region specified by the point cloud data indicating the three-dimensional surface shape of the material pile extracted by the extraction unit, the material information and the shape information may be associated with each other.

[0084] In the above-described embodiment, the identification information of the material pile was included in the pile name of the material information, but the present invention is not limited to this. For example, identification information may be assigned separately from the pile name of the material pile.

[0085] In the above-described embodiment, after associating the material information and the shape information, the shape information associated with the pile name (identification information) of the material information was stored in the shape information storage unit 142, but the present invention is not limited to this. For example, after associating the material information and the shape information, a label of the shape information associated with the material information may be stored in the material information storage unit 141. Alternatively, a data table for storing the pile name of the material information and the label of the shape information associated with each other in the storage unit 140 in a corresponding manner may be provided.

[0086] In the above-described embodiment, a material pile with cement stacked thereon was described as an example, but the present invention is not limited to this. The material pile may be, for example, a pile of coal, iron ore, gravel, compost, fertilizer, feed, or garbage stacked for each brand. Further, the material pile may be a pile of parts or materials obtained by processing raw materials, for example, coils, steel materials, or wood.

[0087] In the above-described embodiment, various data was stored in the storage unit 140 of the inventory management device 100, but the present invention is not limited to this. For example, all or part of the various data may be stored in an external server, computer, or the like via a communication network.

[0088] In the above embodiment, the inventory management device 100 operated based on the programs stored in the storage unit 140 respectively. However, the present invention is not limited to this. For example, the functional configuration realized by the program may be realized by hardware.

[0089] In the above embodiment, the inventory management device 100 was, for example, a general-purpose computer. However, the present invention is not limited to this. For example, the inventory management device 100 may be realized by a computer provided on the cloud.

[0090] In the above embodiment, the processing executed by the inventory management device 100 was realized by a device having the above-described physical configuration executing the program stored in the storage unit 140. However, the present invention may be realized as a program, or may be realized as a storage medium on which the program is recorded.

[0091] Also, a program for executing the above-described processing operation may be stored and distributed in a non-temporary recording medium readable by a computer such as a flexible disk, CD-ROM (Compact Disk Read-Only Memory), DVD (Digital Versatile Disk), MO (Magneto-Optical Disk), etc., and a device for executing the above-described processing operation may be configured by installing the program in a computer.

[0092] The above embodiments are examples, and the present invention is not limited to these. Various embodiments are possible without departing from the spirit of the invention described in the claims. The constituent elements described in each embodiment and modification can be freely combined. Also, inventions equivalent to the invention described in the claims are included in the present invention.

Explanation of Reference Numerals

[0093] 1 Inventory management system 2 Measuring device 3 Crane 4 Boom 5 Travel mechanism 6 Rail 100 Inventory management device 110 Operation unit 120 Display unit 130 Communication unit 140 Memory unit 141 Material information memory unit 142 Shape information memory unit 150 Control unit 151 Acquisition unit 152 Correction unit 153 Extraction unit 154 Association unit 155 Output unit 156 Classification unit 157 Estimation unit

Claims

1. A storage unit that stores material information, which is information about the material pile and includes position data indicating an area set on a horizontal plane so as to surround at least a part of the material pile stacked on the yard; An acquisition unit that acquires shape information indicating the three-dimensional surface shape of a target area of the yard including the material pile; An extraction unit that extracts shape information indicating the three-dimensional surface shape of the material pile from the shape information indicating the three-dimensional surface shape of the target area acquired by the acquisition unit; A linking unit that links the material information stored in the storage unit and the shape information extracted by the extraction unit when at least a part of the area set on the horizontal plane so as to surround at least a part of the material pile indicated by the position data included in the material information stored in the storage unit overlaps with at least a part of the area specified on the horizontal plane by the shape information indicating the three-dimensional surface shape of the material pile extracted by the extraction unit; An inventory management device comprising the above.

2. Each shape information includes point cloud data composed of a large number of points arranged on the surfaces of the target area and the material pile, respectively. The extraction unit extracts, as point cloud data indicating the three-dimensional surface shape of the material pile, point cloud data composed of points with a height equal to or higher than an extraction threshold value from the point cloud data indicating the three-dimensional surface shape of the target area. The inventory management device according to Claim 1.

3. The extraction unit removes the material pile as noise when the volume of the area covered by the point cloud data indicating the three-dimensional surface shape of the extracted material pile is equal to or less than a noise threshold value. The inventory management device according to Claim 2.

4. The linking unit links the material information stored in the storage unit and the shape information extracted by the extraction unit when at least a part of the rectangular area indicated by the position data included in the material information stored in the storage unit overlaps with at least a part of the rectangular area specified by the point cloud data indicating the three-dimensional surface shape of the material pile extracted by the extraction unit. The inventory management device according to Claim 2 or 3.

5. When at least a part of a plurality of material piles overlap to form an integrated material pile, the material information includes shape data of each material pile generated based on the positions of the vertices of each material pile and the angle of repose of the material deposited on each material pile. The inventory management device further includes a classification unit that generates shape information indicating the three-dimensional surface shape of each material pile based on the shape data of each material pile included in the material information stored in the storage unit and the shape information indicating the three-dimensional surface shape of a lump of material piles extracted by the extraction unit. The association unit associates the material information stored in the storage unit with the shape information indicating the three-dimensional surface shape of each material pile generated by the classification unit. The inventory management device according to any one of claims 1 to 4.

6. The inventory management device further includes an estimation unit that estimates shape information indicating the three-dimensional surface shape of each material pile from the shape information indicating the three-dimensional surface shape of a lump of material piles in which at least a part of a plurality of material piles overlap, extracted by the extraction unit. The association unit associates the material information stored in the storage unit with the shape information indicating the three-dimensional surface shape of each material pile estimated by the estimation unit. The inventory management device according to any one of claims 1 to 4.

7. A storage unit that stores material information, which is information about the material pile including position data indicating a region where the material pile is stacked on the yard, An acquisition unit that acquires shape information indicating the three-dimensional surface shape of a target region of the yard including the material pile, An extraction unit that extracts shape information indicating the three-dimensional surface shape of the material pile from the shape information indicating the three-dimensional surface shape of the target region acquired by the acquisition unit, An estimation unit that estimates shape information indicating the three-dimensional surface shape of each material pile from the shape information indicating the three-dimensional surface shape of a lump of material piles in which at least a part of a plurality of material piles overlap, extracted by the extraction unit, An association unit that associates the material information stored in the storage unit with the shape information indicating the three-dimensional surface shape of each material pile estimated by the estimation unit when at least a part of the region indicated by the position data included in the material information stored in the storage unit overlaps with the region specified by the shape information indicating the three-dimensional surface shape of each material pile estimated by the estimation unit, Comprising Each shape information includes point cloud data composed of a large number of points arranged on the surfaces of the target region and the material pile, respectively. The estimation unit extracts, from point cloud data indicating the three-dimensional surface shape of a lump of material mountains where at least a part of a plurality of material mountains overlap, point cloud data composed of points with a height equal to or greater than a division threshold value as the point cloud data of the central region of each material mountain, and assigns the point cloud data of the peripheral region around the central region among the point cloud data indicating the three-dimensional surface shape of the lump of material mountains as the point cloud data of each material mountain according to the distance from the central region of each material mountain. Inventory management device. **Claim 8** An inventory management method executed by an inventory management device, comprising: a step in which an acquisition unit acquires shape information indicating the three-dimensional surface shape of a target region of a yard including a material mountain; a step in which an extraction unit extracts shape information indicating the three-dimensional surface shape of the material mountain from the shape information indicating the three-dimensional surface shape of the target region acquired by the acquisition unit; a step in which an association unit associates material information with the shape information extracted by the extraction unit when a region set on a horizontal plane so as to surround at least a part of the material mountain stacked on the yard indicated by position data included in the material information, which is information regarding the material mountain, and a region specified on the horizontal plane by the shape information indicating the three-dimensional surface shape of the material mountain extracted by the extraction unit overlap even partially; An inventory management method including the above. **Claim 9** A program for causing a computer to function as acquisition means for acquiring shape information indicating the three-dimensional surface shape of a target region of a yard including a material mountain, extraction means for extracting shape information indicating the three-dimensional surface shape of the material mountain from the shape information indicating the three-dimensional surface shape of the target region acquired by the acquisition means, association means for associating material information with the shape information extracted by the extraction means when a region set on a horizontal plane so as to surround at least a part of the material mountain stacked on the yard indicated by position data included in the material information, which is information regarding the material mountain, and a region specified on the horizontal plane by the shape information indicating the three-dimensional surface shape of the material mountain extracted by the extraction means overlap even partially. ​

Images