Container measurement system

Abstract

The present invention provides a container measurement system capable of accurately calculating the three-dimensional information of a container. The container measurement system includes: a distance image acquisition unit (40) installed in an engineering machine for loading a container, and capable of acquiring a distance image of the container; a calculation unit (66) that calculates three-dimensional information including the three-dimensional position and three-dimensional shape of the container based on the acquired distance image; a storage unit (70) that stores the three-dimensional shapes of multiple types of containers; a determination unit (68) that determines the type of container based on the calculated three-dimensional shape of the container and the three-dimensional shapes of multiple types of containers stored in the storage unit (70); and a correction unit (69) that corrects the calculated three-dimensional information of the container based on the three-dimensional shape of the container whose type has been determined.

Description

Technical Field

[0001] This invention relates to a container measurement system for calculating three-dimensional information of a container. Background Technology

[0002] For example, Patent Document 1 describes a technique for calculating the three-dimensional information of a container. The technique described in this document involves using a camera to capture four feature points of the container and calculating the three-dimensional position of each feature point.

[0003] Existing technology

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Publication No. 2000-064359

[0006] The technology described in Patent Document 1 is difficult to reliably calculate the three-dimensional information of feature points in situations where it is impossible to properly capture feature points using a camera. As a result, it is difficult to reliably calculate the three-dimensional information of the container.

[0007] To address this, one could consider using well-known deep learning techniques to extract feature points. Even when it's impossible to properly capture feature points using a camera, as mentioned above, the 3D information of the feature points can still be calculated. However, even with deep learning techniques, feature points sometimes cannot be extracted correctly. Summary of the Invention

[0008] The purpose of this invention is to provide a container measurement system that can accurately calculate the three-dimensional information of a container.

[0009] The present invention provides a container measurement system. This container measurement system includes: a distance image acquisition unit, installed on an engineering machine performing a loading operation on a container, capable of acquiring a distance image of the container; a calculation unit, which calculates three-dimensional information including the three-dimensional position and three-dimensional shape of the container based on the acquired distance image; a storage unit, which stores multiple types of three-dimensional shapes of the container; a determination unit, which determines the calculated type of the container based on the three-dimensional shape of the container calculated by the calculation unit and the multiple types of three-dimensional shapes of the container stored in the storage unit; and a correction unit, which corrects the three-dimensional information of the container calculated by the calculation unit based on the type of container three-dimensional shape determined by the determination unit. Attached Figure Description

[0010] Figure 1 This is a side view of a transport vehicle and engineering machinery according to one embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of a transport vehicle as seen from one embodiment of the engineering machinery involved in the present invention.

[0012] Figure 3 This is a block diagram of a container measurement system according to one embodiment of the present invention.

[0013] Figure 4 This is a schematic diagram showing three-dimensional information of a container according to an embodiment of the present invention.

[0014] Figure 5 This is an explanatory diagram of a method for inspecting feature points according to an embodiment of the present invention.

[0015] Figure 6 This is a schematic diagram showing a first overlapping image of a container measurement system according to an embodiment of the present invention.

[0016] Figure 7 This is a schematic diagram showing a second overlapping image of a container measurement system according to an embodiment of the present invention.

[0017] Figure 8 This is a flowchart of a container measurement process according to an embodiment of the present invention. Detailed Implementation

[0018] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0019] (Composition of the transport vehicle)

[0020] One embodiment of the present invention provides a container measurement system that calculates three-dimensional information of a container. Figure 1 This is a side view of the transport vehicle 10 and the construction machinery 20 involved in this embodiment. Figure 1 As shown, the transport vehicle 10 is a vehicle with a container 12. The transport vehicle 10 is a vehicle used to transport things (loaded objects) loaded by the engineering machinery 20, and can be a dump truck or a truck.

[0021] The transport vehicle 10 has a main body 11 and a container 12. The main body 11 is traversable on the ground and supports the container 12. The main body 11 has a driver's cab 11a. The container 12 is positioned behind the transport vehicle 10, beyond the driver's cab 11a.

[0022] Container 12 is the loading platform of transport vehicle 10. Container 12 may have, for example, an open-top box shape (see reference). Figure 2 Container 12 has a flat portion 13. Container 12 contains the object to be loaded. The object to be loaded in container 12 may be, for example, sand or waste (industrial waste, etc.). Container 12 may be a movable container that can move relative to the main body 11, or it may be fixed to the main body 11. Furthermore, container 12 may not be a loading platform for transport vehicle 10, for example, it may be placed directly on the ground.

[0023] Hereinafter, regarding the direction related to the transport vehicle 10, the direction from the cab 11a toward the container 12 will be referred to as the "rear direction of the transport vehicle", and the direction from the container 12 toward the cab 11a will be referred to as the "front direction of the transport vehicle".

[0024] Figure 2 This is a schematic diagram showing the transport vehicle 10 as viewed from the engineering machinery 20 involved in this embodiment. (As shown...) Figure 2 As shown, the planar portion 13 is a planar or substantially planar part of the container 12. The planar portion 13 only needs to be planar or substantially planar in general. The planar portion 13 may have unevenness or a gently curved surface.

[0025] The planar portion 13 includes a bottom plate surface 13a, a rear baffle surface 13b, a side baffle surface 13c, and a gantry surface 13d. The bottom plate surface 13a is the bottom surface (lower side) of the container 12. The rear baffle surface 13b is the rear side of the container 12's transport vehicle, protruding upwards from the rear portion of the bottom plate surface 13a. The side baffle surfaces 13c are the left and right sides of the container 12, protruding upwards from the left and right ends of the bottom plate surface 13a. The gantry surface 13d is the front side of the container 12's transport vehicle, protruding upwards from the front portion of the bottom plate surface 13a. The gantry surface 13d protrudes upwards more than the side baffle surfaces 13c and more than the rear baffle surface 13b.

[0026] (Composition of construction machinery)

[0027] like Figure 1 As shown, the construction machinery 20 is machinery that performs the operation of loading objects into container 12 (loading operation). The construction machinery 20 may be, for example, a construction machinery that digs up the object to be loaded, or a construction machinery that can clamp and grab the object to be loaded. The construction machinery 20 may be, for example, construction machinery that performs construction operations, and may be, for example, a hydraulic excavator.

[0028] The construction machinery 20 includes a lower traveling body 21, an upper slewing body 23, and auxiliary devices 25. The lower traveling body 21 enables the construction machinery 20 to move. The lower traveling body 21 may have tracks, for example. The upper slewing body 23 is rotatably mounted on the lower traveling body 21. The upper slewing body 23 has a driver's cab 23a.

[0029] The auxiliary device 25 is the part that moves the loaded object. The auxiliary device 25 includes a boom 25a, a stick 25b, and a front auxiliary device 25c. The boom 25a is mounted on the upper rotating body 23 in an up-and-down (rotatable) manner. The stick 25b is rotatably mounted on the boom 25a (extendable and retractable). The front auxiliary device 25c is located at the front end of the auxiliary device 25 and is rotatably mounted on the stick 25b. The front auxiliary device 25c can be a bucket for digging up the loaded object (e.g., sand), or it can be a device for clamping and grabbing the loaded object (e.g., a grab bucket).

[0030] (Composition of a container measurement system)

[0031] The container measurement system is a system for measuring the three-dimensional information (3D information) of container 12. The "3D information of container 12" includes information on its three-dimensional position (3D coordinates) and three-dimensional shape. The container measurement system measures the position, orientation, and shape of container 12 relative to the engineering machinery 20. As an example, a specified origin (reference point) is set for the engineering machinery 20.

[0032] Figure 3 This is a block diagram of the container measurement system 30. (Example) Figure 3 As shown, the container measurement system 30 includes a distance image acquisition unit 40, a display (display device) 50, a controller 60, and a data storage unit 70.

[0033] The distance image acquisition unit (distance image acquisition unit) 40 acquires such as Figure 2 The distance image D shown includes container 12. Distance image D is an image containing distance information (depth information). That is, distance image D corresponds to the distribution of distances between various measurement points around the construction machinery 20 and the origin. A distance image acquisition unit 40 is provided on the construction machinery 20. The distance image acquisition unit 40 is positioned such that it can acquire the distance image D of container 12 and its periphery when the construction machinery 20 is performing loading operations. The distance image acquisition unit 40 can be configured (installed) inside the cab 23a, for example, or outside the cab 23a. Figure 1 In the example shown, it is disposed on the upper surface of the cockpit 23a. Furthermore, the distance image acquisition unit 40 can also automatically track the container 12 by acquiring a distance image D of the container 12.

[0034] like Figure 3 As shown, the distance image acquisition unit 40 includes a two-dimensional information acquisition unit 42 and a three-dimensional information acquisition unit 43. The two-dimensional information acquisition unit 42 acquires (captures) two-dimensional information (images) containing the container 12. The two-dimensional information acquisition unit 42 is a camera or the like.

[0035] The 3D information acquisition unit 43 acquires 3D information (point group data) of the container 12. The 3D information acquisition unit 43 measures the distance from the 3D information acquisition unit 43 to each part of the container 12 (details will be described below). Specifically, the 3D information acquisition unit 43 may be equipped with LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging), a stereo camera, or a TOF (Time of Flight) sensor.

[0036] The display 50 can be installed in the cab 11a of the transport vehicle 10, or in the cab 23a of the construction machinery 20, or on the outside of both the transport vehicle 10 and the construction machinery 20.

[0037] The controller 60 performs signal input / output, judgment, calculation, and information storage. The controller 60 includes an image processing unit 61, a position coordinate estimation unit 62, a three-dimensional coordinate acquisition unit 63, a coordinate transformation unit 64, a first overlap unit 65, and a feature point position detection unit 66.

[0038] The image processing unit 61 preprocesses the image acquired by the two-dimensional information acquisition unit 42. The preprocessing is a process of removing noise from the image or enhancing the edges within the image.

[0039] The position coordinate estimation unit 62 estimates the shape of the container 12 based on the image preprocessed in the image processing unit 61. Specifically, as follows: Figure 2 As shown, the position coordinate estimation unit 62 extracts (identifies, infers) the positions of feature points F (F1 to F8) in the image and the positions of connecting lines L (L1 to L9) in the image. The position coordinate estimation unit 62 extracts the feature points F and connecting lines L, for example, by executing a program that extracts a specific shape from the image. For example, this program uses deep learning technology. Thus, the position coordinate estimation unit 62 obtains two-dimensional information including the image containing the container 12, the positions of the feature points F in the image, and the positions of the connecting lines L in the image.

[0040] like Figure 2 As shown, feature point F is the point corresponding to an angle of planar portion 13. Connecting line L is a line segment connecting feature points F to each other, and it corresponds, for example, to an edge of planar portion 13. Figure 2In the example shown, the positions of feature point F correspond to the positions of two points at the upper end of the gantry surface 13d (F1, F2), the positions of two points at the upper front end of the side baffle surface 13c and the transport vehicle side of 13c (F3, F4), and the positions of four points at the corners of the rear baffle surface 13b (F5-F8). For example, if the lower end of the gantry surface 13d is mapped into the image, the position of the lower end of the gantry surface 13d can also be extracted as feature point F (see reference). Figure 1 ).

[0041] exist Figure 2 In the example shown, the positions of the connecting lines L correspond to the positions of the four sides of the quadrilateral in the upper part of the gantry surface 13d compared to the side baffle surface 13c (L1 to L4). In addition, the positions of the connecting lines L correspond to the upper edge of the side baffle surface 13c (L5, L6) and the four sides of the rear baffle surface 13b (more specifically, the four sides of the quadrilateral that substantially overlaps with the rear baffle surface 13b) (L7 to L10).

[0042] return Figure 3 The three-dimensional coordinate acquisition unit 63 acquires the three-dimensional coordinates of each point group data captured by the three-dimensional information acquisition unit 43. The coordinate transformation unit 64 performs perspective projection transformation on each three-dimensional coordinate of the point group data to acquire each two-dimensional coordinate of the point group data. The coordinate system of this two-dimensional coordinate is consistent with the coordinate system of the image acquired by the two-dimensional information acquisition unit 42.

[0043] The first overlapping portion 65 overlaps the two-dimensional coordinates of the point group data acquired by the coordinate transformation unit 64 with the shape of the container 12 predicted by the position coordinate estimation unit 62. Therefore, it can be known that the points in the point group data overlap with the coordinates of the container 12. Figure 2 The point corresponding to feature point F shown.

[0044] Alternatively, a coordinate system such as mechanical coordinates can be used to overlap the two-dimensional coordinates of the point group data with the inferred shape of the container 12. Mechanical coordinates are coordinates with a specific position of the engineering machinery 20 as the origin. When using the mechanical coordinate coordinate system, the coordinate system of each two-dimensional coordinate of the point group data is made consistent with the mechanical coordinate coordinate system, and the coordinate system of the image acquired by the two-dimensional information acquisition unit 42 is made consistent with the mechanical coordinate coordinate system (making their relative positions consistent).

[0045] Feature point location detection unit (computation unit) 66 Detection (computation) Figure 2 The feature point F and the connecting line L are shown in three-dimensional coordinates. Specifically, the feature point position detection unit 66 detects the point group data that overlaps with the shape of the container 12 in the first overlapping part 65, corresponding to... Figure 2The three-dimensional coordinates of feature point F are shown, and from this, the three-dimensional coordinates of the angle of the plane portion 13 corresponding to the position of feature point F are calculated. The three-dimensional information of the connecting line L connecting feature point F can also be calculated. Through this calculation, the three-dimensional information of container 12 (specifically feature point F and connecting line L) is obtained.

[0046] Figure 4 This refers to the three-dimensional information of container 12 detected by feature point location detection unit 66. Figure 4 In the example shown, as circled, the 3D information of feature point F1 out of the eight feature points F (F1~F8) is incorrect. Thus, even with deep learning techniques, it is sometimes impossible to extract feature points correctly.

[0047] In addition, such as Figure 3 As shown, the controller 60 includes a container shape detection unit 67, a container determination unit 68, a feature point correction unit 69, a second overlap unit 71, a third overlap unit 72, and a display control unit 73.

[0048] The container shape detection unit 67 detects the three-dimensional shape of the container 12 based on the three-dimensional coordinates of the feature point F calculated by the feature point position detection unit 66 and the three-dimensional information of the connecting line L.

[0049] The container shape detection unit 67 checks eight feature points F (F1 to F8) in the detected three-dimensional shape of the container 12. If three or more feature points F have correct three-dimensional coordinates, the container shape detection unit 67 adopts the detected three-dimensional shape of the container 12. Conversely, if fewer than three feature points F have correct three-dimensional coordinates, the container shape detection unit 67 does not adopt the detected three-dimensional shape of the container 12. In this case, the feature point position detection unit 66 re-detects (calculates) the three-dimensional coordinates of the feature points F and the connecting line L.

[0050] The method for examining each of the eight feature points F is as follows. Figure 5 This is an illustration of the feature point inspection method. For example... Figure 5 As shown, taking feature point F1 as the reference, the inner product between vector A from feature point F1 towards feature point F2 and vector B from feature point F1 towards feature point F3 is obtained. Furthermore, if the angle between the vectors is set to θ, it is determined whether cosθ is 0.

[0051] Next, if cosθ is 0, the three-dimensional coordinates of feature point F1 are determined to be correct; if cosθ is not 0, the three-dimensional coordinates of feature point F1 are determined to be incorrect. Furthermore, the origin of the three-dimensional coordinates of each of the eight feature points F is set at any position on the engineering machinery 20.

[0052] Next, using feature point F2 as a reference, obtain the inner product between the vector C from feature point F2 towards feature point F4 and the vector (-A) from feature point F2 towards feature point F1 (the vector with the opposite direction to vector A). Then, perform the same judgment. This judgment is performed using each of the eight feature points F1 to F8.

[0053] return Figure 3 The data storage unit (storage unit) 70 pre-stores the three-dimensional shapes of multiple types of containers. The container determination unit (determination unit) 68 determines the type of container 12 based on the calculated three-dimensional shape of the container 12 and the three-dimensional shapes of multiple types of containers stored in the data storage unit 70.

[0054] Here, the container determination unit 68 selects the container whose three-dimensional shape is closest to the calculated three-dimensional shape of the container 12 from the three-dimensional shapes of multiple types of containers stored in the data storage unit 70, thereby determining the type of container.

[0055] In cases where the 3D information of the container calculated from the distance image is incorrect, sometimes among the multiple types of container 3D shapes stored in the data storage unit 70, there is no container whose 3D shape matches the 3D shape of container 12 calculated from the distance image. Even in this case, by selecting the container whose 3D shape is closest to the calculated 3D shape of container 12, the type of container 12 can still be properly determined.

[0056] The feature point correction unit (correction unit) 69 corrects the calculated three-dimensional information of container 12 based on the three-dimensional shape of the container with a determined type among the multiple types of container three-dimensional shapes stored in the data storage unit 70. In this way, by correcting the three-dimensional information of container 12, even if the three-dimensional information of container 12 calculated from the distance image is incorrect, the three-dimensional information of container 12 can be calculated correctly.

[0057] The three-dimensional information of container 12 can be used for various controls. For example, it can be used for the automatic operation of construction machinery 20, or for assisting in the operation of construction machinery 20. Additionally, it can be used for controls to avoid collisions between construction machinery 20 and container 12, or for automatically changing the relative position between construction machinery 20 and container 12, or for informing operators of the relative position between construction machinery 20 and container 12. Furthermore, it can be used to calculate the trajectory of the front-end auxiliary device 25c from its current position to the position where the loaded object is released (release position (e.g., soil discharge position)).

[0058] The second overlapping portion (overlapping portion) 71 generates a first overlapping image by overlapping the preprocessed image in the image processing unit 61 with the corrected three-dimensional information of the container 12. Here, as... Figure 2 As shown, an AR (Augmented Reality) marker 80 is provided as a mark at the corner of the planar portion 13 corresponding to the position of feature point F5. The second overlapping portion 71 aligns the image with the corrected container 12 in three dimensions using the AR marker 80 as a reference. Then, the aligned images are superimposed to generate a first overlapping image. Alternatively, the marker used for alignment could be a lit lamp (LED, light-emitting diode), etc.

[0059] In this way, using AR marker 80 as a reference, the three-dimensional position of the image and the corrected container 12 is aligned. This improves the accuracy of the alignment and reduces the workload required for the alignment process.

[0060] return Figure 3 The third overlapping section (overlapping section) 72 overlaps the point group data acquired by the three-dimensional coordinate acquisition section 63 with the three-dimensional information of the corrected container 12 to generate a second overlapping image. The third overlapping section 72 generates the second overlapping image by overlapping the point group data and the three-dimensional information of the corrected container 12 based on positional alignment. Alternatively, AR markers or similar symbols can be used for positional alignment.

[0061] The display control unit (display control unit) 73 displays the first overlapping image generated by the second overlapping unit 71 on the display 50. Figure 6 This is an example of the first overlapping image. Additionally, the display control unit 73 displays the second overlapping image generated by the third overlapping unit 72 on the display 50. Figure 7 This is an example of a second overlapping image.

[0062] For example, when an operator remotely operates the construction machinery 20, the operator can visually identify the depth up to the container 12 by visually recognizing the first or second overlapping image.

[0063] (Operation of the container measurement system)

[0064] Figure 8 This is a flowchart of the container measurement process. Next, refer to... Figure 8 Explain the operation of the container measurement system 30.

[0065] First, the controller 60 acquires an image from the two-dimensional information acquisition unit 42 (step S1), and preprocesses the acquired image (step S2). Next, the controller 60 infers the shape of the container 12 based on the preprocessed image using deep learning techniques or the like (step S3).

[0066] Simultaneously with steps S1 to S3, the controller 60 acquires point group data from the 3D information acquisition unit 43 (step S4), and acquires the 3D coordinates of each acquired point group data (step S5). Next, the controller 60 performs perspective projection transformation on the 3D coordinates of each point group data to acquire the 2D coordinates of each point group data (step S6).

[0067] Next, the controller 60 overlaps the two-dimensional coordinates of each point group data with the shape of the container 12 (step S7). Then, the controller 60 calculates the three-dimensional coordinates of the feature point F and the connecting line L (step S8). Finally, the controller 60 detects the three-dimensional shape of the container 12 based on the three-dimensional coordinates of the feature point F and the three-dimensional information of the connecting line L (step S9).

[0068] Next, the controller 60 checks each of the eight feature points F (F1 to F8) (step S10). Then, the controller 60 determines whether there are more than three feature points F with correct three-dimensional coordinates (step S11). If, in step S11, it is determined that there are not more than three feature points F with correct three-dimensional coordinates (S11: "No"), the controller 60 returns to step S8 and recalculates the three-dimensional coordinates of the feature points F and the connecting line L.

[0069] On the other hand, if in step S11, three or more feature points F are determined to have correct three-dimensional coordinates (step S11 is "Yes"), the controller 60 determines the type of container 12 based on the calculated three-dimensional shape of container 12 and the three-dimensional shapes of multiple types of containers stored in the data storage unit 70 (step S12). Then, the controller 60 corrects the calculated three-dimensional information of container 12 based on the three-dimensional shape of the container whose type has been determined (step S13).

[0070] Next, the controller 60 generates a first overlapping image and a second overlapping image (step S14). Specifically, the controller 60 generates a first overlapping image by overlaying the preprocessed image with the corrected 3D information of the container 12. In addition, the controller 60 generates a second overlapping image by overlaying the acquired point group data with the corrected 3D information of the container 12.

[0071] Next, the controller 60 displays either the first overlapping image or the second overlapping image on the display 50 (step S15). Then, the process ends.

[0072] As described above, according to the container measurement system 30 of this embodiment, three-dimensional information including the three-dimensional position and three-dimensional shape of the container 12 is calculated based on the acquired distance image of the container 12. Next, based on the calculated three-dimensional shape of the container 12 and the three-dimensional shapes of multiple types of containers stored in the data storage unit 70, the calculated type of the container 12 is determined. After determining the type of the container 12, the calculated three-dimensional information of the container 12 is corrected based on the three-dimensional shape of the container whose type has been determined. In this way, by correcting the three-dimensional information of the container 12, even if the three-dimensional information of the container 12 calculated from the distance image of the container 12 is incorrect, the three-dimensional information of the container 12 can be calculated correctly.

[0073] Furthermore, from the multiple types of container three-dimensional shapes stored in the data storage unit 70, the container whose three-dimensional shape is closest to the calculated three-dimensional shape of container 12 is selected, thereby determining the type of the calculated container 12. In cases where the three-dimensional information of container 12 calculated based on the distance image of container 12 is incorrect, sometimes no container among the multiple types of container three-dimensional shapes stored in the data storage unit 70 has a three-dimensional shape that matches the three-dimensional shape of container 12 calculated based on the distance image. Even in such cases, by selecting the container whose three-dimensional shape is closest to the calculated three-dimensional shape of container 12, the type of the calculated container 12 can still be appropriately determined.

[0074] Furthermore, after aligning the acquired distance image of container 12 with the corrected three-dimensional position of container 12, the three-dimensional information of the corrected container 12 is overlaid on the distance image to generate overlapping images (first overlapping image, second overlapping image). Then, the overlapping images are displayed on display 50. For example, when an operator remotely operates the construction machinery 20, the operator can visually recognize the depth up to container 12 by visually identifying the overlapping images.

[0075] Furthermore, using the mark (AR mark 80) set on container 12 as a reference, the distance image of container 12 is aligned with the corrected three-dimensional position of container 12. This improves alignment accuracy and reduces the workload required for position alignment processing.

[0076] The above description illustrates one embodiment of the present invention. However, this embodiment is merely an example of the present invention and does not specifically limit the present invention. Appropriate design changes can be made to the specific structure, etc. Furthermore, the effects and benefits described in the embodiments of the invention only represent the optimal effects and benefits produced by the present invention, and the effects and benefits of the present invention are not limited to those described in the embodiments of the present invention.

[0077] This invention provides a container measurement system. The container measurement system includes: a distance image acquisition unit, installed on an engineering machine performing a loading operation on a container, capable of acquiring a distance image of the container; a calculation unit, which calculates three-dimensional information including the three-dimensional position and three-dimensional shape of the container based on the acquired distance image; a storage unit, which stores multiple types of three-dimensional shapes of the container; a determination unit, which determines the calculated type of the container based on the three-dimensional shape of the container calculated by the calculation unit and the multiple types of three-dimensional shapes of the container stored in the storage unit; and a correction unit, which corrects the three-dimensional information of the container calculated by the calculation unit based on the type of container three-dimensional shape determined by the determination unit.

[0078] In the above structure, the determining unit can also determine the type of the calculated container by selecting the container whose three-dimensional shape is closest to the three-dimensional shape of the container calculated by the calculation unit from the three-dimensional shapes of the multiple types of containers stored in the storage unit.

[0079] The above structure may also include: a display device; an overlapping unit that, based on the alignment of the distance image of the container acquired by the distance image acquisition unit with the three-dimensional position of the container corrected by the correction unit, superimposes the corrected three-dimensional information of the container onto the distance image to generate an overlapping image; and a display control unit that displays the overlapping image on the display device.

[0080] The above structure may also include a mark set on the container in a manner that is included in the distance image, and the overlapping portion, with the mark as a reference, aligns the distance image with the corrected three-dimensional position of the container.

[0081] According to the present invention, based on the acquired distance image of the container, three-dimensional information including the three-dimensional position and three-dimensional shape of the container is calculated. Next, based on the calculated three-dimensional shape of the container and the three-dimensional shapes of multiple types of containers stored in the storage unit, the calculated type of container is determined. After determining the type of container, the calculated three-dimensional information of the container is corrected based on the three-dimensional shape of the container whose type has been determined. In this way, by correcting the three-dimensional information of the container, even if the three-dimensional information of the container calculated from the distance image of the container is incorrect, the three-dimensional information of the container can still be calculated correctly.

Claims

1. A container measurement system, characterized in that... include: A distance image acquisition unit is installed in an engineering machine that performs loading operations on a container, and is used to acquire a distance image of the container; The computing unit calculates three-dimensional information, including the three-dimensional position and three-dimensional shape of the container, based on the acquired distance image. The storage unit stores the three-dimensional shapes of the containers of various types; The determining unit determines the type of container based on the three-dimensional shape of the container calculated by the calculation unit and the three-dimensional shapes of multiple types of containers stored in the storage unit; The correction unit corrects the three-dimensional information of the container calculated by the calculation unit based on the three-dimensional shape of the container of the type determined by the determination unit. Display device; The overlapping part generates an overlapping image by aligning the distance image of the container obtained by the distance image acquisition unit with the three-dimensional position of the container corrected by the correction unit. as well as, The display control unit causes the overlapping image to be displayed on the display device.

2. The container measurement system according to claim 1, characterized in that: The determining unit determines the type of container by selecting, from the multiple types of container three-dimensional shapes stored in the storage unit, the container whose three-dimensional shape is closest to the three-dimensional shape of the container calculated by the calculation unit.

3. The container measurement system according to claim 1 or 2, characterized in that... Also includes: The markers set in the container in a manner that are included in the distance image. The overlapping portion, using the mark as a reference, aligns the distance image with the corrected three-dimensional position of the container.

Images