Extraction device and extraction method

Abstract

An extraction device and an extraction method according to the present disclosure: extract, for each frame, a cluster of feature points of a preset first object from point group data, which is a set of points having three-dimensional information; identify, for each frame, a position at which the first object exists using the extracted cluster; remove, for each frame, points at the position at which the first object exists; and extract a second object different from the first object using point group data after the removal.

Description

Extraction apparatus and extraction method

[0001] This disclosure relates to a technique for extracting a specific object from point cloud data, which is a collection of points having three-dimensional information.

[0002] A technology has been developed to create a 3D model of an outdoor structure from point cloud data acquired by a fixed 3D laser scanner (see, for example, Patent Document 1). In Patent Document 1, locations where multiple circularly arranged point clouds exist in the height direction are extracted, and cylindrical objects are extracted from the extracted point cloud information.

[0003] International Publication No. 2024 / 023900

[0004] Patent Document 1 had a problem in that when multiple circular shapes were included in the xy plane, the specific object to be extracted could not be properly extracted. Therefore, the present disclosure aims to enable the extraction of a specific object to be extracted even when multiple circular shapes are included in the xy plane.

[0005] The extraction device of this disclosure performs the extraction method of this disclosure. In the extraction method of this disclosure, the extraction device extracts a cluster of feature points of a pre-set first object from point cloud data, which is a collection of points having three-dimensional information, for each frame, identifies the location where the first object exists using the extracted cluster, removes the point at the location where the first object exists for each frame, and uses the point cloud data after removal to extract a second object that is different from the first object.

[0006] This disclosure can improve the extraction accuracy of the target object, the second object, by removing the point where the first object is located. Here, in this disclosure, since the removal is performed frame by frame, the first object can be identified in real time and at high speed from the point cloud data, and the target object, the second object, can be extracted with high accuracy.

[0007] The extraction device may identify the location of the first object by generating a plane connecting the cluster of feature points and a preset reference point, and remove points included in the cluster located at the location on the plane for each frame.

[0008] The extraction device may create an object model of the first object using the point cloud included in the cluster of locations where the first object exists, and then remove the points that constitute the object model of the first object, thereby removing the points where the first object exists.

[0009] Furthermore, the above disclosures can be combined as much as possible.

[0010] According to this disclosure, this technology enables the accurate extraction of target objects in real time while minimizing the need for hardware modifications.

[0011] This document shows an example of the system configuration of this disclosure. This document shows an example of the site conditions during construction. This document shows an example of the configuration of the object extraction unit and the proximity detection unit. This document shows an example of the extraction method of this disclosure. This document shows an example of attaching a marker to a crane boom. This document shows an example of the extraction method for the first object. This is an explanatory diagram showing an example of a plane indicating the location of the first object. This is an explanatory diagram showing an example of a plane indicating the location of the first object. This document shows an example of extracting clusters that intersect with the plane of the first object. This is an explanatory diagram showing an example of a plane indicating the location of the first object. This is an explanatory diagram showing an example of a plane indicating the location of the first object. This is an explanatory diagram showing an example of a plane indicating the location of the first object. This is an explanatory diagram showing an example of a plane indicating the location of the first object. This document shows an example of extracting clusters that intersect with the plane of the first object. This document shows an example of extracting clusters that intersect with the plane of the first object.

[0012] Embodiments of this disclosure will be described in detail below with reference to the drawings. However, this disclosure is not limited to the embodiments shown below. These examples are illustrative, and this disclosure can be implemented in various modified and improved forms based on the knowledge of those skilled in the art. In this specification and in the drawings, components with the same reference numerals refer to the same components.

[0013] (Overall Configuration) Figure 1 shows an example of the system configuration of the present disclosure. The point cloud processing system of the present disclosure comprises a point cloud processing device 91 and a point cloud measuring device 92. The point cloud measuring device 92 measures point cloud data, which is a collection of points having three-dimensional information. The point cloud processing device 91 acquires the point cloud data measured by the point cloud measuring device 92 in real time, frame by frame, and uses it to perform arbitrary calculation processing.

[0014] The point cloud measurement device 92 can be any possible device. For example, a 3D LiDAR can be used to measure the distance and direction to an object by irradiating it with laser light, measuring the time it takes for the laser light to hit the object and reflect back. By using a 3D LiDAR in the point cloud measurement device 92, it is possible to measure point cloud data that indicates the three-dimensional position of the reflected laser light. Note that the point cloud measurement device 92 is not limited to LiDAR; it can also be a stereo camera or data generated from images.

[0015] The point cloud processing device 91 includes an object extraction unit 10 and a proximity detection unit 20. The point cloud processing device 91 reads the input 3D point cloud data in real time and inputs it to the object extraction unit 10 every frame. The object extraction unit 10 extracts objects from the 3D point cloud data. The proximity detection unit 20 determines that the object extracted by the object extraction unit 10 has come close to an obstacle.

[0016] (Problems in the prior art) Figure 2 shows an example of a construction site during construction. During construction, a crane is used to move the utility pole 81 to be constructed. At this time, the utility pole 81 is suspended from the tip of the crane boom 83 by a wire 82. When this state is measured with a point cloud measuring device 92, two columnar bodies of similar size are found. For example, if the point cloud divided along the Z axis is divided into division regions 71 and circles are detected using only the X and Y components, both the crane boom 83 and the utility pole 81 are detected in each division region 71. Furthermore, the arrangement of the point cloud constituting the crane boom 83 is similar to that of the utility pole 81 to be constructed. As a result, it is difficult to distinguish them based on their shape or position, and since both circles are moving, there is a problem in distinguishing between the crane boom 83 and the utility pole 81.

[0017] (First Embodiment) The object extraction unit 10 functions as the "extraction device" of this disclosure. Specifically, when there are structures that could be confused with the utility pole 81 to be constructed, such as a crane boom 83, the object extraction unit 10 excludes the point cloud data of such structures. Hereinafter, as an example of this disclosure, an example will be described in which the point cloud data to be excluded, which is the "pre-set first object," is the point cloud data of the crane boom 83, and the "second object, which is different from the first object," is the utility pole 81 to be constructed.

[0018] Figure 3 shows an example configuration of the object extraction unit 10 and the proximity detection unit 20. The object extraction unit 10 includes a real-time point cloud reading function 11, a feature point extraction function 12, a first object extraction function 13, a point cloud removal function 14, a second object extraction function 15, and an object model creation function 16. The proximity detection unit 20 includes a proximity detection function 21 and an alarm activation function 22.

[0019] Figure 4 shows an example of the extraction method of this disclosure. The extraction method of this embodiment comprises steps S11 to S16, steps S21 and S22.

[0020] (Procedure S11) The real-time point cloud reading function 11 acquires point cloud data in real time, frame by frame. For example, the real-time point cloud reading function 11 reads and saves point cloud data measured by the point cloud measuring device 92 in real time from the point cloud measuring device 92. At this time, the point cloud data for each frame is saved linked to the time. The object extraction unit 10 and the proximity detection unit 20 perform the following processing for each frame.

[0021] (Procedure S12) The feature point extraction function 12 extracts clusters of feature points of the crane boom 83 from the point cloud data for each frame. The feature point extraction function 12 may also perform clustering of the point cloud data and extract clusters of predetermined feature points from among the clusters.

[0022] (Procedure S13) The first object extraction function 13 uses the cluster of feature points extracted in procedure S12 to identify the location of the crane boom 83 for each frame.

[0023] (Procedure S14) The point cloud removal function 14 removes the points at the locations where the crane boom 83 exists, extracted in procedure S13, frame by frame. At this time, the point cloud removal function 14 may also remove the points at the locations where the crane boom 83 exists by creating an object model of the crane boom 83 using the point cloud included in the cluster of locations where the crane boom 83 exists, and then removing the points that constitute the object model of the crane boom 83.

[0024] In this embodiment, step S13 shows an example of identifying the location of the crane boom 83 by generating a plane connecting clusters of feature points and pre-set reference points. In this configuration, in step S14, points included in the clusters located at the positions on the plane can be removed frame by frame.

[0025] (Procedure S15) The second object extraction function 15 extracts the utility poles 81 to be constructed frame by frame using the point cloud obtained after executing procedure S14. Extraction of the utility poles 81 to be constructed can be achieved, for example, by extracting the point cloud of cylindrical objects. In this embodiment, since the point cloud of the crane boom 83 in Figure 2 is removed in the extraction of the point cloud of cylindrical objects, the point cloud of the utility poles 81 to be constructed can be extracted with high accuracy.

[0026] In addition, the extraction of utility poles 81 in step S15 may be performed frame by frame, or multiple frames may be used to extract utility poles 81. By using multiple frames, it becomes possible to extract utility poles 81 based on their movement.

[0027] (Procedure S16) The object model creation function 16 creates a three-dimensional model using the point cloud of the utility pole 81 to be constructed, which was extracted in procedure S15. This makes it possible to create an object model of the utility pole 81 (second object) to be constructed.

[0028] (Procedure S21) The proximity detection function 21 calculates the distance between the object model created in procedure S16 and the obstacle and detects that the object model is in close proximity to the obstacle. Here, the position of the obstacle can be determined using any known method, such as pre-setting the 3D position or model of the obstacle.

[0029] (Procedure S22) The alarm activation function 22 activates an alarm depending on the conditions. Any known method can be used for this procedure as well.

[0030] (Point cloud extraction of the crane boom 83 in steps S12 and S13) In this embodiment, as shown in Figure 5, in order to extract the point cloud of the crane boom 83, a marker 84A is attached to the crane boom 83, and the point cloud data is measured using the point cloud measuring device 92 in that state. Here, the marker 84A can employ any method that can detect the crane boom 83, for example, reflective tape and reflective paint with high reflectivity can be exemplified. In this form, the feature point extraction function 12 extracts point clouds with high reflectivity from the point cloud data, and extracts the point cloud of the marker 84A from the extracted point clouds.

[0031] Here, the closer the point is to the point cloud measurement device 92, the greater the reflection intensity of the point cloud data. Therefore, the feature point extraction function 12 extracts point clouds where the reflection intensity is high relative to the distance, rather than point clouds where the reflection intensity is high relative to the distance. For example, the feature point extraction function 12 divides the reflection intensity into multiple stages (e.g., 256 stages) and identifies the feature point that is furthest from a reference coordinate that is above a predetermined threshold. The reference coordinate can be the position of the point cloud measurement device 92, which is the origin of the point cloud data.

[0032] In this embodiment, the reference point is shown as the origin, but the reference point is not limited to the origin; it can be any point as long as its relative coordinates within the measured point cloud data are known. For example, markers 84A and 84B may be attached to both ends of the crane boom 83, and marker 84B may be used as the reference point. In this case, the feature point extraction function 12 only needs to extract the two points that are furthest apart from the object with high reflectivity. Furthermore, the feature point extraction function 12 may extract the point cloud of marker 84A based not only on reflectivity, but also on at least one of the reflectivity, size, and shape of marker 84A.

[0033] Figure 6 shows an example of a method for extracting the first object. In step S12, the feature point extraction function 12 executes steps S111 and S112. Then, in step S13, the first object extraction function 13 executes steps S113, S114A, S115A, S116 and S117, or steps S113, S114B, S115B, S116 and S117.

[0034] (Procedure S111) The feature point extraction function 12 creates clusters based on the three-dimensional coordinates of the point cloud data. In this embodiment, in addition to the clusters of markers 84A and 84B shown in Figure 5, a cluster of the crane boom 83 is created.

[0035] (Procedure S112) The feature point extraction function 12 identifies clusters containing point clouds with high reflectance. In this embodiment, the reflectance of the point clouds of markers 84A and 84B is higher than that of the crane boom 83. Therefore, the feature point extraction function 12 identifies clusters of markers 84A and 84B.

[0036] (Procedure S113) The first object extraction function 13 determines whether or not to use the point cloud measurement device 92 as a reference point. If the point cloud measurement device 92 is installed near the crane boom 83, the first object extraction function 13 determines to use the point cloud measurement device 92 as a reference point and proceeds to procedure S114A. If the point cloud measurement device 92 is installed at a different location from the crane, the first object extraction function 13 determines not to use the point cloud measurement device 92 as a reference point and proceeds to procedure S114B. Here, the first object extraction function 13 may also automatically estimate where it is installed by comparing the measured data with pre-set conditions. Alternatively, the determination may be made by a person and set in the first object extraction function 13.

[0037] (Procedure S114A) The first object extraction function 13 selects the cluster containing the point cloud with the highest reflectivity at the furthest point. As a result, cluster C of marker 84A is selected, as shown in Figure 7. Ａ The following is selected. (Procedure S115A) The first object extraction function 13 is selected as shown in Figure 7, at the reference point, i.e., the position P of the point cloud measuring device 92. ９２ and cluster C Ａ The plane P connecting these points ８３ Create a plane P.８３ , for example, cluster C Ａ any two points from and the position P ９２ a plane connecting can be exemplified.

[0038] (Procedure S114B) The first object extraction function 13 selects two clusters that are the combination with the longest distance from among clusters including point groups having high reflection hardness. As a result, as shown in FIG. 8, the cluster C of the marker 84A Ａ and the cluster C of the marker 84B Ｂ is selected. (Procedure S115B) The first object extraction function 13, as shown in FIG. 8, two clusters C Ａ and C Ｂ connecting the plane P ８３ is created. The plane P ８３ , for example, cluster C Ａ any two points from and cluster C Ｂ a plane connecting any two points from can be exemplified.

[0039] (Procedure S116) By executing procedures S114A and S115A, or procedures S114B and S115B, the plane P along the crane boom 83 ８３ can be created. The first object extraction function 13, as shown in FIG. 9, the plane P ８３ extracts clusters C3, C4, C5 intersecting with.

[0040] (Procedure S117) The first object extraction function 13 extracts point groups included in clusters C3, C4, C5. Thus, the point group of the crane boom 83 can be extracted.

[0041] In step S117, the first object extraction function 13 may create a three-dimensional model from the point cloud of the crane boom 83. In this embodiment, in step S21, the proximity detection function 21 uses the three-dimensional model created in step S117 as an obstacle model, calculates the distance between the object model created in step S16 and the obstacle model, and can also detect that the object model has come close to an obstacle.

[0042] As described above, in this embodiment, the point cloud processing system can accurately extract the point cloud of the utility pole 81 to be constructed because the point cloud processing device 91 removes the point cloud of the crane boom 83. In this embodiment, the point cloud of the crane boom 83 can be removed simply by placing a marker 81A on the end of the crane boom 83, and the process for removing the point cloud of the crane boom 83 is also simple. For this reason, the point cloud processing system in this embodiment can extract the point cloud of the utility pole 81 to be constructed in real time with high accuracy at low cost.

[0043] (Second Embodiment) In this embodiment, an alternative form of procedure S13 will be described. There are two forms of procedure S13, S115A and S115B, but in this embodiment, the example of procedure S115B will be described. In addition, in this embodiment, in order to reduce the computational load, instead of a spatial region of any shape that constitutes the cluster, a box is used which is a section of the cluster cut along the spatial axis.

[0044] (Procedure S115B) The first object extraction function 13 is used as shown in Figure 10, for cluster C Ａ Box B Ａ Create cluster C. Ｂ The same applies to box B. Ｂ Create.

[0045] The first object extraction function 13 is as shown in Figure 10, box B Ａ Center of gravity P ＡＣ Box B, with the same Z coordinate Ａ Points P included in each side Ａ -1, P Ａ -2, P Ａ -3, P Ａ Select -4. Selected point P Ａ -1, P Ａ -2, P Ａ -3, P Ａ -4 is referred to as "a candidate point that constitutes the plane." The first object extraction function 13 is box B Ｂ Regarding the center of gravity P, ＢＣ Box B, with the same Z coordinate Ｂ Points P included in each side Ｂ -1, P Ｂ -2, PＢ -3, P Ｂ Select -4.

[0046] The first object extraction function 13 is used for each box B Ａ and B Ｂ From the four points selected, two points are chosen that have the same X coordinate or the same Y coordinate. For example, as shown in Figure 11, point P Ａ -1 and point P Ａ The x-coordinate of -3 is the same, and point P Ｂ -1 and point P Ｂ Point P such that the x-coordinate of -3 is the same Ａ -1, P Ａ -3, point P Ｂ -1, P Ｂ Select -3.

[0047] Next, the first object extraction function 13 targets point P Ａ -1 and P Ａ Connect point -3 with a straight line, and point P Ｂ -1 and P Ｂ Connect point -3 with a straight line. In this disclosure, this straight line is referred to as a "predetermined side". This connects point P Ａ -1 and P Ａ -3 is a predetermined side L Ａ , point P Ｂ -1 and P Ｂ -3 is a predetermined side L Ｂ This is generated.

[0048] Next, the first object extraction function 13 uses the "predetermined edges" and the "candidate points that constitute the plane" of the opposing box to extract point P as follows: Ａ -1, P Ａ -3, P Ｂ -1, P Ｂ - Select a set of 3. For example, as shown in Figure 11, • Predetermined side L Ａ It is perpendicular to point P. Ｂ Find the edge L-1 that passes through -1. • The previously determined edge L Ａ It is perpendicular to point P. Ｂ Find the side L-3 that passes through -3. • Find point P such that the difference between side L-1 and side L-3 is minimized. Ａ -1, P Ａ -3, P Ｂ -1, PＢ select the set of -3.

[0049] Next, as shown in FIG. 12, the first object extraction function 13 obtains a point P Ａ -1, P Ａ -3, P Ｂ -1, P Ｂ -3 to generate a plane P ８３ . Here, the point P Ａ -3 and the point P Ｂ -1, the point P Ａ -1 and the point P Ｂ -3 will result in intersecting lines that cannot form a plane if connected. Therefore, the first object extraction function 13 generates the plane P ８３ such that the sides L-1 and L-3 are included therein.

[0050] Note that, as shown in FIG. 13, the first object extraction function 13 may select point P Ａ -1 and point P Ａ -3 having the same x-coordinate, and point P Ｂ -2 and point P Ｂ -4 having the same y-coordinate, that is, select Ａ P-1, P Ａ -3, point P Ｂ -2, P Ｂ -4. In this case, the first object extraction function 13 selects the set of point P Ａ -1, P Ａ -3, point P Ｂ -2, P Ｂ -4 as follows. - Obtain a side L-2 that is perpendicular to a predetermined side L Ａ and passes through point P Ｂ -2. - Obtain a side L-4 that is perpendicular to the predetermined side L Ａ and passes through point P Ｂ -4. - Select the set of point P-1, P Ａ -1, P Ａ -3, point P Ｂ -2, P Ｂ -4 such that the difference between the side L-2 and the side L-4 is minimized.

[0051] (Procedure S116) The first object extraction function 13 uses a plane equation that expresses the plane P ８３ to represent the plane P ８３Extract clusters that intersect with the curve. Any planar equation including sides L-1 and L-3 can be used.

[0052] For example, point P Ａ Starting from -1, P Ａ V is a vector that passes through -3. ｘ = (a １ , a ２ , a ３ ), point P Ａ Starting from -1, P Ｂ V is a vector that passes through -1. ｙ = (b １ , b ２ , b ３ When V ｘ and V ｙ One vector perpendicular to both of them is given by the following equation: (Mathematics 1) V ｘ ×V ｙ = (a ２ b ３ -b ２ a ３ , a ３ b １ -b ３ a １ , a １ b ２ -b １ a ２ ) (1)

[0053] At this time, point P Ａ If -1 = (A1, A2, A3), then the plane equation is shown below. (Mathematics 2) (a ２ b ３ -b ２ a ３ )(x-A1) +(a ３ b １ -b ３ a １ )(x-A2) +(a １ b ２ -b １ a ２ )(x-A3)=0 (2)

[0054] The first object extraction function 13 extracts each side L parallel to the z-axis direction of the planar equation and box B3. ３ -1, L ３ -2, L ３ -3, L ３Determine if -4 intersects. For example, edge L ３ The z-coordinate is obtained by applying the x and y coordinates of -1 to the plane equation, and the obtained z-coordinate is the side L. ３ Determine whether the z-axis is within the range of -1. The obtained z-coordinate is on edge L. ３ When the z-axis is not included in the range of -1, side L ３ -1, L ３ -2, L ３ -3, L ３ As shown in -4, plane P ８３ They do not intersect. In this case, the first object extraction function 13 does not extract the point cloud within box B3.

[0055] The first object extraction function 13 performs the same processing on each box created in step S111 as it does on box B3. L parallel to the z-axis direction of box B4 ４ By applying the x and y coordinates of -1 to the plane equation, we can find the z coordinate, and the obtained z coordinate is on side L. ４ It falls within the z-axis range of -1. In this case, the first object extraction function 13 extracts the point cloud within box B4.

[0056] plane P ８３ The position of the crane boom 83 is represented by the plane P. ８３ By extracting box B4 which intersects with the crane boom 83, the point cloud of the crane boom 83 can be removed. In this embodiment, in step S116, plane P ８３ Although a box B4 of a size equal to or smaller than that of the crane boom 83 was identified as the point cloud of the crane boom 83, this disclosure is not limited thereto. For example, the first object extraction function 13 identifies a plane P ８３ You can also extract boxes that are larger than or shifted horizontally.

[0057] (Other Embodiments) In the example shown, the point cloud data to be excluded, which is the "pre-set first object," is the point cloud data of the crane boom 83, and the "second object, which is different from the first object," is the utility pole 81 that is the target of construction. However, this disclosure is not limited to this. For example, the "second object" may be any object to be tracked, and the "first object" may be any object that is easily confused with the "second object."

[0058] Furthermore, while the "location of the first object" is represented using a plane connecting marker 84A and the reference point, it may also be a plane connecting three or more points. Moreover, the "location of the first object" is not limited to a plane, but may also be a straight line such as L-2 or L-4 shown in Figure 13.

[0059] Furthermore, the point cloud processing device 91 of this disclosure can also be implemented using a computer and a program, and the program can be recorded on a recording medium or provided via a network.

[0060] 10: Object extraction unit 11: Real-time point cloud reading function 12: Feature point extraction function 13: First object extraction function 14: Point cloud removal function 15: Second object extraction function 16: Object model creation function 20: Proximity detection unit 21: Proximity detection function 22: Alarm activation function 81: Utility pole 91: Point cloud processing unit 92: Point cloud measurement device

Claims

1. An extraction device that extracts clusters of feature points of a pre-defined first object from point cloud data, which is a collection of points having three-dimensional information, for each frame; identifies the location of the first object for each frame using the extracted clusters; removes the points at the location of the first object for each frame; and extracts a second object different from the first object using the point cloud data after removal.

2. The extraction device according to claim 1, which generates a plane connecting the cluster of feature points and a pre-set reference point to identify the location of the first object, and removes points included in the cluster located at the location on the plane for each frame.

3. An extraction device according to claim 1, comprising creating an object model of the first object using a point cloud included in the cluster of locations where the first object exists, and removing points that constitute the object model of the first object to remove points where the first object exists.

4. An extraction method comprising: extracting clusters of feature points of a pre-defined first object from point cloud data, which is a collection of points having three-dimensional information, for each frame; identifying the location of the first object for each frame using the extracted clusters; removing the points at the location of the first object for each frame; and extracting a second object different from the first object using the point cloud data after removal.

Images