Point cloud processing device, point cloud processing method and program

Abstract

To detect a point cloud indicating a difference between a background point cloud belonging to a background region of a real space and a point cloud representing the real space, with higher accuracy.SOLUTION: A point cloud processing apparatus includes: a point cloud acquisition unit which acquires an input point cloud, which is a three-dimensional point cloud obtained by a sensor measuring a real space including an object; a distribution model creation unit which performs statistical processing on a reference background point cloud, which is a three-dimensional point cloud representing a background region of the object, to create a distribution model representing, in a three-dimensional shape, a status of distribution of the reference background point cloud; and a differential point cloud acquisition unit which acquires a differential point cloud indicating a difference between the input point cloud and the reference background point cloud, based on the distribution model.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a point cloud processing device, a program, and a point cloud processing method. [Background technology]

[0002] In recent years, research has been conducted into technologies for acquiring three-dimensional measurement data of real space using sensors and analyzing the information of the real space. For example, a sensor device called LiDAR (Laser Imaging Detection and Ranging) is known as a sensor for acquiring three-dimensional measurement data of real space.

[0003] Regarding the analysis of information on real space, for example, Patent Document 1 discloses a technology for acquiring a point cloud representing real space using LiDAR and extracting a background point cloud belonging to a background area such as a floor or wall. It also discloses a technology for detecting an object based on the difference between the extracted background point cloud and the point cloud representing real space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-219248 A Summary of the Invention [Problem to be solved by the invention]

[0005] One method for detecting an object from the difference between a background point cloud and a point cloud representing real space is to divide each real space, including the background point cloud and the point cloud representing real space, into three-dimensional regular grids (voxels) and compare them. Specifically, it is conceivable to identify voxels containing background point clouds among voxels containing point clouds representing real space, delete the point clouds contained in the identified voxels from the point cloud representing real space, and detect the remaining point cloud as a point cloud representing an object. However, in a method of detecting a point cloud representing an object by comparing voxels, if the background point cloud and the point cloud representing the object are close to each other, the point cloud representing the object will be included in the voxel containing the background point cloud, and the point cloud representing the object will be deleted, making it impossible to detect part or all of the object.

[0006] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a new and improved point cloud processing device, point cloud processing method, and program that are capable of detecting with higher accuracy a point cloud that indicates the difference between a background point cloud belonging to the background area of ​​real space and a point cloud that represents real space. [Means for solving the problem]

[0007] In order to solve the above problem, according to one aspect of the present invention, there is provided a point cloud processing device comprising: a point cloud acquisition unit that acquires an input point cloud, which is a three-dimensional point cloud obtained by measuring a real space including an object with a sensor; a distribution model creation unit that performs statistical processing on a reference background point cloud, which is a three-dimensional point cloud that represents a background region of the object, and creates a distribution model that represents the distribution situation of the reference background point cloud in a three-dimensional shape; and a difference point cloud acquisition unit that acquires a difference point cloud that indicates the difference between the input point cloud and the reference background point cloud based on the distribution model.

[0008] The distribution model creation unit may create the distribution model by dividing a real space including the reference background point cloud into three-dimensional solids and performing statistical processing on a point cloud included in each solid.

[0009] The distribution model creation unit may perform principal component analysis on a point cloud included in each solid to identify three mutually orthogonal principal component directions, identify the distances in each principal component direction between points that are farthest apart in the point cloud included in the solid, and create a rectangular parallelepiped containing all of the point clouds in the solid as the distribution model, with the identified three distances corresponding to the lengths of three sides.

[0012] The shape of the expansion range may be similar to the three-dimensional shape of the distribution model, and the center of gravity of the expansion range may coincide with the center of gravity of the distribution model.

[0013] The point cloud processing device may further include a reference background point cloud creation unit that creates the reference background point cloud from the point cloud data acquired from the sensor.

[0014] The point cloud processing device may further include an object detection unit that detects a point cloud corresponding to the same object from the difference point cloud.

[0015] The point cloud processing device may further include a display unit that displays the difference point cloud or the point cloud corresponding to the identical object.

[0016] In order to solve the above problem, the method includes acquiring an input point cloud, which is a three-dimensional point cloud obtained by measuring a real space including an object with a sensor, performing statistical processing on a reference background point cloud, which is a three-dimensional point cloud representing a background region of the object, to create a distribution model that represents the distribution situation of the reference background point cloud in a three-dimensional shape, and acquiring a difference point cloud that represents a difference between the input point cloud and the reference background point cloud based on the distribution model, acquiring the difference point cloud includes performing a calculation process on three-dimensional coordinates of the distribution model, acquiring an enlarged range by enlarging the distribution model, and acquiring, as the difference point cloud, a point cloud that is not included in the enlarged range among the input point clouds. A computer-implemented method for processing point clouds is provided.

[0017] In order to solve the above problem, a computer is made to function as a point cloud acquisition unit that acquires an input point cloud, which is a three-dimensional point cloud obtained by measuring a real space including an object with a sensor; a distribution model creation unit that performs statistical processing on a reference background point cloud, which is a three-dimensional point cloud that represents a background region of the object, and creates a distribution model that represents the distribution situation of the reference background point cloud in a three-dimensional shape; and a difference point cloud acquisition unit that acquires a difference point cloud that indicates a difference between the input point cloud and the reference background point cloud based on the distribution model, The difference point cloud acquisition unit performs calculation processing on the three-dimensional coordinates of the distribution model, acquires an enlarged range by enlarging the distribution model, and acquires, as the difference point cloud, a point cloud that is not included within the enlarged range from among the input point clouds. , the program is provided. [Effects of the Invention]

[0018] According to the present invention as described above, it is possible to detect with higher accuracy a point group that indicates the difference between a background point group that belongs to the background region of real space and a point group that represents real space. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is an explanatory diagram illustrating a point cloud processing system according to an embodiment of the present invention. [Figure 2] 1 is an explanatory diagram showing a specific example of a point cloud in real space acquired by a sensor 100. FIG. [Figure 3] FIG. 10 is an explanatory diagram showing a method for obtaining a difference point cloud according to a comparative example. [Figure 4] FIG. 1 is an explanatory diagram showing the configuration of a point cloud processing apparatus 200 according to an embodiment of the present invention. [Figure 5] 4 is an explanatory diagram showing a reference background point group P1 included in a voxel B1 shown in FIG. 3. FIG. [Figure 6] FIG. 6 is an explanatory diagram showing a principal component analysis of the reference background point cloud shown in FIG. 5. [Figure 7] FIG. 6 is an explanatory diagram showing a method for creating a rectangular parallelepiped distribution model from the reference background point cloud shown in FIG. 5. [Figure 8] 3 is an explanatory diagram showing a distribution model of a reference background point cloud for the entire real space shown in FIG. 2. FIG. [Figure 9] 9 is an explanatory diagram of a method for comparing the distribution model M shown in FIG. 8 with the input point cloud P to obtain a difference point cloud. [Figure 10] FIG. 10 is an explanatory diagram showing a specific example in which the distribution model M is enlarged and a difference point group is acquired. [Figure 11] 1 is a flowchart showing the operation of the point cloud processing apparatus 200 according to the embodiment of the present invention. [Figure 12] FIG. 10 is an explanatory diagram showing the configuration of a point cloud processing device 200 according to a modified example. [Figure 13] 10 is a flowchart showing the operation of the point cloud processing device 200 according to a modified example. [Figure 14] FIG. 2 is a block diagram showing the hardware configuration of a point cloud processing device 200. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0021] In addition, in this specification and drawings, multiple components having substantially the same functional configuration may be distinguished by adding different letters to the same reference numeral. However, if there is no need to particularly distinguish between multiple components having substantially the same functional configuration, only the same reference numeral will be used to distinguish each of the multiple components.

[0022] <Outline of the point cloud processing system> An embodiment of the present invention relates to a point cloud processing system that detects a point cloud that indicates a difference between a point cloud representing a real space and a background point cloud that belongs to a background region of the real space by processing a point cloud obtained by measuring a real space with a sensor. Prior to a detailed description of the embodiment of the present invention, an overview of the point cloud processing system according to the embodiment of the present invention will be described.

[0023] Fig. 1 is an explanatory diagram showing a point cloud processing system according to an embodiment of the present invention. As shown in Fig. 1, the point cloud processing system according to the embodiment of the present invention includes a sensor 100 and a point cloud processing device 200. The sensor 100 and the point cloud processing device 200 may be connected by wire, wirelessly, or via a network.

[0024] (sensor) The sensor 100 is a measuring device that measures a three-dimensional environment in real space. The sensor 100 is installed at a certain point and supported by a tripod 110, for example, as shown in Fig. 1. The sensor 100 outputs a point cloud obtained by measurement to the point cloud processing device 200.

[0025] The sensor 100 is not particularly limited as long as it is a sensor that can acquire a three-dimensional point cloud, and may be, for example, a 3D-LiDAR or a depth camera.

[0026] (Point Cloud Processing Device) The point cloud processing device 200 acquires a difference point cloud indicating the difference between a background point cloud belonging to a background region of real space and a point cloud representing real space, based on the point cloud input from the sensor 100. Note that there are no particular limitations on the objects that the point cloud processing device 200 detects, and for example, the point cloud processing device 200 may detect moving objects such as people or vehicles. Here, a specific example of extraction of a difference point cloud by the point cloud processing device 200 will be described with reference to Figs. 2 and 3.

[0027] Fig. 2 is an explanatory diagram showing a specific example of a point cloud in real space acquired by sensor 100. In the real space shown in Fig. 2, there are a background object BG belonging to the background region and detection target objects O1 and O2 belonging to the detection target region. As shown in Fig. 2, an input point cloud P, which is a point cloud in real space acquired by sensor 100, is scattered in the region where the background object BG and the detection target objects O1 and O2 exist. Note that Fig. 2 merely shows a simplified representation of the input point cloud; in reality, an input point cloud consisting of more points may be obtained, or the points may be distributed in an arc shape.

[0028] Fig. 3 is an explanatory diagram showing a method for obtaining a difference point cloud according to a comparative example. In the comparative example shown in Fig. 3, each real space including a background point cloud and an input point cloud is divided into three-dimensional regular lattices (voxels), and a comparison is performed to obtain a difference point cloud. More specifically, among voxels including point clouds representing the real space, voxels including the background point cloud are identified, and the point clouds included in those voxels are deleted from the point cloud representing the real space, and the remaining point cloud is detected as a difference point cloud.

[0029] (Identifying issues) In the above-described method of detecting a point cloud representing an object by comparing voxels, if the background point cloud and the point cloud representing the object are close to each other, the point cloud representing the object will be included in the voxel containing the background point cloud, and the point cloud representing the object will be deleted, making it impossible to detect part or all of the object. In the example shown in Figure 3, point cloud Po4, which is a point cloud representing object O2 to be detected shown in Figure 2, is detected as a difference point cloud because it is located in voxel B4, which does not contain the background point cloud. On the other hand, point cloud Po3, which is a point cloud representing object O1 to be detected shown in Figure 2, is not detected as a difference point cloud because it is located in voxel B3, which contains background point cloud Pb3.

[0030] Therefore, the present inventors have taken the above circumstances into consideration and have come up with an embodiment of the present invention. A point cloud processing device 200 according to an embodiment of the present invention is capable of detecting with high accuracy the difference between a background point cloud belonging to a background region in real space and a point cloud representing real space. The configuration and operation of the point cloud processing device 200 according to the embodiment of the present invention will be sequentially described in detail below.

[0031] <Configuration of point cloud processing device> 4 is an explanatory diagram showing the configuration of a point cloud processing apparatus 200 according to an embodiment of the present invention. As shown in FIG. 4, the point cloud processing apparatus 200 according to the embodiment of the present invention includes a point cloud acquisition unit 220, a storage unit 230, a distribution model creation unit 240, a difference point cloud acquisition unit 250, a display control unit 260, and a display unit 270.

[0032] (Point cloud acquisition part) The point cloud acquisition unit 220 acquires input point cloud data, which is a point cloud in real space input from the sensor 100. There are two possible cases where the input point cloud contains a mixture of a background point cloud belonging to the background region and a detection target point cloud belonging to the detection target region without being distinguished, and there are also possible cases where the input point cloud contains only a background point cloud.

[0033] Furthermore, the point cloud acquisition unit 220 acquires a reference background point cloud and stores the reference background point cloud in the storage unit 230. The point cloud acquisition unit 220 may use the sensor 100 to measure a space in which no detection target object is present within the measurement range and only background objects belonging to a background region are present, and acquire the obtained point cloud as the reference background point cloud. Furthermore, the point cloud acquisition unit 220 may have the function of a reference background point cloud creation unit. The reference background point cloud creation unit may create the reference background point cloud by extracting a background point cloud from an input point cloud, or may perform measurements multiple times and create the reference background point cloud from the results of the multiple measurements.

[0034] (Storage part) The storage unit 230 stores the input point cloud acquired by the point cloud acquisition unit 220 , the reference background point cloud, and the distribution model created by the distribution model creation unit 240 .

[0035] (Distribution Model Creation Department) The distribution model creation unit 240 creates a distribution model that represents the distribution state of the reference background point cloud in a three-dimensional shape from the reference background point cloud stored in the storage unit 230. A specific method for creating the distribution model will be described below.

[0036] The distribution model creation unit 240 first divides the real space including the reference background point cloud into three-dimensional solids. The shape of the solids is not particularly limited, and may be, for example, voxels, a rectangular parallelepiped, or a tetrahedron. One method for dividing the real space into voxels is to define a three-dimensional Cartesian coordinate system in the space including the reference background point cloud, and divide the space between the maximum and minimum values ​​of the reference background point cloud on each axis by a fixed length. Below, we will explain the processing when the three-dimensional solids are voxels.

[0037] Next, the distribution model creation unit 240 creates a distribution model by performing statistical processing on the reference background point cloud included in each voxel that divides the real space. Figure 5 is an explanatory diagram showing the reference background point cloud P1 included in the voxel B1 shown in Figure 3. Here, with reference to Figures 6 to 8, a method of creating a rectangular parallelepiped distribution model from the reference background point cloud P1 will be described as a specific example of distribution model creation.

[0038] Fig. 6 is an explanatory diagram showing principal component analysis of the reference background point group shown in Fig. 5. The distribution model creation unit 240 performs principal component analysis to find directions with large variance of the reference background point group P1 included in voxel B1, and identifies three mutually orthogonal principal component directions PC1, PC2, and PC3.

[0039] Next, the distribution model creation unit 240 determines the distance in each principal component direction between the points that are the furthest apart in the principal component direction found above in the point group contained in voxel B1. For example, in the example shown in FIG. 6, the two points that are the furthest apart in the principal component direction PC2 in the point group contained in voxel B1 are points p21 and p22. Therefore, the distribution model creation unit 240 determines the distance D2 between these two points in the principal component direction PC2.

[0040] Fig. 7 is an explanatory diagram showing a method for creating a rectangular parallelepiped distribution model from the reference background point cloud shown in Fig. 5. The distribution model creation unit 240 creates a rectangular parallelepiped containing all point clouds within voxel B1 as the distribution model M1 of voxel B1, with the identified three distances corresponding to the lengths of the three sides. The length of one of the three sides of the distribution model M1 is the distance D2 described in the example above.

[0041] Fig. 8 is an explanatory diagram showing a distribution model of a reference background point cloud for the entire real space shown in Fig. 2. The distribution model creation unit 240 creates a distribution model for each voxel that includes a reference background point cloud among the voxels that divide the real space. Specifically, for each of voxels B1, B2, and B3, distribution models M1, M2, and M3 of the reference background point cloud included in each voxel are created.

[0042] So far, we have explained a method of creating a rectangular parallelepiped distribution model by performing principal component analysis as a specific example of distribution model creation, but the distribution model creation unit 240 may create a distribution model using other statistical processing. For example, the distribution model creation unit 240 may calculate the normal distribution of the point cloud included in each voxel and create an ellipsoid distribution model.

[0043] Here, a method for creating an ellipsoid distribution model will be described. First, the distribution model creation unit 240 defines a three-dimensional orthogonal coordinate system within each voxel and calculates the average value and variance value for each axis. Next, the distribution model creation unit 240 creates an ellipsoid as the distribution model M for the voxel, whose central position coordinate is the average value for each axis and whose three-axis diameter is the result of multiplying the square root of the variance value for each axis by a constant. Note that the proportion of the points in the voxel that are included in the ellipsoid is determined based on the constant multiplied by the variance value for each axis.

[0044] (Difference point cloud acquisition part) The difference point cloud acquisition unit 250 acquires a difference point cloud indicating the difference between the input point cloud and the reference background point cloud based on the distribution model M created by the distribution model creation unit 240. For example, the difference point cloud acquisition unit 250 acquires, as the difference point cloud, a point cloud from among the points included in the input point cloud that does not satisfy a condition regarding its positional relationship with the distribution model M. Note that the difference point cloud may be a collection of points from among the points included in the input point cloud that are not located near any points included in the reference background point cloud. A method for acquiring a difference point cloud will be described in more detail below with reference to FIG. 9.

[0045] 9 is an explanatory diagram of a method for comparing the distribution model M shown in FIG. 8 with the input point cloud P to acquire a difference point cloud. The difference point cloud acquisition unit 250 compares the distribution models M1, M2, and M3 created by the distribution model creation unit 240 with the input point cloud P shown in FIG. 2. The difference point cloud acquisition unit 250 deletes point clouds from the input point cloud P that satisfy the condition of being located within the distribution models M1, M2, and M3, and acquires point clouds that do not satisfy this condition as difference point clouds. Here, the point cloud Po4, which is a point cloud representing the detection target object O2 shown in FIG. 2, and the point cloud Po3, which is a point cloud representing the detection target object O1, are not included in the distribution models, and are therefore detected as difference point clouds.

[0046] Although the above describes an example in which the input point cloud P is compared with distribution models M1, M2, and M3, the target of comparison with the input point cloud P may be a range obtained by changing the range indicated by distribution model M through calculation processing. For example, the difference point cloud acquisition unit 250 may perform calculation processing on the three-dimensional coordinates of the distribution model to obtain an enlarged or reduced range of distribution model M, and acquire, as the difference point cloud, a point cloud from the input point cloud P that is not included in the enlarged or reduced range. Furthermore, as the enlarged range obtained by enlarging distribution model M, a range that is similar to the three-dimensional shape of distribution model M and whose center of gravity coincides with that of the distribution model may be acquired. Here, a specific example in which an enlarged range that is similar to distribution model M and whose center of gravity coincides with that of distribution model M is acquired, and a point cloud from the input point cloud P that is not included in the enlarged range is acquired as the difference point cloud, will be described with reference to FIG. 10 .

[0047] FIG. 10 is an explanatory diagram showing a specific example in which a distribution model M is enlarged and a difference point cloud is acquired. The distribution model M10 is a rectangular parallelepiped distribution model, and FIG. 10 shows a plan view of the distribution model M10 as seen from the perpendicular direction of one face. When the distribution model M10 is compared with the input point cloud P without being enlarged, point clouds Pb10 and Po10 are acquired as difference point clouds. Here, there is an error between the position of the acquired reference background point cloud and the position of the background point cloud included in the input point cloud, and even if point cloud Pb10 is actually part of the background point cloud included in the input point cloud, a difference point cloud including point cloud Pb10 is acquired.

[0048] Therefore, the difference point cloud acquisition unit 250 creates, as the enlarged range EM10, a rectangular parallelepiped whose center of gravity coincides with the center of gravity position C of the distribution model M10 and whose side lengths are calculated by multiplying the side lengths of the distribution model M10 by a certain coefficient. Specifically, the difference point cloud acquisition unit 250 multiplies each of the side lengths L1 and L2 of the distribution model M10 by the same coefficient, and calculates the multiplication results as the side lengths L1e and L2e of the enlarged range EM10. The difference point cloud acquisition unit 250 also calculates the length of the other side of the enlarged range EM10 (not shown) in the same way as the side lengths L1e and L2e of the enlarged range EM10. In other words, the difference point cloud acquisition unit 250 multiplies each of the side lengths L1 and L2 of the distribution model M10 by the coefficient obtained by multiplying each of the side lengths L1 and L2 of the distribution model M10 by the length of the other side of the distribution model M10 (not shown), and calculates the multiplication result as the length of the other side of the enlarged range EM10 (not shown). Using the method described above, the difference point cloud acquisition unit 250 acquired, as the enlarged range EM10, a rectangular parallelepiped that is similar to the distribution model M10 and whose center of gravity coincides with that of the distribution model.

[0049] When the obtained enlarged range EM10 is compared with the input point cloud P, the point cloud Pb10 is not acquired as the difference point cloud, and only the point cloud Po10 is acquired as the difference point cloud. The method of acquiring the difference point cloud based on the enlarged range EM of the distribution model M is effective as a method of acquiring the difference point cloud when there is a possibility of an error between the position of the reference background point cloud and the position of the background point cloud included in the input point cloud.

[0050] 10, a method for creating a distribution model that matches the surface of a background object from a reference background point cloud, and acquiring a difference point cloud using the distribution model or an expanded range of the distribution model as a boundary, has been described as an example of acquiring a difference point cloud. According to this method, when a sensor that has the characteristic of being able to acquire a point cloud of an object surface, such as 3D-Lidar, is used as sensor 100, a distribution model that matches the surface of the background object is created, making it possible to appropriately acquire a difference point cloud from an input point cloud.

[0051] (Display control unit) The display control unit 260 generates a detection result screen showing the difference point cloud acquired by the difference point cloud acquisition unit 250. The method of generating the detection result screen includes arranging the difference point cloud, arranging the reference background point cloud, arranging the input point cloud, distinguishing the difference point cloud, the reference background point cloud, and the input point cloud from each other using different colors, and drawing the distribution model or the enlargement range with a line.

[0052] (Display) The display unit 270 displays the detection result screen generated by the display control unit 260, thereby visualizing the difference point cloud.

[0053] <Operation of point cloud processing device> The configuration of the point cloud processing apparatus 200 according to the embodiment of the present invention has been described above. Next, the operation of the point cloud processing apparatus 200 according to the embodiment of the present invention will be described with reference to FIG.

[0054] 11 is a flowchart showing the operation of the point cloud processing device 200 according to an embodiment of the present invention. As shown in FIG. 11, first, the point cloud acquisition unit 220 acquires a point cloud in real space input from the sensor 100 as an input point cloud P. The storage unit 230 stores the input point cloud P acquired by the point cloud acquisition unit (S304). The point cloud acquisition unit 220 also acquires a reference background point cloud and stores it in the storage unit 230 (S308).

[0055] Then, the distribution model creation unit 240 divides the real space including the reference background point cloud stored in the storage unit 230 into voxels (S312). The distribution model creation unit 240 performs statistical processing on the reference background point cloud included in the solid for each voxel dividing the real space, and creates a distribution model M that represents the distribution situation of the reference background point cloud as a three-dimensional shape (S316).

[0056] Next, the difference point cloud acquisition unit 250 acquires a difference point cloud indicating the difference between the background point cloud belonging to the background area of ​​the real space and the point cloud representing the real space from the positional relationship between the distribution model M created by the distribution model creation unit 240 and the input point cloud P (S320).

[0057] Thereafter, the display unit 270 displays the difference point cloud acquired by the difference point cloud acquisition unit 250 (S324).

[0058] <Action and effect> According to the embodiment of the present invention described above, a variety of effects can be obtained.

[0059] For example, the difference point cloud acquisition unit 250 acquires a difference point cloud by comparing the input point cloud with the distribution model created by the distribution model creation unit 240. With this configuration, the distribution model creation unit 240 creates a distribution model according to the shape of the background object, so the difference point cloud acquisition unit 250 can acquire a point cloud that indicates the detection target object as the difference point cloud with high accuracy. In other words, it is possible to reduce missed detections of the detection target object compared to when a distribution model is not created.

[0060] As a comparative example, a method is also conceivable in which, without creating a distribution model, a space including a reference background point cloud and a space including an input point cloud are divided into voxels, and the difference point cloud is acquired by comparing each voxel. In the comparative example, finer voxels can be considered to prevent missed detection of the detection target object, but finer voxels increase the amount of processing required by the computer. Furthermore, in the comparative example, in order to acquire a difference point cloud with high accuracy, it is necessary to appropriately adjust the size of the voxels according to the shape of the background object to prevent both missed detection and false detection. In contrast, in an embodiment of the present invention, the distribution model creation unit 240 creates a distribution model according to the shape of the background object, so that the difference point cloud acquisition unit 250 can accurately acquire a point cloud representing the detection target object as a difference point cloud without finely adjusting the size of the voxels.

[0061] Furthermore, the distribution model creation unit 240 divides the space including the reference background point cloud into three-dimensional solids and creates a distribution model for each solid including the reference background point cloud. With this configuration, a distribution model that more closely matches the shape of the background object is created compared to when the distribution model creation unit 240 creates a distribution without dividing the space into solids. Therefore, the difference point cloud acquisition unit 250 can acquire a point cloud that indicates the detection target object as a difference point cloud with higher accuracy.

[0062] Furthermore, the distribution model creation unit 240 performs principal component analysis on the point clouds contained in each solid body including the reference background point cloud, and creates a rectangular parallelepiped distribution model containing all the point clouds in the solid body. Meanwhile, a method for creating a rectangular parallelepiped distribution model may involve performing statistical processing other than principal component analysis on the point clouds within the solid body. Specifically, the distribution model creation unit 240 first divides real space into voxels and establishes a three-dimensional Cartesian coordinate system with axes parallel to three sides of the voxels. Next, the distribution model creation unit 240 calculates the minimum and maximum values ​​on each axis of the point cloud contained in the voxels. The distribution model creation unit 240 then identifies eight points whose coordinates are the calculated minimum or maximum values ​​on each axis, and creates a rectangular parallelepiped distribution model having the identified points as vertices. Compared to a rectangular parallelepiped distribution model created using the above-mentioned creation method, a rectangular parallelepiped distribution model created using principal component analysis is more likely to produce a distribution model that shows a narrower range. The difference point cloud acquisition unit 250 acquires the difference point cloud based on a distribution model that indicates a narrower range, thereby reducing the risk of missing a detection of the difference point cloud.

[0063] <Modification> The above is a description of an embodiment of the present invention. Below, a modification of the embodiment of the present invention will be described. This modification may be applied in place of the configuration described in the embodiment of the present invention, or may be applied in addition to the configuration described in the embodiment of the present invention.

[0064] In the above, an example has been shown in which the difference point cloud acquisition unit 250 compares the input point cloud P with the distribution model M to acquire the difference point cloud, and the display unit 270 displays the difference point cloud acquired by the difference point cloud acquisition unit 250. This modified example is a modified example in which object detection is performed from the difference point cloud. This modified example will be described in more detail with reference to FIG. 12 .

[0065] Fig. 12 is an explanatory diagram showing the configuration of a modified point cloud processing device 200. As shown in Fig. 12, the modified point cloud processing device 200 has an object detection unit 280 in addition to the configuration of the point cloud processing device 200 according to the embodiment of the present invention.

[0066] The object detection unit 280 detects point clouds corresponding to the same object from the difference point cloud acquired by the difference point cloud acquisition unit 250. One method for the object detection unit 280 to detect a point cloud indicating an object is to group the points constituting the difference point cloud based on the distance between the points. First, the object detection unit 280 calculates the distance between the points constituting the difference point cloud. If the distance between two points is shorter than a predetermined distance, the two points are considered to be a point cloud indicating the same object. The object detection unit 280 sequentially examines each point constituting the difference point cloud and groups the points.

[0067] The display control unit 260 generates a detection result screen that shows objects present in real space that have been detected by the object detection unit 280. The method of generating the detection result screen includes drawing a rectangle that surrounds each group of points grouped by the object detection unit 280, and distinguishing each group of points grouped by the object detection unit 280 by a different color.

[0068] Fig. 13 is a flowchart showing the operation of the point cloud processing device 200 according to the modified example. The processes of S304 to S320 are as described with reference to Fig. 11. The object detection unit 280 detects a point cloud corresponding to the same object from the difference point cloud acquired by the difference point cloud acquisition unit 250 (S328). Then, the display unit 270 displays the point cloud corresponding to the same object detected by the object detection unit 280 (S332).

[0069] According to this configuration, when there are a plurality of detection target objects in a space, the user can easily distinguish and recognize each of the detection target objects.

[0070] <Hardware configuration> The above has described the embodiments of the present invention. Information processing such as the extraction of the background point cloud and the update of the integrated background point cloud described above is realized by cooperation between software and the hardware of the point cloud processing device 200 described below.

[0071] 14 is a block diagram showing the hardware configuration of a point cloud processing device 200. The point cloud processing device 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and a host bus 204. The point cloud processing device 200 also includes a bridge 205, an external bus 206, an interface 207, an input device 208, a display device 209, an audio output device 210, a storage device (HDD) 211, a drive 212, and a network interface 215.

[0072] The CPU 201 functions as an arithmetic processing unit and control unit, and controls the overall operation of the point cloud processing device 200 in accordance with various programs. The CPU 201 may also be a microprocessor. The ROM 202 stores programs used by the CPU 201, calculation parameters, etc. The RAM 203 temporarily stores programs used in the execution of the CPU 201, parameters that change as appropriate during the execution, etc. These are interconnected by a host bus 204 that includes a CPU bus, etc. Cooperation between the CPU 201, ROM 202, RAM 203, and software can realize functions such as the point cloud acquisition unit 220, distribution model creation unit 240, difference point cloud acquisition unit 250, display control unit 260, and object detection unit 280 described above.

[0073] The host bus 204 is connected to an external bus 206, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 205. Note that the host bus 204, bridge 205, and external bus 206 do not necessarily need to be configured separately, and these functions may be implemented on a single bus.

[0074] The input device 208 is composed of input means such as a mouse, keyboard, touch panel, buttons, microphone, sensors, switches, and levers that allow the user to input information, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 201. By operating the input device 208, the user of the point cloud processing device 200 can input various data to the point cloud processing device 200 and instruct processing operations.

[0075] The display device 209 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, a projector device, an OLED (Organic Light Emitting Diode) device, a lamp, etc. The audio output device 210 includes audio output devices such as a speaker and headphones.

[0076] The storage device 211 is a data storage device configured as an example of a storage unit of the point cloud processing device 200 according to this embodiment. The storage device 211 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, and a deletion device that deletes data recorded on the storage medium. The storage device 211 is configured, for example, by an HDD (Hard Disk Drive) or an SSD (Solid Storage Drive), or a memory having equivalent functions. This storage device 211 drives storage and stores programs executed by the CPU 201 and various data.

[0077] The drive 212 is a reader / writer for a storage medium, and is built into or externally attached to the point cloud processing device 200. The drive 212 reads information recorded on a removable storage medium 24, such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 203 or the storage device 211. The drive 212 can also write information to the removable storage medium 24.

[0078] The network interface 215 is a communication interface configured with, for example, a communication device for connecting to a network, etc. The network interface 215 may be a wireless LAN (Local Area Network) compatible communication device or a wired communication device for performing wired communication.

[0079] <Supplementary information> Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modifications and alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention.

[0080] For example, the steps in the processing of the point cloud processing device 200 in this specification do not necessarily have to be processed in chronological order according to the order described in the flowchart. For example, the steps in the processing of the point cloud processing device 200 may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

[0081] Furthermore, there are no particular limitations on the object measured by the sensor 100 according to the embodiment of the present invention. For example, the object measured by the sensor 100 may be indoors, outdoors, or at a construction site.

[0082] It is also possible to create a computer program that causes hardware such as a CPU, ROM, and RAM built into the point cloud processing device 200 to perform functions equivalent to those of the above-described components of the point cloud processing device 200. A storage medium storing the computer program is also provided. [Explanation of symbols]

[0083] 100 sensors 110 Tripod 200 Point Cloud Processing Device 220 Point cloud acquisition part 230 Storage section 240 Distribution Model Creation Department 250 Difference point cloud acquisition part 260 Display control unit 270 Display section 280 Object detection unit

Claims

1. a point cloud acquisition unit that acquires an input point cloud, which is a three-dimensional point cloud obtained by measuring a real space including an object with a sensor; a distribution model creation unit that performs statistical processing on a reference background point cloud, which is a three-dimensional point cloud that represents a background region of the object, and creates a distribution model that represents a distribution situation of the reference background point cloud in a three-dimensional shape; a difference point cloud acquisition unit that acquires a difference point cloud indicating a difference between the input point cloud and the reference background point cloud based on the distribution model; Equipped with The difference point cloud acquisition unit performs calculation processing on the three-dimensional coordinates of the distribution model, acquires an expanded range by expanding the distribution model, and acquires, from the input point cloud, a point cloud that is not included within the expanded range as the difference point cloud.

2. The point cloud processing device according to claim 1 , wherein the distribution model creation unit creates the distribution model by dividing a real space including the reference background point cloud into three-dimensional solids and performing statistical processing on the point cloud included in each solid.

3. The distribution model creation unit, for each solid, performing principal component analysis on the point group included in the solid to identify three mutually orthogonal principal component directions; Identifying the distances in each principal component direction between points that are farthest apart in each principal component direction in the point cloud included in the solid; The point cloud processing apparatus according to claim 2 , wherein the three specified distances correspond to the lengths of three sides, and a rectangular parallelepiped including all the point clouds within the solid body is created as the distribution model.

4. The point cloud processing apparatus according to claim 1 , wherein the shape of the expansion range is similar to the three-dimensional shape of the distribution model, and the center of gravity of the expansion range coincides with the center of gravity of the distribution model.

5. The point cloud processing device according to any one of claims 1 to 4, further comprising a reference background point cloud creation unit that creates the reference background point cloud from point cloud data acquired from the sensor.

6. The point cloud processing device according to claim 1, further comprising an object detection unit that detects a point cloud corresponding to the same object from the difference point cloud.

7. The point cloud processing device according to claim 6 , further comprising a display unit that displays the difference point cloud or the point cloud corresponding to the identical object.

8. Acquiring an input point cloud, which is a three-dimensional point cloud obtained by measuring a real space including an object with a sensor; performing statistical processing on a reference background point cloud, which is a three-dimensional point cloud representing a background region of the object, and creating a distribution model representing the distribution of the reference background point cloud in a three-dimensional shape; acquiring a difference point cloud indicating a difference between the input point cloud and the reference background point cloud based on the distribution model; Including, A point cloud processing method executed by a computer, wherein acquiring the difference point cloud includes performing calculations on three-dimensional coordinates of the distribution model, acquiring an expanded range by expanding the distribution model, and acquiring, as the difference point cloud, a point cloud from the input point cloud that is not included within the expanded range.

9. Computer, a point cloud acquisition unit that acquires an input point cloud, which is a three-dimensional point cloud obtained by measuring a real space including an object with a sensor; a distribution model creation unit that performs statistical processing on a reference background point cloud, which is a three-dimensional point cloud that represents a background region of the object, and creates a distribution model that represents a distribution situation of the reference background point cloud in a three-dimensional shape; a difference point cloud acquisition unit that acquires a difference point cloud indicating a difference between the input point cloud and the reference background point cloud based on the distribution model; It functions as The difference point cloud acquisition unit performs calculations on the three-dimensional coordinates of the distribution model, acquires an expanded range by expanding the distribution model, and acquires, from the input point cloud, a point cloud that is not included within the expanded range as the difference point cloud.

Images