Noise removal device, noise removal method, and program
The noise removal device addresses SLAM-generated noise at object-background boundaries by using voxel-based point density and normal vector calculations to accurately distinguish and eliminate noise, enhancing three-dimensional image clarity.
Patent Information
- Application Number
- JP2023082669
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
SLAM technology generates noise at the boundaries between objects and their backgrounds in three-dimensional images due to errors in acquiring and processing three-dimensional point cloud data, leading to inaccuracies such as thicker object appearances and floating noise.
A noise removal device that utilizes voxel-based processing to calculate point density and normal vectors, estimates point cloud data as thickness, edges, or surfaces, and determines and removes noise based on these characteristics.
Effectively removes noise at or around the boundaries between objects and their backgrounds in three-dimensional images, improving the accuracy of object representation.
Smart Images

Figure 2024166519000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a noise removal device, a noise removal method, and a program. [Background technology]
[0002] Simultaneous Localization and Mapping (SLAM) is known as a technology for generating an environmental map, which is a three-dimensional image corresponding to real space. SLAM is a technology for simultaneously estimating a self-position and generating an environmental map. As a conventional technology related to SLAM, for example, there is a three-dimensional data processing device described in Patent Document 1. This three-dimensional data processing device receives reflected light of a laser irradiated on the surface of an object, generates point cloud data which is a collection of position data of the reflection points, derives normals of pixels in the position data of the point cloud data, calculates the brightness of the pixels based on the normals, and constructs a three-dimensional image of the object based on the position data and brightness of the pixels. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2012-141758 A Summary of the Invention [Problem to be solved by the invention]
[0004] In addition to errors when acquiring 3D point cloud data, SLAM can also have errors caused by the algorithm used to process the point cloud data. When these errors occur, there is a problem in that noise appears at or near the boundary between the object and its background when the object is displayed as a 3D image using point cloud data. For example, when point cloud data caused by the above errors occurs at the boundary between a building displayed as a 3D image and its background, the building's walls may appear thicker in the 3D image than they actually are, making it impossible to accurately reconstruct the building. Point cloud data caused by the above errors may also appear as floating noise along the boundary between the object and its background.
[0005] The three-dimensional data processing device described in Patent Document 1 uses pixel brightness calculated based on pixel normals in the position data of the point cloud data to remove specular reflection components that change in synchronization with the observation angle of the object that changes with the movement of the user. Therefore, although it can remove noise related to brightness in a three-dimensional image, it cannot remove unnecessary point cloud data that occurs at the boundary between an object and its background.
[0006] The present disclosure is devised to solve the above-mentioned problems, and aims to provide a noise removal device capable of removing noise that occurs at or around the boundary between an object represented by three-dimensional point cloud data and its background. [Means for solving the problem]
[0007] The noise removal device disclosed herein includes a data acquisition unit that acquires point cloud data, which is a collection of three-dimensional points in a target area measured by a sensor, a voxel setting unit that sets multiple voxels in the target area, a point density calculation unit that calculates a point density, which is the density of three-dimensional points in the voxel, a normal vector calculation unit that calculates a normal vector of a partial plane including the multiple three-dimensional points, an estimation unit that estimates point cloud data corresponding to either a thickness portion, an edge or a surface of an object based on a difference in normal vectors between partial planes in the voxel, a noise determination unit that determines whether the point cloud data in the voxel is noise based on the point cloud data estimated to be either a thickness portion, an edge or a surface of the object and the point density, and a noise removal unit that removes the point cloud data determined to be noise. Effect of the Invention
[0008] According to the present disclosure, the noise removal device obtains point cloud data, which is a set of three-dimensional points in a target area measured by a sensor, sets multiple voxels in the target area, calculates a point density, which is the density of three-dimensional points in the voxels, calculates a normal vector of a partial plane including the multiple three-dimensional points, estimates point cloud data corresponding to any of the thickness part, edge, or face of an object based on a difference in normal vectors between the partial planes in the voxel, determines whether the point cloud data in the voxel is noise based on the point cloud data estimated to be any of the thickness part, edge, or face of the object and the point density, and removes the point cloud data determined to be noise. As a result, the noise removal device according to the present disclosure can remove noise occurring at or around the boundary between an object represented by three-dimensional point cloud data and its background. [Brief description of the drawings]
[0009] [Figure 1] 1 is a block diagram showing an example of the configuration of a noise removal device according to a first embodiment. [Diagram 2] FIG. 1 is a schematic diagram showing an overview of processing on voxel and point cloud data. [Diagram 3] FIG. 11 is a schematic diagram showing an overview of a noise determination process. [Figure 4]FIG. 1 is a schematic diagram showing an overview of a 3D image generation process using SLAM. [Diagram 5] FIG. 11 is a screen diagram showing an example of a three-dimensional image generated using point cloud data from which noise has been removed. [Figure 6] 3 is a flowchart showing a noise removal method according to the first embodiment. [Figure 7] 7A and 7B are block diagrams showing a hardware configuration for realizing the functions of the noise removal device according to the first embodiment. [Figure 8] FIG. 11 is a block diagram showing an example of the configuration of a noise removal device according to a second embodiment. [Figure 9] FIG. 13 is a schematic diagram showing an outline of a process for calculating the center of gravity of a voxel using the distribution of distance measurement data. [Figure 10] FIG. 13 is a schematic diagram showing an outline of a process for calculating the center of gravity of a voxel using the distribution of reflection intensity data. [Figure 11] FIG. 13 is a schematic diagram showing an outline of a process for calculating the center of gravity of a voxel using the distribution of RGB data. [Figure 12] FIG. 13 is a schematic diagram showing an outline of a process for calculating the center of gravity of a voxel using the distribution of distances to an estimated plane; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Embodiment 1 Fig. 1 is a block diagram showing an example of the configuration of a noise removal device 1 according to embodiment 1. In Fig. 1, the noise removal device 1 removes noise from three-dimensional point cloud data of a target area acquired by SLAM using a sensor 2. In particular, the noise removal device 1 removes noise that occurs at or near the boundary between an object represented by point cloud data in a target region and its background. Noise occurring at or near the boundary between an object and its background can, for example, cause the thickness of an object (such as the wall of a building) in a three-dimensional image to appear thicker than it actually is. This noise is caused by the occurrence of a large number of unnecessary three-dimensional points that do not actually exist at the boundary between the object and its background. Another type of noise is stray noise that occurs along the boundary between an object and its background. These noises are generated, for example, by errors that occur when acquiring point cloud data using SLAM or errors caused by the algorithm used to process the point cloud data. For this reason, noises are unavoidable when generating 3D images using SLAM.
[0011] The noise removal device 1 is realized, for example, as one function of a measuring device that executes SLAM. The measuring device is a tablet terminal, a smartphone, or a notebook PC. The noise removal device 1 may also be a device provided separately from the measuring device. In this case, the noise removal device 1 is connected to the measuring device via a wired or wireless connection, and acquires point cloud data from the measuring device via a wired or wireless connection.
[0012] The sensor 2 is a sensor that detects point cloud data, which is a collection of three-dimensional points in a target area, and there are sensors according to the type of SLAM. Examples of SLAM include LiDAR SLAM, Visual SLAM, and Depth SLAM. LiDAR SLAM is SLAM that uses LiDAR (Light Detection and Ranging) technology. LiDAR irradiates the surroundings with pulsed laser light, detects the light reflected from the surroundings, and obtains point cloud data indicating the positions and distances of objects present in the surroundings using the detection information of the reflected light. In LiDAR SLAM, the sensor 2 is a LiDAR.
[0013] LiDAR SLAM involves a LiDAR mounted on a moving object such as a vehicle, robot, or drone scanning its surroundings with pulsed light, and mapping the environment around the moving object based on the point cloud data obtained. In addition, LiDAR SLAM can have errors due to errors when acquiring point cloud data and errors caused by the algorithm used to process the point cloud data. Errors when acquiring point cloud data include, for example, errors caused by the accuracy of LiDAR and errors caused by the movement of a moving object. The accuracy of LiDAR is affected not only by measurement errors but also by vibrations or shaking that occur when the LiDAR moves. The algorithm for processing the point cloud data is, for example, to reconstruct the surrounding environment from the point cloud data. Therefore, when the surrounding environment of the moving object changes (for example, the movement of surrounding pedestrians or vehicles), an error occurs in reconstructing the surrounding environment. Also, an error may occur in the learning model for reconstructing the surrounding environment.
[0014] Visual SLAM uses camera images to estimate the self-location of a moving object equipped with a camera and create a map of the surrounding environment. A monocular camera is assumed to be used to obtain camera images, but a stereo camera can also be used. In Visual SLAM, the sensor 2 is, for example, the camera described above. For example, Visual SLAM calculates the 3D coordinates corresponding to each pixel from the camera image, acquires these coordinates as 3D point cloud data, and uses this point cloud data to estimate the position and orientation of the camera. In other words, Visual SLAM uses point cloud data to extract feature points, and the surrounding environment is extracted as a subset of the 3D point cloud.
[0015] Visual SLAM is subject to errors caused by errors when acquiring point cloud data and errors resulting from the algorithm used to process the point cloud data. Errors when acquiring point cloud data include, for example, measurement errors in the position or orientation of the camera, observation noise caused by erroneous detection of feature points on the image, and environmental changes caused by changes in the surrounding lighting conditions. The algorithm for processing the point cloud data is similar to LiDAR SLAM, for example, to reconstruct the surrounding environment from the point cloud data. Therefore, when the surrounding environment of the moving object changes (for example, the movement of surrounding pedestrians or vehicles), errors occur in reconstructing it. Also, errors may occur in the learning model for reconstructing the surrounding environment.
[0016] Depth SLAM uses a depth sensor such as an RGB-D camera or ToF camera to acquire three-dimensional point cloud data of the surroundings, and uses this point cloud data to estimate the self-position and generate an environmental map. Specifically, Depth SLAM extracts feature points using image information captured by a camera of the surroundings, calculates the three-dimensional position of each feature point based on the depth information (point cloud data) detected by the depth sensor, estimates the self-position of the moving object based on the positional relationship of the feature points, and generates a map of the surrounding environment. In Depth SLAM, sensor 2 is, for example, the camera and depth sensor mentioned above.
[0017] Depth SLAM can also be subject to errors when acquiring point cloud data and errors caused by the algorithm used to process the point cloud data. Errors that occur when acquiring point cloud data include, for example, errors caused by the accuracy of a depth sensor and errors caused by the movement of a moving object. The algorithm for processing the point cloud data is similar to LiDAR SLAM, for example, to reconstruct the surrounding environment from the point cloud data. Therefore, when the surrounding environment of the moving object changes (for example, the movement of surrounding pedestrians or vehicles), errors occur in reconstructing it. Also, errors may occur in the learning model for reconstructing the surrounding environment.
[0018] As described above, in SLAM, in addition to errors when acquiring three-dimensional point cloud data, there are also errors due to the algorithm of the processing applied to the point cloud data, and the occurrence of these errors causes noise to occur at the boundary between an object and its background or in its vicinity. The noise removal device 1 removes noise that occurs at the boundary between such an object and its background or in its vicinity. In the following description, it is assumed that the noise removal device 1 removes noise contained in the point cloud data obtained using LiDAR SLAM.
[0019] The noise removal device 1 is a device including at least a calculation unit and a storage unit. The calculation unit controls the overall operation of the noise removal device 1. The calculation unit includes a data acquisition unit 11, a voxel setting unit 12, a point density calculation unit 13, a normal vector calculation unit 14, an estimation unit 15, a noise determination unit 16, and a noise removal unit 17 shown in Fig. 1. The calculation unit executes an information processing application for removing noise from point cloud data, thereby realizing various functions of the data acquisition unit 11, the voxel setting unit 12, the point density calculation unit 13, the normal vector calculation unit 14, the estimation unit 15, the noise determination unit 16, and the noise removal unit 17.
[0020] The storage unit stores an information processing application and information used for the arithmetic processing of the arithmetic unit. The storage unit is a storage device included in a computer that functions as the noise removal device 1, and includes a storage such as a hard disk drive (HDD) or a solid state drive (SSD), or a memory 204 in FIG. 7B described later. The storage unit may be provided outside the noise removal device 1 as long as it is accessible by the noise removal device 1.
[0021] The data acquisition unit 11 acquires point cloud data, which is a collection of three-dimensional points of a target area measured by the sensor 2. The noise removal device 1 is connected to the sensor 2 in a wired or wireless manner so as to be able to communicate data with the sensor 2, and the data acquisition unit 11 acquires the point cloud data from the sensor 2 via an input interface provided in the noise removal device 1.
[0022] When the sensor 2 is a LiDAR, the LiDAR irradiates a laser pulse to a target area at a predetermined measurement period (e.g., every 100 milliseconds) and receives the reflected light of the laser pulse from the target area to measure the distance to an object present in the target area and detects point cloud data, which is a collection of three-dimensional points identified by the measured distance. The LiDAR is used while moving together with a mobile object. The mobile object is, for example, a vehicle capable of moving around the target area, a flying object capable of moving in the air above the target area, or a worker carrying a terminal equipped with a LiDAR.
[0023] The voxel setting unit 12 sets a plurality of voxels in the target region. For example, the voxel setting unit 12 sequentially determines a three-dimensional space of a certain size along the movement trajectory of the moving object, and sets the determined three-dimensional space as the three-dimensional space of the target region in which the voxels are set. Then, the voxel setting unit 12 divides the three-dimensional space of each target region into a plurality of voxels having the same size set in advance. Note that the size of the voxel is determined according to, for example, the accuracy of three-dimensionally restoring the real space. Also, the size of the voxel may be set by the user using an operation device not shown in FIG. 1.
[0024] If the three-dimensional space of the target area is a cube, cubic voxels are used, and if the three-dimensional space of the target area is a sphere or a complex shape with eight or more sides, voxels that match the shape of the space of the target area, such as octahedral voxels or hexahedral voxels, are used. When the voxel setting unit 12 sets multiple voxels in the space of the target region, it generates voxel point cloud data in which the voxels correspond to the point cloud data within the voxels, based on the position information of each three-dimensional point included in the point cloud data.
[0025] The point density calculation unit 13 calculates a point density, which is the density of three-dimensional points in a voxel. For example, the point density calculation unit 13 calculates a point density for each voxel in the target region using the voxel point cloud data generated by the voxel setting unit 12. The point density is, for example, the number of three-dimensional points per voxel.
[0026] The normal vector calculation unit 14 sets a combination of a plurality of three-dimensional points in a voxel, identifies a partial plane including a plurality of three-dimensional points in each combination, and calculates a normal vector of the identified partial plane. The calculation of the normal vector of the partial plane will be described with reference to FIG. 2. FIG. 2 is a schematic diagram showing an overview of processing related to voxels and point cloud data. The diagram on the left side of FIG. 2 is a cubic three-dimensional space representing a target region in which a plurality of voxels 100 are set. The voxel 100 is a cubic voxel. The voxel 100A is a voxel located in the center of the three-dimensional space representing the target region, and the voxel 100B is a voxel located on the surface side of the three-dimensional space.
[0027] 2 shows point cloud data in voxel 100A, and the right side of Fig. 2 shows point cloud data in voxel 100B. For example, normal vector calculation unit 14 sets combinations of three 3D points in voxel 100, identifies partial plane 101 including three 3D points 103A, 103B, and 103C, and calculates normal vector 102 of partial plane 101.
[0028] Although partial plane 101 defined by three three-dimensional points is shown, as long as multiple partial planes 101 can be defined by voxel 100 and the tendency of the three-dimensional points of the partial plane can be analyzed, the number of three-dimensional points on partial plane 101 is not limited to three. For example, the number of three-dimensional points on each partial plane 101 of voxel 100 may be the same or different.
[0029] Specifically, the normal vector calculation unit 14 calculates the cross product of a vector V1 between the three-dimensional point 103A and the three-dimensional point 103B and a vector V2 between the three-dimensional point 103A and the three-dimensional point 103C as the normal vector 102 of the partial plane 101. Alternatively, the normal vector calculation unit 14 may calculate the normal vector 102 by using a normal vector equation of a plane having the coordinates of the three-dimensional points 103A, 103B, and 103C included in the partial plane 101 as parameters.
[0030] The estimation unit 15 estimates point cloud data corresponding to any one of a thickness portion, an edge, or a face of an object based on the difference in normal vectors 102 between partial planes 101 in a voxel 100. Specifically, when the difference in normal vectors 102 between partial planes 101 in a voxel 100 exceeds an estimation threshold, the estimation unit 15 estimates that the point cloud data in the voxel 100 corresponds to the thickness portion of an object.
[0031] For example, it is considered that the 3D point cloud is not uniform but is detected randomly from the thickness part of the object. If the 3D points are randomly distributed, the difference in normal vectors 102 between partial planes 101 becomes large. Therefore, by appropriately setting the estimation threshold, it is possible to estimate whether or not the point cloud data exists in the thickness part of the object.
[0032] 2, the three-dimensional point cloud in the voxel 100A is not uniform, so the difference in normal vectors 102 between partial planes 101 in the voxel 100A exceeds the estimation threshold value. Therefore, the estimation unit 15 estimates that the point cloud data in the voxel 100A corresponds to the thickness part of the object.
[0033] Furthermore, when the difference in normal vectors 102 between partial planes 101 in a voxel 100 is equal to or less than the estimation threshold, the estimation unit 15 estimates that the point cloud data in the voxel 100 corresponds to an edge or face of an object. For example, it is considered that a three-dimensional point cloud is uniformly detected from the edge or face of an object. If the three-dimensional point cloud is uniform, the difference in normal vectors 102 between partial planes 101 becomes small. Therefore, by appropriately setting the estimation threshold, it is possible to estimate whether or not the point cloud data exists on the edge or face of an object.
[0034] 2, the 3D point cloud in the voxel 100B is uniformly present in a vertically elongated region, so it is considered that these 3D points are detected from the edge of the object. In this case, the difference between the normal vectors 102 of the partial planes 101 in the voxel 100B is equal to or smaller than the estimation threshold. Therefore, the estimation unit 15 estimates that the point cloud data in the voxel 100B corresponds to the edge of the object.
[0035] In addition, when the 3D point cloud exists uniformly on a plane, it is considered that these 3D points are detected from the surface of the object. Even in this case, the difference between the normal vectors 102 of the partial planes 101 in the voxel 100 is equal to or smaller than the estimation threshold. Therefore, the estimation unit 15 estimates that the point cloud data in the voxel 100 corresponds to the surface of the object.
[0036] The noise determination unit 16 determines whether the point cloud data in the voxel 100 is noise or not based on the point cloud data estimated to be either a thickness portion, an edge, or a surface of an object and the point density. The determination of whether the point cloud data in the voxel 100 is noise or not will be described with reference to FIG. 3. FIG. 3 is a schematic diagram showing an overview of the noise determination process. The diagram on the left side of FIG. 3 is a cubic three-dimensional space representing a target region in which a plurality of voxels 100 are set. Voxel 100C and voxel 100D are voxels adjacent to voxel 100A.
[0037] 3 shows point cloud data in voxel 100C, and the right side of Fig. 3 shows point cloud data in voxel 100D. For example, normal vector calculation unit 14 sets combinations of three 3D points in voxel 100, identifies partial plane 101 including three 3D points 103A, 103B, and 103C, and calculates normal vector 102 of partial plane 101.
[0038] Since the three-dimensional point cloud in the voxel 100C in Fig. 3 is not uniform, the difference in normal vectors 102 between the partial planes 101 in the voxel 100C exceeds the estimation threshold. Therefore, the estimation unit 15 estimates that the point cloud data in the voxel 100C corresponds to the thickness part of the object. Since the three-dimensional points included in the partial planes 101 in the voxel 100D in Fig. 3 are uniform, the difference in normal vectors 102 between the partial planes 101 is equal to or smaller than the estimation threshold. Therefore, the estimation unit 15 estimates that the point cloud data in the voxel 100D corresponds to the edge of the object.
[0039] When the point density of a voxel whose point cloud data is estimated to correspond to the thickness part of an object is lower by a first judgment threshold or more than the point density of surrounding voxels including the adjacent voxels, the noise judgment unit 16 judges the point cloud data in the voxel estimated to correspond to the thickness part of the object to be noise.
[0040] 3, voxel 100C, whose point cloud data is estimated to correspond to the thickness portion of the object, is adjacent to voxel 100A, whose point cloud data is similarly estimated to correspond to the thickness portion of the object. When the point density of voxel 100C is lower than the point density of voxel 100A by at least the first determination threshold, noise determination unit 16 determines that the point cloud data in voxel 100C, whose point cloud data is estimated to correspond to the thickness portion of the object, is noise.
[0041] When the point density of a voxel whose point cloud data is estimated to be either an edge or a surface of an object is lower than a second judgment threshold, the noise judgment unit 16 judges the point cloud data in the voxel whose point cloud data is estimated to be either an edge or a surface of an object to be noise.
[0042] 3, voxel 100D, whose point cloud data is estimated to correspond to an edge or surface of an object, is present in the vicinity of voxel 100B, whose point cloud data is similarly estimated to correspond to an edge or surface of an object, and adjacent voxel 100A. When the point density of voxel 100D is lower than the point density of voxel 100B by the second determination threshold or more, noise determination unit 16 determines that the point cloud data in voxel 100D, whose point cloud data is estimated to correspond to an edge or surface of an object, is noise.
[0043] The first and second judgment thresholds are determined, for example, according to the noise judgment accuracy in design. The first and second judgment thresholds may be set by a user using an operation device not shown in FIG.
[0044] The noise removal unit 17 removes the point cloud data determined to be noise. For example, the noise removal unit 17 removes the point cloud data determined to be noise from the point cloud data acquired by the data acquisition unit 11. For example, the point cloud data from which the point cloud data determined to be noise has been removed is output from the noise removal device 1 to a three-dimensional image generating device (not shown in FIG. 1). The three-dimensional image generating device generates a three-dimensional image using the point cloud data input from the noise removal device 1.
[0045] FIG. 4 is a schematic diagram showing an overview of a three-dimensional image generation process using SLAM. The diagram on the left side of FIG. 4 is a bird's-eye view A of an area where sensor 2 has acquired data. In this area, there are objects B1 to B4, which are flowerbed-like structures. A moving body (for example, a vehicle) equipped with sensor 2 moves along a trajectory C. The moving body moving along trajectory C passes through D1, which is the start point of movement, orientation points D2 to D5 including D1, and E, which is the end point of movement. In addition, verification points E1 to E6 are also installed in this area.
[0046] The control points D1 to D5 are reference points used to estimate the position and attitude of the sensor 2 mounted on the moving body, and are physical landmarks or points used when generating an environmental map. In LiDAR SLAM, the control points D1 to D5 must be identified in order to estimate the self-position of the moving body and generate an environmental map. The control points D1 to D5 are identified by detection information of the sensor 2.
[0047] The verification points E1 to E6 are observation points for identifying the relative positional relationship between the orientation points D1 to D5 and the moving body, and are used to calculate the distance from the orientation point, the angle, or the position on the image. For example, in LiDAR SLAM, the verification points E1 to E6 are used to calculate the distance from the orientation point D4 to the verification points E1 to E3, the angle from the orientation point D4 to the verification points E1 to E3, or the position on the image for the object B1.
[0048] The diagram on the right side of Fig. 4 is an environmental map (three-dimensional image) F1 generated using point cloud data acquired along a trajectory C, and the environmental map F1 includes images H1-H4 showing objects B1-B4. If the noise removal device 1 has not removed noise from the point cloud data, for example, noise N1 and noise N2 occur in image H1 in a rectangular partially enlarged view G. Noise N1 is noise that causes the thickness of object B1 to appear thicker than it actually is. Moreover, noise N2 is stray noise that occurs along the boundary between object B1 and its background.
[0049] Fig. 5 is a screen diagram showing an example of a three-dimensional image generated using noise-removed point cloud data, and shows an environmental map (three-dimensional image) F2 obtained by removing noise from the point cloud data used to generate the environmental map F1 in Fig. 4. The noise removal device 1 removes, for example, noise N1 occurring in the boundary portion between images H1-H4 showing objects B1-B4 shown in Fig. 4 and their background, and noise N2 occurring in the periphery thereof. As a result, as shown in a partially enlarged view G in Fig. 5, the actual thickness of object B1 is accurately restored in image H1.
[0050] FIG. 6 is a flowchart showing the noise removal method according to the first embodiment. The data acquisition unit 11 acquires point cloud data of a target area (step ST1). For example, the data acquisition unit 11 acquires point cloud data used to generate the environmental map F2 of FIG. The voxel setting unit 12 sets voxels 100 in the target region (step ST2). The point density calculation unit 13 calculates the point density within the voxel 100 (step ST3). The normal vector calculation unit 14 calculates the normal vector 102 of the partial plane 101 that includes a plurality of three-dimensional points (step ST4). The estimation unit 15 estimates point cloud data corresponding to any one of the thickness portions, edges, or faces of the objects B1 to B4 based on the difference in normal vectors 102 between the partial planes 101 in the voxel 100 (step ST5). The noise determination unit 16 determines whether the point cloud data in the voxel 100 is noise or not, based on the point cloud data estimated to be any of the thickness portions, edges, or faces of the objects B1 to B4 and the point density (step ST6). The noise removal unit 17 removes the point cloud data determined to be noise (step ST7). The noise removal device 1 performs the noise removal method shown in FIG. 6, thereby making it possible to remove noise occurring at or near the boundaries between the objects B1 to B4 represented by the point cloud data and their background.
[0051] Next, a hardware configuration for realizing the functions of the noise removal device 1 will be described. The functions of the data acquisition unit 11, the voxel setting unit 12, the point density calculation unit 13, the normal vector calculation unit 14, the estimation unit 15, the noise determination unit 16 and the noise removal unit 17 included in the noise removal device 1 are realized by a processing circuit. That is, the noise removal device 1 includes a processing circuit for executing each process from step ST1 to step ST7 shown in Fig. 6. The processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.
[0052] Fig. 7A is a block diagram showing a hardware configuration for realizing the functions of the noise removal device 1. Fig. 7B is a block diagram showing a hardware configuration for executing software for realizing the functions of the noise removal device 1. In Figs. 7A and 7B, an input interface 200 is an interface that relays detection information output from the sensor 2 to the noise removal device 1. An output interface 201 is an interface that relays three-dimensional point cloud data output from the noise removal device 1 to a downstream device.
[0053] When the processing circuit is the dedicated hardware processing circuit 202 shown in FIG. 7A, the processing circuit 202 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of the data acquisition unit 11, voxel setting unit 12, point density calculation unit 13, normal vector calculation unit 14, estimation unit 15, noise determination unit 16 and noise removal unit 17 of the noise removal device 1 may be realized by separate processing circuits, or these functions may be realized together by a single processing circuit.
[0054] 7B, the functions of the data acquisition unit 11, the voxel setting unit 12, the point density calculation unit 13, the normal vector calculation unit 14, the estimation unit 15, the noise determination unit 16, and the noise removal unit 17 included in the noise removal device 1 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 204.
[0055] The processor 203 reads out and executes the programs stored in the memory 204, thereby realizing the functions of the data acquisition unit 11, the voxel setting unit 12, the point density calculation unit 13, the normal vector calculation unit 14, the estimation unit 15, the noise determination unit 16, and the noise removal unit 17 included in the noise removal device 1. For example, the noise removal device 1 includes a memory 204 for storing a program that, when executed by the processor 203, results in the processing of steps ST1 to ST7 shown in FIG. These programs cause the computer to execute the processing procedures or methods performed by the data acquisition unit 11, voxel setting unit 12, point density calculation unit 13, normal vector calculation unit 14, estimation unit 15, noise determination unit 16 and noise removal unit 17. The memory 204 may be a computer-readable storage medium storing programs for causing the computer to function as a data acquisition unit 11, a voxel setting unit 12, a point density calculation unit 13, a normal vector calculation unit 14, an estimation unit 15, a noise determination unit 16 and a noise removal unit 17.
[0056] Memory 204 may be, for example, a non-volatile or volatile semiconductor memory such as a Random Access Memory (RAM), a Read Only Memory (ROM), a flash memory, an Erasable Programmable Read Only Memory (EPROM), an Electrically-EPROM (EEPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD, etc.
[0057] Some of the functions of the data acquisition unit 11, voxel setting unit 12, point density calculation unit 13, normal vector calculation unit 14, estimation unit 15, noise determination unit 16 and noise removal unit 17 included in the noise removal device 1 may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware. For example, the data acquisition unit 11 has its functions realized by a processing circuit 202, which is dedicated hardware, and the voxel setting unit 12, point density calculation unit 13, normal vector calculation unit 14, estimation unit 15, noise determination unit 16 and noise removal unit 17 have their functions realized by a processor 203 reading and executing a program stored in a memory 204. Thus, the processing circuitry may implement the above-described functions through hardware, software, firmware or a combination of these.
[0058] As described above, the noise removal device 1 according to the first embodiment includes a data acquisition unit 11 that acquires point cloud data of a target region, a voxel setting unit 12 that sets a plurality of voxels 100 in the target region, a point density calculation unit 13 that calculates a point density in the voxel 100, a normal vector calculation unit 14 that calculates a normal vector 102 of a partial plane 101 including a plurality of three-dimensional points 103, an estimation unit 15 that estimates point cloud data in the voxel 100 that corresponds to any of the thickness portions, edges or faces of the objects B1 to B4 based on the difference between the normal vectors 102 between the partial planes 101 in the voxel 100, a noise determination unit 16 that determines whether the point cloud data in the voxel 100 is noise or not based on the point cloud data in the voxel 100 estimated to be any of the thickness portions, edges or faces of the objects B1 to B4 and the point density, and a noise removal unit 17 that removes the point cloud data determined to be noise. This enables the noise removal device 1 to remove noise that occurs at or near the boundaries between the objects B1 to B4 represented by the three-dimensional point cloud data and their background.
[0059] In the noise removal device 1 according to the first embodiment, when the difference in normal vectors 102 between partial planes 101 in a voxel 100 exceeds an estimation threshold, the estimation unit 15 estimates that the point cloud data in the voxel 100 corresponds to a thickness part of an object, and when the difference in normal vectors 102 is equal to or smaller than the estimation threshold, the estimation unit 15 estimates that the point cloud data in the voxel 100 corresponds to an edge or a surface of the object. This allows the noise removal device 1 to appropriately estimate the point cloud data corresponding to the thickness part, edge or surface of an object.
[0060] In the noise removal device 1 according to the first embodiment, the noise determination unit 16 determines that the point cloud data in the voxel 100 is noise when the point density of the voxel 100 whose point cloud data is estimated to correspond to the thickness portion of the object is lower than the point density of the voxel 100 adjacent to the voxel 100 or the surrounding voxels 100 including the adjacent voxels by a first determination threshold or more, and also determines that the point cloud data in the voxel 100 is noise when the point density of the voxel 100 whose point cloud data is estimated to correspond to either the edge or the surface of the object is lower than a second determination threshold. This allows the noise removal device 1 to appropriately determine noise from the point cloud data determined to correspond to the thickness portion, edge or surface of the object.
[0061] The noise removal method according to the first embodiment includes step ST1 in which a data acquisition unit 11 acquires point cloud data of a target region, step ST2 in which a voxel setting unit 12 sets a plurality of voxels in the target region, step ST3 in which a point density calculation unit 13 calculates a point density in a voxel 100, step ST4 in which a normal vector calculation unit 14 calculates a normal vector 102 of a partial plane 101 including a plurality of three-dimensional points, step ST5 in which an estimation unit 15 estimates point cloud data corresponding to any of a thickness portion, an edge or a face of the objects B1 to B4 based on a difference in normal vectors 102 between the partial planes 101 in the voxel 100, step ST6 in which a noise determination unit 16 determines whether or not the point cloud data in the voxel 100 is noise based on the point cloud data estimated to be any of a thickness portion, an edge or a face of the objects B1 to B4 and the point density, and step ST7 in which a noise removal unit 17 removes the point cloud data determined to be noise. The noise removal device 1 that executes this method can remove noise that occurs at or near the boundaries between the objects B1 to B4 represented by the three-dimensional point cloud data and their background.
[0062] The computer that executes the program according to the first embodiment functions as a noise removal device 1 that includes a data acquisition unit 11 that acquires point cloud data of a target region, a voxel setting unit 12 that sets a plurality of voxels 100 in the target region, a point density calculation unit 13 that calculates a point density in the voxel 100, a normal vector calculation unit 14 that calculates a normal vector 102 of a partial plane 101, an estimation unit 15 that estimates point cloud data corresponding to any of the thickness parts, edges, or faces of the objects B1 to B4 based on the difference between the normal vectors 102 between the partial planes 101 in the voxel 100, a noise determination unit 16 that determines whether the point cloud data in the voxel 100 is noise based on the point cloud data estimated to be any of the thickness parts, edges, or faces of the objects B1 to B4 and the point density, and a noise removal unit 17 that removes the point cloud data determined to be noise. It is possible to provide a noise removal device 1 that removes noise occurring at or near the boundary between the objects B1 to B4 represented by three-dimensional point cloud data and their background.
[0063] Embodiment 2 Fig. 8 is a block diagram showing a configuration example of a noise removal device 1A according to the second embodiment. In Fig. 8, the noise removal device 1A removes noise in three-dimensional point cloud data of a target area acquired by SLAM using a sensor 2. In particular, the noise removal device 1A removes noise occurring at or near a boundary between an object represented by point cloud data in the target area and its background. In Fig. 8, the same components as those in Fig. 1 are denoted by the same reference numerals, and description thereof will be omitted.
[0064] The noise removal device 1A is a device that includes at least a calculation unit and a storage unit. The calculation unit controls the overall operation of the noise removal device 1A. The calculation unit includes a data acquisition unit 11, a voxel setting unit 12, a point density calculation unit 13, a normal vector calculation unit 14, an estimation unit 15, a noise determination unit 16, a noise removal unit 17, and a center of gravity calculation unit 18 shown in FIG. 8. The calculation unit executes an information processing application for removing noise from point cloud data, thereby realizing various functions of the data acquisition unit 11, the voxel setting unit 12, the point density calculation unit 13, the normal vector calculation unit 14, the estimation unit 15, the noise determination unit 16, the noise removal unit 17, and the center of gravity calculation unit 18.
[0065] The storage unit stores an information processing application and information used for the arithmetic processing of the arithmetic unit. The storage unit is a storage device included in a computer that functions as the noise removal device 1A, and includes a storage such as a hard disk drive (HDD) or a solid state drive (SSD), or the memory 204 in FIG. 7B. The storage unit may be provided outside the noise removal device 1A as long as it is accessible by the noise removal device 1A.
[0066] The center of gravity calculation unit 18 calculates the distribution of measurement accuracy related information related to the measurement accuracy of the point cloud data in the voxel 100, weights the three-dimensional points in the voxel 100 based on the distribution of the measurement accuracy related information, and calculates the center of gravity of the point cloud data in the voxel 100 according to the weighting. Here, the measurement accuracy related information includes distance measurement data, reflection intensity data, RGB data, and distance information from the estimated plane. The distance measurement data and reflection intensity data are information added to each three-dimensional point in the point cloud data. The RGB data are color information of R (red), G (green), and B (blue) of pixels corresponding to each three-dimensional point in the point cloud data. The distance information from the estimated plane is information indicating the distance between the estimated plane estimated to be closest to the target area and the point cloud data of the target area.
[0067] Weighting of each 3D point in the point cloud data within the voxel 100 is a process of reflecting a coefficient representing the weight assigned to the 3D point in the coordinates of the 3D point. For example, the center of gravity calculation unit 18 calculates a weighted average of the coordinate values of each three-dimensional point in the voxel 100 according to the weight of the coordinates, and sets each weighted average value as the center of gravity coordinate of the point cloud data in the voxel 100. By calculating the center of gravity of the three-dimensional points in the voxel 100 according to the weights assigned to the three-dimensional points based on the measurement accuracy related information related to the measurement accuracy of the point cloud data, it is possible to calculate a center of gravity closer to a true value, i.e., closer to the actual environment.
[0068] The center of gravity calculation unit 18 may also calculate the center of gravity of the point cloud data in the voxel 100 in which point cloud data estimated to be one of the thickness portion, edge, or surface of the object exists, and perform processing to thin out point cloud data that does not exist within a certain range from the center of gravity in the voxel 100. The certain range is a range defined by the measurement accuracy related information used to calculate the center of gravity. For example, when weighting of three-dimensional points is performed based on distance measurement data, and the center of gravity is calculated according to the distribution of three-dimensional points according to the weighting, the center of gravity calculation unit 18 thins out point cloud data that does not exist within a certain distance range from the center of gravity in the voxel 100. This allows the noise removal device 1A to obtain a distribution of three-dimensional points close to the actual environment.
[0069] The center of gravity calculation unit 18 may calculate the center of gravity of the point cloud data in the voxel 100 set by the voxel setting unit 12, and perform processing to thin out point cloud data that does not exist within a certain range from the center of gravity in the voxel 100. In this case, a series of processing is performed by the point density calculation unit 13, the normal vector calculation unit 14, the estimation unit 15, the noise determination unit 16, and the noise removal unit 17 using the point cloud data that has been thinned out by the center of gravity calculation unit 18. This allows the noise removal device 1A to remove noise from a distribution of three-dimensional points that is close to the actual environment.
[0070] Furthermore, the center of gravity calculation unit 18 may weight the three-dimensional points in the voxel 100 based on the distribution of the distance measurement data of the point cloud data (hereinafter referred to as the distance measurement distribution), and calculate the center of gravity of the point cloud data in the voxel 100 according to the weighting. The data of each three-dimensional point in the point cloud data includes distance measurement data indicating the distance from the three-dimensional point to the sensor 2. In SLAM, the closer a three-dimensional point is to the sensor 2, the better the measurement accuracy, i.e., the closer it tends to be to the true value. However, simply extracting a predetermined number of 3D points with close (short) distance measurement distances cannot accurately represent the distribution of 3D points within the voxel 100 . Therefore, the center of gravity calculation unit 18 assigns a larger weight to the three-dimensional points having closer distances in the measured distance distribution, and calculates the center of gravity of the point cloud data in the voxel 100 according to the weights assigned to the three-dimensional points.
[0071] Fig. 9 is a schematic diagram showing an outline of the process of calculating the center of gravity of the voxel 100A using the distribution of distance measurements. For example, the distribution of distance measurements of the weighted three-dimensional point 103 is the distribution shown on the left side of Fig. 9. As shown by the arrows in the left side of Fig. 9, the dark plots indicate three-dimensional points with close distance measurements, and the light plots indicate three-dimensional points with far distance measurements. As shown in the right side of Fig. 9, the dark three-dimensional points in the voxel 100A are present on the right side of the voxel 100A. Therefore, the center of gravity calculation unit 18 assigns a large weight to three-dimensional points with close measurement distances in the measurement distance distribution, calculates a weighted average of the coordinate values of each three-dimensional point within voxel 100 based on the weight of the coordinates, and sets each weighted average value as the center of gravity coordinate of the point cloud data within voxel 100.
[0072] Furthermore, the center of gravity calculation unit 18 may weight the three-dimensional points in the voxel 100 based on the distribution of reflection intensity data of the point cloud data (hereinafter, referred to as reflection intensity distribution), and calculate the center of gravity of the point cloud data in the voxel 100 according to the weight. The data of each three-dimensional point in the point cloud data includes reflection intensity data indicating the reflection intensity (received light intensity of reflected light) of the measurement light (pulse laser light irradiated from the LiDAR) from the three-dimensional point. In SLAM, the stronger the reflection intensity of a 3D point, the better the measurement accuracy, i.e., the closer it is to the true value. However, simply extracting a certain number of 3D points with strong reflection intensity cannot accurately represent the distribution of 3D points within the voxel 100. Therefore, the center of gravity calculation unit 18 assigns increasing weights to three-dimensional points in the reflection intensity distribution in descending order of reflection intensity, and calculates the center of gravity of the point cloud data within the voxel 100 according to the weights assigned to the three-dimensional points.
[0073] Fig. 10 is a schematic diagram showing an outline of the process of calculating the center of gravity of a voxel 100A using a reflection intensity distribution. For example, the reflection intensity distribution of a weighted three-dimensional point 103 is the distribution shown on the left side of Fig. 10. As shown by the arrows in the left diagram of Fig. 10, the strength of the reflection intensity is such that a dark-colored plot indicates a three-dimensional point 103 with a strong reflection intensity, and a light-colored plot indicates a three-dimensional point 103 with a weak reflection intensity. As shown in the right diagram of Fig. 10, the dark-colored three-dimensional point 103 in the voxel 100A is present on the right side of the voxel 100A. Therefore, the center of gravity calculation unit 18 assigns a large weight to three-dimensional points with strong reflection intensity in the reflection intensity distribution, calculates a weighted average of the coordinate values of each three-dimensional point within the voxel 100 based on the weight of the coordinates, and sets each weighted average value as the center of gravity coordinate of the point cloud data within the voxel 100.
[0074] Furthermore, the center of gravity calculation unit 18 may weight the three-dimensional points in the voxel 100 based on the distribution of RGB data of the point cloud data (hereinafter referred to as color distribution), and calculate the center of gravity of the point cloud data in the voxel 100 according to the weighting. The data of each three-dimensional point in the point cloud data obtained by Visual SLAM includes RGB data indicating the colors R (red), G (green), and B (blue) of the pixel corresponding to the three-dimensional point. Point cloud data including RGB data can also be obtained by adding RGB data for each pixel of a captured image in which a target area is captured by a camera to the point cloud data obtained by LiDAR SLAM. In SLAM, the darker the 3D point, the better the measurement accuracy, i.e., the closer it is to the true value. However, simply extracting a certain number of dark 3D points will not accurately represent the distribution of 3D points within the voxel 100. Therefore, the center of gravity calculation unit 18 assigns a larger weight to the three-dimensional points in the color distribution, starting with the darker color, and calculates the center of gravity of the point cloud data within the voxel 100 according to the weights assigned to the three-dimensional points.
[0075] Fig. 11 is a schematic diagram showing an outline of the process of calculating the center of gravity of the voxel 100A using color distribution. For example, the color distribution of the weighted three-dimensional point 103 is the distribution shown on the left side of Fig. 11. As shown by the arrows in the left side of Fig. 11, the dark plots indicate three-dimensional points with dark colors, and the light plots indicate three-dimensional points with light colors. As shown in the right side of Fig. 11, the dark three-dimensional point 103 in the voxel 100A is present on the right side of the voxel 100A. Therefore, the center of gravity calculation unit 18 assigns a large weight to three-dimensional points with darker colors in the color distribution, calculates a weighted average of the coordinate values of each three-dimensional point within the voxel 100 based on the weight of the coordinates, and sets each weighted average value as the center of gravity coordinate of the point cloud data within the voxel 100.
[0076] Alternatively, the center of gravity calculation unit 18 may estimate a plane 104 including three-dimensional points, weight the three-dimensional points in the voxel 100 based on the distribution of distances between the estimated plane 104 and the three-dimensional points 103 in the voxel 100A, and calculate the center of gravity of the point cloud data in the voxel 100 according to the weights. For example, the center of gravity calculation unit 18 estimates the plane 104 including three-dimensional points using the RANSAC method. In the RANSAC method, the number of three-dimensional points that are inliers is counted for each parameter of a function that represents a plane candidate, and the parameter with the largest count is determined as the optimal parameter. From the plane candidates represented by the function to which the determined parameters are applied, for example, the plane that includes the largest number of inlier three-dimensional points is determined as the estimation result.
[0077] Fig. 12 is a schematic diagram showing an outline of a process for calculating the center of gravity of a voxel 100A using a distribution of distances to an estimated plane 104. For example, the distribution of distances between the weighted three-dimensional point 103 and the plane 104 is shown on the left side of Fig. 12. As shown by the arrows in the left side of Fig. 12, the dark plots indicate three-dimensional points that are close to the estimated plane 104, and the light plots indicate three-dimensional points that are far from the plane 104. As shown in the right side of Fig. 12, the three-dimensional point 103 in the voxel 100A that is close to the plane 104 is present on the right side of the voxel 100A. Therefore, the center of gravity calculation unit 18 assigns a larger weight to three-dimensional points that are close in the distribution of distances to the plane 104, calculates a weighted average of the coordinate values of each three-dimensional point within the voxel 100 based on the weight of the coordinates, and sets each weighted average value as the center of gravity coordinate of the point cloud data within the voxel 100.
[0078] As described above, the noise removal device 1A according to the second embodiment includes a centroid calculation unit 18 that calculates the distribution of measurement-accuracy related information related to the measurement accuracy of the point cloud data in the voxel 100, weights the three-dimensional points in the voxel based on the distribution of the measurement-accuracy related information, and calculates the centroid of the point cloud data in the voxel in accordance with the weight. By calculating the centroid of the point cloud data in the voxel 100 in accordance with the weights assigned to the three-dimensional points based on the distribution of the measurement-accuracy related information, it is possible to calculate a centroid that is closer to the actual environment.
[0079] In the noise removal device 1A according to the second embodiment, the center of gravity calculation unit 18 calculates the center of gravity of the point cloud data in the voxel 100 in which point cloud data estimated to be any one of a thickness portion, an edge, or a surface of an object exists, and thins out the point cloud data that does not exist within a certain range from the center of gravity in the voxel 100. This enables the noise removal device 1A to obtain a distribution of three-dimensional points close to the actual environment.
[0080] In the noise removal device 1A according to the second embodiment, the center of gravity calculation unit 18 calculates the center of gravity of the point cloud data in the voxel 100 set by the voxel setting unit 12, and thins out the point cloud data that does not exist within a certain range from the center of gravity in the voxel 100. A series of processes are performed by the point density calculation unit 13, the normal vector calculation unit 14, the estimation unit 15, the noise determination unit 16, and the noise removal unit 17 using the point cloud data that has been thinned out by the center of gravity calculation unit 18. This allows the noise removal device 1A to remove noise from a distribution of three-dimensional points that is close to the actual environment.
[0081] In the noise removal device 1A according to the second embodiment, the center of gravity calculation unit 18 weights the three-dimensional points in the voxel 100 based on the ranging distance distribution, and calculates the center of gravity of the point cloud data in the voxel 100 according to the weights. By calculating the center of gravity of the point cloud data in the voxel 100 according to the weights assigned to the three-dimensional points based on the ranging distance distribution, the noise removal device 1A can calculate a center of gravity closer to the actual environment.
[0082] In the noise removal device 1A according to the second embodiment, the center of gravity calculation unit 18 weights the three-dimensional points in the voxel 100 based on the reflection intensity distribution, and calculates the center of gravity of the point cloud data in the voxel 100 according to the weights. By calculating the center of gravity of the point cloud data in the voxel 100 according to the weights assigned to the three-dimensional points based on the reflection intensity distribution, the noise removal device 1A can calculate a center of gravity closer to the actual environment.
[0083] In the noise removal device 1A according to the second embodiment, the center of gravity calculation unit 18 weights the three-dimensional points in the voxel 100 based on the color distribution, and calculates the center of gravity of the point cloud data in the voxel 100 according to the weights. By calculating the center of gravity of the point cloud data in the voxel 100 according to the weights assigned to the three-dimensional points based on the color distribution, the noise removal device 1A can calculate a center of gravity closer to the actual environment.
[0084] In the noise removal device 1A according to the second embodiment, the center of gravity calculation unit 18 estimates a plane including three-dimensional points, weights the three-dimensional points in the voxel 100 based on the distance distribution from the estimated plane, and calculates the center of gravity of the point cloud data in the voxel 100 according to the weights. By calculating the center of gravity of the point cloud data in the voxel 100 according to the weights assigned to the three-dimensional points based on the distance distribution from the estimated plane, the noise removal device 1A can calculate a center of gravity closer to the actual environment.
[0085] Various aspects of the present disclosure are summarized below as appendices.
[0086] (Appendix 1) A data acquisition unit that acquires point cloud data, which is a set of three-dimensional points of a target area measured by a sensor; a voxel setting unit that sets a plurality of voxels in the target region; a point density calculation unit that calculates a point density, which is a density of the three-dimensional points within the voxel; A normal vector calculation unit that calculates a normal vector of a partial plane including a plurality of the three-dimensional points; an estimation unit that estimates the point cloud data corresponding to any one of a thickness portion, an edge, or a surface of an object based on a difference in the normal vector between the partial planes in the voxel; a noise determination unit that determines whether the point cloud data in a voxel is noise based on the point cloud data estimated to be one of a thickness portion, an edge, or a surface of the object and the point density; a noise removal unit that removes the point cloud data that is determined to be noise; A noise removal device comprising: (Appendix 2) When a difference between the normal vectors of the partial planes in the voxel exceeds an estimation threshold, the estimation unit estimates that the point cloud data in the voxel corresponds to a thickness part of the object, and when the difference between the normal vectors is equal to or smaller than the estimation threshold, the estimation unit estimates that the point cloud data in the voxel corresponds to an edge or a face of the object. 2. The noise removal device according to claim 1, (Appendix 3) The noise determination unit determines the point cloud data in the voxel estimated to correspond to the thickness portion of the object as noise when the point density of the voxel estimated to correspond to the thickness portion of the object is lower than the point density of the voxel adjacent to the voxel or the surrounding voxels including the adjacent voxels by a first determination threshold or more, and determines the point cloud data in the voxel estimated to correspond to either the edge or the surface of the object as noise when the point density of the voxel estimated to correspond to the edge or the surface of the object is lower than a second determination threshold. 3. The noise removal device according to claim 1 or 2, (Appendix 4) a center of gravity calculation unit that calculates a distribution of measurement accuracy related information regarding the measurement accuracy of the point cloud data for the three-dimensional points in the voxel, weights the three-dimensional points in the voxel based on the distribution of the measurement accuracy related information, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weighting. 4. The noise removal device according to claim 1, (Appendix 5) The center of gravity calculation unit calculates a center of gravity of the point cloud data in the voxel in which the point cloud data estimated to be one of a thickness portion, an edge, or a surface of the object exists, and thins out the point cloud data that does not exist within a certain range from the center of gravity in the voxel. 5. The noise removal device according to claim 4, (Appendix 6) the center of gravity calculation unit calculates a center of gravity of the point cloud data in the voxel set by the voxel setting unit, and thins out the point cloud data that does not exist within a certain range from the center of gravity in the voxel; A series of processes are performed by the point density calculation unit, the normal vector calculation unit, the estimation unit, the noise determination unit, and the noise removal unit using the point cloud data that has been thinned out by the center of gravity calculation unit. 5. The noise removal device according to claim 4, (Appendix 7) The center of gravity calculation unit weights the three-dimensional points in the voxel based on a distribution of distance measurement data, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights. 7. The noise removal device according to claim 4, wherein: (Appendix 8) The center of gravity calculation unit weights the three-dimensional points in the voxel based on a distribution of reflection intensity data, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights. 7. The noise removal device according to claim 4, wherein: (Appendix 9) The center of gravity calculation unit weights the three-dimensional points in the voxel based on a distribution of RGB data, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights. 7. The noise removal device according to claim 4, wherein: (Appendix 10) The center of gravity calculation unit estimates a plane including the three-dimensional points, weights the three-dimensional points in the voxel based on a distribution of distances between the estimated plane and the three-dimensional points in the voxel, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights. 7. The noise removal device according to claim 4, wherein: (Appendix 11) A noise removal method executed by a noise removal device, comprising: A step in which a data acquisition unit acquires point cloud data which is a set of three-dimensional points of a target area measured by a sensor; A voxel setting unit sets a plurality of voxels in the target region; A step of calculating a point density, which is a density of the three-dimensional points in the voxel, by a point density calculation unit; A normal vector calculation unit calculates a normal vector of a partial plane including a plurality of the three-dimensional points; an estimation unit estimating the point cloud data corresponding to any one of a thickness portion, an edge, or a face of an object based on a difference in the normal vector between the partial planes in the voxel; a noise determination unit determining whether or not the point cloud data in the voxel is noise based on the point cloud data estimated to be any one of a thickness portion, an edge, or a surface of the object and the point density; A noise removal unit removes the point cloud data determined to be noise. A noise removal method comprising: (Appendix 12) Computer, a data acquisition unit that acquires point cloud data, which is a collection of three-dimensional points of a target area measured by a sensor; a voxel setting unit that sets a plurality of voxels in the target region; a point density calculation unit that calculates a point density, which is the density of the three-dimensional points within the voxel; a normal vector calculation unit that calculates a normal vector of a partial plane including a plurality of the three-dimensional points; an estimation unit that estimates the point cloud data corresponding to any one of a thickness portion, an edge, or a surface of an object based on a difference in the normal vector between the partial planes in the voxel; a noise determination unit that determines whether the point cloud data in a voxel is noise based on the point cloud data estimated to be any one of a thickness portion, an edge, or a surface of the object and the point density; a noise removal unit that removes the point cloud data determined to be noise; A program to function as a
[0087] It is possible to combine the embodiments, modify any of the components of the embodiments, or omit any of the components of the embodiments. [Explanation of symbols]
[0088] 1,1A noise removal device, 2 sensor, 11 data acquisition unit, 12 voxel setting unit, 13 point density calculation unit, 14 normal vector calculation unit, 15 estimation unit, 16 noise determination unit, 17 noise removal unit, 18 center of gravity calculation unit, 100,100A,100B,100C,100D voxel, 101 partial plane, 102 normal vector, 103,103A,103B,103C three-dimensional point, 104 plane, 200 input interface, 201 output interface, 202 processing circuit, 203 processor, 204 memory.
Claims
1. A data acquisition unit that acquires point cloud data, which is a set of three-dimensional points of a target area measured by a sensor; a voxel setting unit that sets a plurality of voxels in the target region; a point density calculation unit that calculates a point density, which is a density of the three-dimensional points within the voxel; A normal vector calculation unit that calculates a normal vector of a partial plane including a plurality of the three-dimensional points; an estimation unit that estimates the point cloud data corresponding to any one of a thickness portion, an edge, or a surface of an object based on a difference in the normal vector between the partial planes in the voxel; a noise determination unit that determines whether the point cloud data in a voxel is noise based on the point cloud data estimated to be one of a thickness portion, an edge, or a surface of the object and the point density; a noise removal unit that removes the point cloud data that is determined to be noise; A noise removal device comprising:
2. When a difference between the normal vectors of the partial planes in the voxel exceeds an estimation threshold, the estimation unit estimates that the point cloud data in the voxel corresponds to a thickness part of the object, and when the difference between the normal vectors is equal to or smaller than the estimation threshold, the estimation unit estimates that the point cloud data in the voxel corresponds to an edge or a face of the object.
2. The noise removal device according to claim 1.
3. The noise determination unit determines the point cloud data in the voxel estimated to correspond to a thickness portion of the object as noise when the point density of the voxel estimated to correspond to a thickness portion of the object is lower than the point density of the voxel adjacent to the voxel or the surrounding voxels including the adjacent voxels by a first determination threshold or more, and determines the point cloud data in the voxel estimated to correspond to either an edge or a surface of the object as noise when the point density of the voxel estimated to correspond to an edge or a surface of the object is lower than a second determination threshold.
3. The noise removal device according to claim 2.
4. a center of gravity calculation unit that calculates a distribution of measurement accuracy related information regarding the measurement accuracy of the point cloud data in the voxel, weights the three-dimensional points in the voxel based on the distribution of the three-dimensional points, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weighting.
4. The noise removal device according to claim 1, wherein the noise removal device is a noise removal device having a first noise removal function.
5. The center of gravity calculation unit calculates a center of gravity of the point cloud data in a voxel estimated to be one of a thickness portion, an edge, or a surface of the object, and thins out the point cloud data that does not exist within a certain range from the center of gravity in the voxel.
5. The noise removal device according to claim 4.
6. the center of gravity calculation unit calculates a center of gravity of the point cloud data in the voxel set by the voxel setting unit, and thins out the point cloud data that does not exist within a certain range from the center of gravity in the voxel; A series of processes are performed by the point density calculation unit, the normal vector calculation unit, the estimation unit, the noise determination unit, and the noise removal unit using the point cloud data that has been thinned out by the center of gravity calculation unit.
5. The noise removal device according to claim 4.
7. The center of gravity calculation unit weights the three-dimensional points in the voxel based on a distribution of distance measurement data, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights.
5. The noise removal device according to claim 4.
8. The center of gravity calculation unit weights the three-dimensional points in the voxel based on a distribution of reflection intensity data, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights.
5. The noise removal device according to claim 4.
9. The center of gravity calculation unit weights the three-dimensional points in the voxel based on a distribution of RGB data, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights.
5. The noise removal device according to claim 4.
10. The center of gravity calculation unit estimates a plane including the three-dimensional points, weights the three-dimensional points in the voxel based on a distribution of distances between the estimated plane and the three-dimensional points in the voxel, and calculates a center of gravity of the point cloud data in the voxel in accordance with the weights.
5. The noise removal device according to claim 4.
11. A noise removal method executed by a noise removal device, comprising: A step in which a data acquisition unit acquires point cloud data which is a set of three-dimensional points of a target area measured by a sensor; A voxel setting unit sets a plurality of voxels in the target region; A step of calculating a point density, which is a density of the three-dimensional points in the voxel, by a point density calculation unit; A normal vector calculation unit calculates a normal vector of a partial plane including a plurality of the three-dimensional points; an estimation unit estimating the point cloud data corresponding to any one of a thickness portion, an edge, or a face of an object based on a difference in the normal vector between the partial planes in the voxel; a noise determination unit determining whether or not the point cloud data in the voxel is noise based on the point cloud data estimated to be any one of a thickness portion, an edge, or a surface of the object and the point density; A noise removal unit removes the point cloud data determined to be noise. A noise removal method comprising:
12. Computer, a data acquisition unit that acquires point cloud data, which is a collection of three-dimensional points of a target area measured by a sensor; a voxel setting unit that sets a plurality of voxels in the target region; a point density calculation unit that calculates a point density, which is the density of the three-dimensional points within the voxel; a normal vector calculation unit that calculates a normal vector of a partial plane including a plurality of the three-dimensional points; an estimation unit that estimates the point cloud data corresponding to any one of a thickness portion, an edge, or a surface of an object based on a difference in the normal vector between the partial planes in the voxel; a noise determination unit that determines whether the point cloud data in a voxel is noise or not based on the point cloud data estimated to be any one of a thickness portion, an edge, or a surface of the object and the point density; a noise removal unit that removes the point cloud data determined to be noise; A program to function as a
Citation Information
Patent Citations
Three-dimensional data processing device, method and program
JP2012141758A
Cited By
Closed polyline generation method and system based on curtain wall
CN121010721A
A curtain wall-based closed polyline generation method and system
CN121010721B
