Point cloud processing apparatus, point cloud processing method, and program for point cloud processing

Abstract

To provide a technique that effectively utilizes non-immovable points in point cloud matching.SOLUTION: A point cloud processing apparatus 200 configured to perform matching between a first point cloud acquired at a first timing and a second point cloud acquired at a second timing which is different from the first timing includes: a data acquisition unit 201 which acquires the first point cloud and the second point cloud; a moving target point cloud extraction unit 203 which extracts a moving target point cloud from the first point cloud; a moving point cloud rearrangement unit 205 which rearranges the moving target point cloud in the first point cloud, in a position at the second timing; and a matching unit 206 which performs matching between the second point cloud and the rearranged first point cloud.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a technique for processing point clouds. [Background technology]

[0002] There is known a technique for obtaining a point cloud (also called point cloud data) by laser scanning or the like. A point cloud is a set of points obtained by measuring the three-dimensional positions of each point of a measurement target. This technique includes a technique for matching (identifying correspondences between) two or more point clouds (see, for example, Patent Document 1). For example, two point clouds from different viewpoints are matched and then integrated. In this case, the matching process is performed on the premise that the point clouds to be matched are fixed points (points that do not move over time).

[0003] Here, points that are not stationary (points that move and change position) cannot be identified as corresponding points and become noise. This noise has a negative impact on the accuracy of point cloud matching. For example, point clouds related to moving objects such as cars or moving people become the above-mentioned noise.

[0004] If the two point clouds to be matched are acquired at the same time (simultaneous), the above problem related to non-stationary points does not occur. However, if the two point clouds to be matched are acquired at different times, the above problem occurs. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2018-147065 Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide a technique for effectively utilizing non-stationary points in point cloud matching. [Means for solving the problem]

[0007] The present invention is a point cloud processing device that matches a first point cloud acquired at a first timing with a second point cloud acquired at a second timing different from the first timing, and includes a moving object point cloud extraction unit that extracts a point cloud of a moving object from the first point cloud, a rearrangement unit that rearranges the point cloud of the moving object in the first point cloud to a position at the second timing, and a matching unit that matches the second point cloud with the first point cloud after the rearrangement.

[0008] In one embodiment of the present invention, a plurality of consecutive images of the object of the first point cloud are taken in a specific time range including the first timing, and the moving object is identified based on the plurality of captured images obtained by the plurality of consecutive images. In another embodiment of the present invention, a moving point cloud displacement amount calculation unit is provided that calculates the amount of displacement of the point cloud of the moving object between the first timing and the second timing, and the rearrangement is performed based on the amount of displacement.

[0009] The present invention is a point cloud processing method for matching a first point cloud acquired at a first timing with a second point cloud acquired at a second timing different from the first timing, which method extracts a point cloud of a moving object from the first point cloud, rearranges the point cloud of the moving object in the first point cloud to its position at the second timing, and matches the second point cloud with the rearranged first point cloud.

[0010] The present invention is a point cloud processing program executed by a computer that matches a first point cloud acquired at a first timing with a second point cloud acquired at a second timing different from the first timing, the program causing the computer to extract a point cloud of a moving object from the first point cloud, rearrange the point cloud of the moving object in the first point cloud to its position at the second timing, and match the second point cloud with the rearranged first point cloud. [Effects of the Invention]

[0011] The present invention provides a technique for effectively utilizing non-stationary points in point cloud matching. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating the principle of the invention. [Figure 2] FIG. 2 is a block diagram of the surveying device. [Figure 3] FIG. 1 is a block diagram of a point cloud data processing device. [Figure 4] 10 is a flowchart illustrating an example of a processing procedure. DETAILED DESCRIPTION OF THE INVENTION

[0013] 1. First embodiment (principle) Using a camera-equipped laser scanner or a surveying device with a camera and laser scanner whose positions and orientations are known, captured images and laser scanned point clouds are obtained at certain intervals. Note that the camera's capture range and the laser scanned range overlap, and the correspondence between the two can be determined.

[0014] FIG. 1(A) shows an image captured at time T1, and FIG. 1(B) shows an image captured at time T2, which is after time T1. These two images capture an overlapping range. Although it is difficult to tell from the figure, the viewpoint positions (camera positions) of FIG. 1(A) and FIG. 1(B) are assumed to be slightly different. Of course, the viewpoint positions of FIG. 1(A) and FIG. 1(B) may be the same.

[0015] In the case of FIG. 1, a vehicle 10 is traveling from the back to the front. In FIG. 1(A), the vehicle 10 is shown in the distance, while in FIG. 1(B), it is shown closer. Note that everything other than the vehicle 10 is assumed to be stationary. Here, the point cloud obtained at time T1 (the point cloud corresponding to FIG. 1(A)) is referred to as the first point cloud, and the point cloud obtained at time T2 (the point cloud corresponding to FIG. 1(B)) is referred to as the second point cloud. Note that the point clouds are obtained using the laser scanner described above. The point clouds may also be obtained by stereo matching using images or MVS (multi-view stereo) matching.

[0016] In this case, everything other than the vehicle 10 is stationary, so the point cloud other than the vehicle 10 is a fixed point. On the other hand, since the vehicle 10 is moving, its position changes between time T1 and time T2, and the positions of the corresponding point clouds are also different.

[0017] Now, let us consider matching (identifying correspondence) between the first point cloud and the second point cloud. In this case, since the point cloud related to the vehicle 10 moves, without some kind of ingenuity, the correspondence cannot be identified and becomes noise. Therefore, the following ingenuity is used to address the above problem. Here, the point cloud related to the vehicle 10 at time T1 is relocated to its position at time T2. For example, by analyzing sequential photographic images, an image of the vehicle 10 moving relative to a stationary object such as a road is extracted. Then, the amount of movement of the point cloud of the vehicle 10 between times T1 and T2 is calculated. Thereafter, the point cloud of the vehicle 10 in the first point cloud is moved by the above-mentioned amount of movement, and the point cloud of the vehicle 10 is relocated. Finally, the first point cloud, in which the point cloud related to the vehicle 10 has been relocated to its expected position at time T2, is compared with the second point cloud, and the correspondence between the two is identified.

[0018] The point cloud of the vehicle 10 rearranged in the first point cloud becomes the position at time T2. Therefore, when matching the first point cloud with the second point cloud, the vehicle 10 also becomes a target of matching. According to this method, compared to when the points related to the vehicle 10 are not moved, the number of points that become noise is reduced and the number of points that become a target of matching is increased, thereby improving the accuracy of matching between the first point cloud and the second point cloud.

[0019] (Surveying equipment) 2 is a block diagram of a surveying instrument 100 according to an embodiment. The surveying instrument 100 performs laser scanning and image capture of a surveying object. The surveying instrument 100 can be used in a fixed state as a stationary type, or can be mounted on a mobile body and used while moving.

[0020] The surveying instrument 100 includes a laser scanner 101, a camera 102, a GNSS position measuring device 103, and an IMU 104. The laser scanner 101 performs laser scanning and obtains a point cloud composed of a laser scan point cloud. The form of the laser scanner 101 is not particularly limited.

[0021] In this example, the position and orientation of the surveying instrument 100 in the absolute coordinate system are unknown or inaccurate, and the point cloud is obtained in a machine coordinate system with the origin at an appropriate position within the surveying instrument 100. The machine coordinate system is a coordinate system specific to the surveying instrument 100 and is a coordinate system fixed to the surveying instrument 100.

[0022] The surveying instrument 100 may be used in a state where its position and orientation in an absolute coordinate system are known. In this case, the obtained point cloud will be described in the absolute coordinate system. The absolute coordinate system is a coordinate system used to describe map information, and is a coordinate system in which positions are described using latitude, longitude, and altitude. A local coordinate system can also be used as the coordinate system.

[0023] The camera 102 captures an image of the laser scan target. The GNSS position measurement device 103 is a device that measures position using a Global Navigation Satellite System, and measures position in an absolute coordinate system using navigation signals from navigation satellites. The IMU 104 is an inertial measurement unit that measures changes in attitude and acceleration. The relationship between the positions and attitudes of the laser scanner 101, camera 102, GNSS position measurement device 103, and IMU 104 is acquired in advance as known information.

[0024] It is also possible to acquire a point cloud using the camera 102. In this case, the surveying device 100 is moved while repeatedly taking photographs with the camera 101, and a stereo image consisting of multiple images taken from different viewpoints that capture overlapping areas is obtained, and a point cloud of the subject is obtained according to the principle of three-dimensional photometry. This technology is known as SLAM (Simultaneous Localization and Mapping) or SFM (Structure from Motion).

[0025] (Point Cloud Processing Device) 3 is a block diagram of a point cloud processing device 200 embodying the present invention. The point cloud processing device 200 performs processing related to matching two point clouds.

[0026] The point cloud processing device 200 is realized using a computer. Here, a software program for realizing the functions of each functional block shown in Fig. 2 is installed in the computer, and the point cloud processing device 200 is realized by executing the software program by the CPU of the computer.

[0027] It is also possible to configure part or all of the configuration in Figure 3 using dedicated hardware. It is also possible to realize part or all of the configuration in Figure 3 using a data processing server. It is also possible to configure the processing performed in the point cloud processing device 100 using distributed processing. In this case, the configuration in Figure 2 is understood as a point cloud processing system. It is also possible to configure the surveying device 100 so that part or all of the point cloud processing device 200 is provided.

[0028] The point cloud processing device 200 includes a data acquisition unit 201, a moving object extraction unit 202, a moving object point cloud extraction unit 203, a moving point cloud displacement calculation unit 204, a moving point cloud rearrangement unit 205, and a matching unit 206.

[0029] The data acquisition unit 201 acquires the surveying data obtained by the surveying instrument 100. Here, data on at least two point clouds and captured images to be matched is acquired. In this example, the point clouds are acquired by the laser scanner 101, and the captured images are acquired by the camera 102. Point clouds obtained by three-dimensional photo measurement (stereo photo measurement) can also be used. The viewpoint positions from which the at least two point clouds to be matched are obtained may be the same or different (FIG. 1 shows a case where they are different). The point clouds used may be point clouds obtained by laser scanning or photographing while moving. The captured images may be videos.

[0030] The point cloud obtained by the laser scanner 101 is obtained as distance and direction data from an appropriate point in the laser scanner 101 or the surveying instrument 100 as the origin for each scan point (each reflection point of the scanning light). Therefore, the obtained point cloud is obtained on a machine coordinate system based on the surveying instrument 100. When the point cloud is obtained while the surveying instrument 100 is moving, point cloud data is used in which each point is coordinate-converted to the machine coordinate system of the surveying instrument 100 at a specific time while moving, based on the measurement data of the GNSS position measuring device 103 and the IMU 104.

[0031] The moving object extraction unit 202 extracts moving objects from captured images. There are several methods for extracting moving objects. The first method is to compare multiple captured images obtained by multiple consecutive captures and extract images of moving objects against a stationary background (buildings or the ground) on the screen. Video can also be used in the first method. The second method is to use the results of laser scans performed with a time lag. There are also methods for extracting moving objects using radar or the Doppler effect. It is also possible to use AI-based recognition and identification technology to extract people or objects that appear to be moving.

[0032] There is also a method for extracting moving objects using the principles of three-dimensional photometry. There are two methods for this. The first method is based on the premise that the surveying instrument 100 moves while continuously capturing images using the camera 102. In this case, multiple captured images are obtained from different viewpoints (camera positions) in overlapping capture ranges, and a three-dimensional model of the captured object is created based on the principles of three-dimensional photometry. By observing the changes in this three-dimensional model over time, moving objects can be separated from non-moving objects, and images of the moving objects can be extracted.

[0033] The moving target point cloud extraction unit 203 extracts a point cloud of the target extracted by the moving target extraction unit 202. In this process, a superimposed image in which the captured image and the point cloud are superimposed is used to extract a point cloud corresponding to the image of the moving target extracted by the moving target extraction unit 202. In this process, for example, a point cloud of a moving vehicle is extracted.

[0034] The laser scanner 101 and the camera 102 have a known positional and postural relationship, and the camera 102 photographs the scanning range of the laser scanner 101. Therefore, by operating the laser scanner 101 and the camera 102 synchronously, it is possible to obtain a superimposed image in which the point cloud obtained by the laser scanner 101 is superimposed as points on the photographed image taken by the camera 102 and displayed. In this superimposed image, for example, data in which the point cloud obtained from the vehicle 10 is superimposed on the image of the vehicle 10 can be obtained. By using this superimposed image, when extracting a moving object (for example, the vehicle 10 in FIG. 1) from the photographed image of the camera 102, the point cloud corresponding to this moving object (for example, the point cloud of the vehicle 10) can be extracted. This process is performed in the point cloud extraction unit 203 for the moving object. Note that by extracting the point cloud corresponding to the moving object, it is also possible to extract the point cloud of the non-moving object.

[0035] The displacement amount calculation unit 204 for the moving point cloud calculates the displacement amount of each point constituting the point cloud of the moving object. This process is performed as follows. First, the acquisition times of the point clouds to be matched are set as T1 and T2 (T1 < T2). That is, the acquisition time of the first point cloud is T1, and the acquisition time of the second point cloud is T2. When there is a width in the scan time, the time in the middle of the scan period is adopted.

[0036] Here, the point cloud of the moving object at time T1 extracted by the point cloud extraction unit 203 for the moving object and the point cloud of the non-moving object (such as a wall, a building, a road, etc.) at time T1 are acquired from the first point cloud.

[0037] Here, the point cloud of the moving object and the non-moving point cloud at time T1 and at a time T1' (T1' = T1 + Δt) in the vicinity of time T1 are compared, and from the displacement amount of the three-dimensional position of the point cloud of the moving object with respect to the non-moving point cloud, the velocity vector of each point of the point cloud of the moving object is calculated. From this velocity vector, the displacement amount (amount of movement) between time T1 and time T2 of each point of the point cloud of the moving object is calculated. This process is performed in the displacement amount calculation unit 204.

[0038] The above velocity vector calculation may be performed more precisely. In this case, velocity vectors are calculated at multiple times on the time axis between times T1 and T2, and the displacement (movement) of each point in the point cloud of the moving object between times T1 and T2 is calculated by integrating these velocity vectors.

[0039] There is also a method for calculating the displacement of a moving point cloud using three-dimensional photogrammetry. This method assumes that the surveying device 100 moves while continuously capturing images using the camera 102. In this case, by obtaining multiple captured images from different viewpoints (camera positions) in overlapping capture ranges, a three-dimensional model of the captured object can be created based on the principles of three-dimensional photogrammetry. By observing the changes in this three-dimensional model over time (changes in the three-dimensional model over time), it is possible to distinguish between moving and non-moving objects, and extract images of the moving object.

[0040] Then, a point cloud corresponding to the image of the moving object is obtained by the function of the point cloud extraction unit 203 of the moving object. On the other hand, the velocity vector of the moving object is calculated from the change over time of the three-dimensional model, and the velocity vector of each corresponding point in the point cloud can also be calculated. In this way, the velocity vector of each point in the point cloud of the moving object is obtained. The displacement amount of the point cloud related to the moving object (the displacement amount between time T1 and T2) is calculated using this velocity vector.

[0041] The moving point cloud rearrangement unit 205 rearranges (moves) points relating to the moving object in the point cloud of interest to positions based on the above-mentioned movement amount. For example, in the point cloud (first point cloud) acquired at time T1, the positions of the points constituting the point cloud relating to the moving object are moved and rearranged by the above-mentioned movement amount.

[0042] That is, in the point cloud obtained at time T1, each point relating to the moving object is moved by the movement amount calculated by the point cloud displacement amount calculation unit 204. The movement destination is the planned arrival position at time T2.

[0043] For example, consider the case of FIG. 1. In this case, the point cloud obtained in the situation of FIG. 1(A) is the first point cloud, and the point cloud obtained in the situation of FIG. 1(B) is the second point cloud. Here, the position of the point cloud of vehicle 10 in FIG. 1(A) is moved to the position at the time of FIG. 1(B) using the calculation result of the displacement amount described above. In this case, the points other than the vehicle 10 in the first point cloud are stationary (do not move), and the point cloud of vehicle 10 is moved to a position at future time T2 (calculated predicted position) within the first point cloud.

[0044] In the point cloud obtained at time T2, the points of the moving object may be rearranged (moved) to their positions at time T1. In this case, the velocity vector of the point cloud of the moving object at time T2 is calculated based on the point cloud obtained at time T2, and the position of the object at time T1 is calculated from this velocity vector. Then, in the second point cloud, a process is performed to return the point cloud of the moving object from its position at time T2 to its position at time T1 (a process to return it to its past position).

[0045] The matching unit 206 matches two point groups (identifies the correspondence relationship). For the matching, known methods such as template matching and ICP (Iterative Closest Point) matching are used. At least one of the two point groups to be matched is a point group in which points related to a moving object have been rearranged.

[0046] (Example of processing) An example of processing performed in the point cloud data processing device 200 will be described below. Fig. 4 is a flowchart showing an example of the processing procedure. A program for executing the processing in Fig. 4 is stored in an appropriate storage medium and executed by the CPU of a computer constituting the point cloud data processing device 200.

[0047] Prior to processing, a first point cloud and a second point cloud of the measurement target are obtained at different times using the laser scanner 101. The position of the laser scanner 101 (surveying instrument 100) at the times when the two point clouds are obtained may be the same or different. One or both of the first point cloud and the second point cloud may be obtained while the surveying instrument 100 is moving or stationary.

[0048] Here, the first point cloud and the second point cloud overlap at least partially. The acquisition time of the first point cloud is defined as T1, and the acquisition time of the second point cloud is defined as T2. Assume also that the overlapping portion of the first point cloud and the second point cloud is captured by camera 102 at times T1 and T2. The captured image captured at time T1 is defined as the first captured image, and the captured image captured at time T2 is defined as the second captured image. Assume also that camera 102 continuously captures the overlapping region during a period including times T1 and T2. The capture interval is, for example, 0.1 ms (10 frames per second). It is also possible to perform continuous capture using a camera other than camera 102. It is also possible to capture a video and use the frame images as continuous photographic images.

[0049] First, the first point cloud data, the second point cloud data, the image data of the first captured image, the image data of the second captured image, and the image data of the continuous capture are acquired (step S101). Next, a moving object is extracted from the continuous captured images acquired during a period including time T1 (step S102). This process is performed by the moving object extraction unit 202.

[0050] Next, a point cloud of a moving object (for example, a point cloud of a moving vehicle) is extracted from the first point cloud (step S103). This process is performed by the moving object point cloud extraction unit 203. Here, the point cloud of the moving object at time T1 is extracted from the first point cloud using a superimposed image of the captured image at time T1 and the first point cloud.

[0051] Next, the displacement (amount of movement) of each point i of the moving point cloud between time T1 and time T2 is calculated (step S104). This process is performed by the displacement calculation unit 204 for the moving point cloud.

[0052] Next, each point i of the moving object point group in the first point cloud is displaced by the displacement amount calculated in step S104 and rearranged in the first point cloud (step S105). As a result, the moving object point group in the first point cloud is rearranged to its position at time T2. In this process, only the moving object point group in the first point cloud is displaced by the above displacement amount and rearranged. This process is performed by the moving point cloud rearrangement unit 205.

[0053] Next, the matching unit 206 identifies (matches) the correspondence between the first point cloud and the second point cloud after the rearrangement of the point cloud of the moving target (step S106).

[0054] By matching the first point cloud with the second point cloud, the correspondence between the first point cloud and the second point cloud is identified, making it possible to integrate the first point cloud and the second point cloud. Multiple point clouds obtained at different times can be matched and integrated in a similar manner. If any of the point clouds to be integrated have coordinates in an absolute coordinate system, the integrated point cloud can be described in the absolute coordinate system.

[0055] (superiority) When matching multiple point clouds (identifying correspondences), it is possible to match point clouds that include moving objects (for example, moving vehicles). This allows for effective use of points that are not stationary in point cloud matching. In addition, since the number of points used for matching increases, matching accuracy can be improved.

[0056] 2. Second embodiment In both the first point cloud and the second point cloud, it is also possible to rearrange each point constituting the "point cloud of the object to be moved". In this case, assuming that the acquisition time of the first point cloud is T1, the acquisition time of the second point cloud is T2, and T1 < T < T2, each point of the "point cloud of the object to be moved" in the first point cloud is rearranged (moved) to the position at time T, and each point of the "point cloud of the object to be moved" in the second point cloud is rearranged (moved) to the position at time T. Then, the first point cloud and the second point cloud are matched. Except for performing the rearrangement in both the first point cloud and the second point cloud, it is the same as other embodiments.

[0057] 3. Third Embodiment The present invention can also be applied to SLAM and Sfm. For example, consider the case where a large number of stereo images are obtained by repeatedly taking pictures of overlapping ranges while moving a single camera, and point cloud data of the shooting object is obtained. In this case, assume that the first point cloud is obtained by the first continuous shooting, and the second point cloud that overlaps with the first point cloud is obtained by the second continuous shooting.

[0058] For example, assume that a UAV flies straight north for 100 m, turns 180°, and flies 100 m south, and during this process, the camera mounted on it continuously and repeatedly takes pictures of the area below. In this case, the first point cloud obtained when flying north and the second point cloud obtained when flying south are obtained. Here, these two point clouds overlap. The present invention is used for matching the first point cloud and the second point cloud.

[0059] 4. Fourth Embodiment For example, assume that a camera mounted on a UAV continuously takes pictures of a certain area, and 30 shooting images of that area are obtained. Here, assume that these 30 shooting images partially overlap. Here, if the shooting interval is 0.1 second, the time difference between the shooting times of the 5th and 25th shooting images is 2 seconds.

[0060] Here, the principles of three-dimensional photometry are used to obtain the first point cloud of the object captured in the fifth captured image. For example, a stereo image is created using the fifth captured image and several captured images before and after it, and the first point cloud of the captured object is obtained. In addition, a similar method is used to obtain the second point cloud of the object captured in the 25th captured image.

[0061] On the other hand, using the fifth captured image and several consecutive captured images before and after it, an image of a moving object among the objects captured in the fifth captured image is extracted. Next, a point cloud of the moving object in the fifth captured image is obtained. Furthermore, based on this point cloud of the moving object, the amount of displacement of the moving object between the time the fifth captured image was acquired and the time the 25th captured image was acquired is calculated. This calculation of the amount of displacement is performed using the method described with respect to the moving point cloud displacement amount calculation unit 204.

[0062] Then, using the above displacement amount, the point cloud of the moving object in the point cloud of the object captured in the fifth captured image (first point cloud) is moved to the position at the time when the 25th captured image was acquired (rearrangement of point clouds).The point cloud of the object captured in the fifth captured image after rearrangement of the point cloud of the moving object (first point cloud) is then matched with the point cloud of the object captured in the 25th captured image (second point cloud).

[0063] 5. Fifth Embodiment The following method can also be used to extract moving objects. For example, in the case of laser scanning, which takes a certain amount of time to scan, the point cloud of a moving object will be distributed in a manner that flows in the direction of movement. If an image is taken while the laser scanner is in use, the degree of flow of this point cloud can be evaluated by comparing it with the captured image, and the point cloud of the moving object can be extracted based on that.

[0064] Furthermore, when photographing a moving subject, if the exposure time is set long, the photographed image of the moving subject will appear to flow across the photographed screen, and the image of the moving subject can be extracted from this flowing image.

[0065] 6. Sixth Embodiment A portion of the point cloud of the moving object is adopted as representative points, and these representative points are used for matching. It is not necessary to use all points related to the moving object. For example, a number of points that roughly characterize the moving object are extracted as a representative point cloud, and this representative point cloud is used for matching. This method reduces the number of points to be relocated, thereby reducing the amount of calculation.

[0066] 7. Seventh Embodiment For example, images of a moving target at time T1 are extracted by continuous shooting using a depth camera. At this time, the three-dimensional velocity vector (three-dimensional movement direction and speed) of the moving target at time T1 is obtained by analyzing the images captured by the depth camera. Then, based on this velocity vector, the predicted three-dimensional position of the moving target at time T2 is calculated. The points of the moving target in the first point cloud are moved to this predicted three-dimensional position. Thereafter, the first point cloud obtained by moving the moving target to the position at time T2 is matched with the second point cloud.

[0067] In cases where it is not necessary to take into account the movement in the depth direction from the measurement viewpoint, such as when capturing images of the ground below from a flying UAV to obtain a point cloud, a normal camera (a camera that captures two-dimensional image information) can be used instead of a depth camera. In this case, the velocity vector at time T1 in the captured image is obtained, and this velocity vector is used to calculate the predicted position of the moving object at time T2.

[0068] It is also possible to extract a moving object by taking continuous photographs with a normal camera, and then use laser positioning or radar to detect the object's movement in the depth direction, thereby determining the three-dimensional velocity vector of the moving object.

[0069] 8.Other As a means for detecting the movement of a moving object, a laser measurement means prepared separately from the laser scanner, a radar, an ultrasonic measurement means, a measurement means using the Doppler effect, or the like can also be used.

[0070] For objects with a fixed moving speed, the amount of movement can be predicted if the time difference is known. In such cases, the amount of movement of the object can be predicted based on information about the known moving speed, and the point cloud can be moved by that amount. Examples of objects with a fixed moving speed include railways, automated transportation systems, ropeways, gondolas, cable cars, mountain climbing lifts, ski lifts, moving walkways, and escalators.

[0071] For an object that moves while rotating or an object that changes direction during movement, there may be no corresponding points at time T1 and time T2. In this case, the increase in error can be suppressed by removing the moving object as noise. Also, point clouds of objects with large prediction errors in the amount of movement (for example, objects with large changes in speed) may be removed as noise.

[0072] The point cloud obtained at time T1 and the point cloud obtained at time T2 may both be obtained in an absolute coordinate system. In this case, by performing matching taking into account the moving object, it is possible to further improve the matching accuracy between the two.

Claims

1. A point cloud processing device that matches a first point cloud acquired at a first timing with a second point cloud acquired at a second timing different from the first timing, a moving object point cloud extraction unit that extracts a moving object point cloud from the first point cloud; a rearrangement unit that rearranges the point cloud of the moving object in the first point cloud to a position at the second timing; a matching unit that performs matching between the second point cloud and the first point cloud after the rearrangement; A point cloud processing device comprising:

2. A plurality of consecutive photographs of the object of the first point cloud are taken within a specific time range including the first timing; The point cloud processing device according to claim 1 , wherein the moving object is identified based on a plurality of captured images obtained by the plurality of consecutive captures.

3. a moving point cloud displacement amount calculation unit that calculates a displacement amount of the point cloud of the moving object between the first timing and the second timing, The point cloud processing apparatus according to claim 1 , wherein the rearrangement is performed based on the amount of displacement.

4. A point cloud processing method for matching a first point cloud acquired at a first timing with a second point cloud acquired at a second timing different from the first timing, comprising: extracting a point cloud of a moving object from the first point cloud; rearranging the points of the moving object in the first point cloud to positions at the second timing; A point cloud processing method for matching the second point cloud with the first point cloud after the rearrangement.

5. A program executed by a computer to match a first point cloud acquired at a first timing with a second point cloud acquired at a second timing different from the first timing, To the computer extracting a point cloud of a moving object from the first point cloud; rearranging the points of the moving object in the first point cloud to positions at the second timing; a point cloud processing program that performs matching between the second point cloud and the first point cloud after the rearrangement;

Images