Point cloud registration

Abstract

To provide a method of point cloud registration.SOLUTION: A computer-implemented method comprises: determining a 2D overlapping portion between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the 2D overlapping portion corresponding to an overlapping region between the source and target geographical areas; dividing the 2D overlapping portion into a plurality of cells and determining at least one overlap cell which includes at least a threshold number of points from the source 2D projection and at least a threshold number of points from the target 2D projection; performing a comparison process for the overlap cell and selecting a best matching target set; determining a transformation between the points of the source set and the points of the best matching target set or vice versa; and applying the transformation to register the source 3D point cloud to the target 3D point cloud or vice versa.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present application relates to point cloud registration, and in particular to a computer-implemented method, a computer program, and an information programming apparatus. [Background technology]

[0002] The task of point cloud registration, also known as point cloud alignment, is the process of registering multiple point clouds in a common coordinate system or aligning one or more point clouds with or to another point cloud. Registration may involve estimating the transformation required for one point cloud to align it with another and applying the estimated transformation. Point cloud registration allows larger point clouds to be constructed from multiple smaller point clouds.

[0003] In view of the above, a point cloud registration method is desirable. Summary of the Invention

[0004] According to one embodiment of a first aspect, a computer-implemented method is disclosed herein, the computer-implemented method comprising: determining a 2D overlap between a source 2D (two-dimensional) projection of a source 3D (three-dimensional) point cloud and a target 2D projection of a target 3D point cloud, wherein the source 3D point cloud and the target 3D point cloud represent a source geographical area and a target geographical area, respectively; the 2D overlap corresponds to an overlap region between the source geographical area and the target geographical area; dividing the 2D overlap into a plurality of cells; determining at least one overlap cell comprising at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection (or determining at least one cell comprising at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection as at least one overlap cell); performing a comparison process for the (or the) overlap cell; and evaluating the comparison process (for the overlap cell). determining a plurality of target set points, each having a first (target) set of points of the target 3D point cloud that fall within the overlap cell (in the target 2D projection) and a second target set of points and a further (target) set of points of the target 3D point cloud that fall within cells adjacent / surrounding the overlap cell (in the target 2D projection); determining a source set of points having points of the source 3D point cloud that fall within the overlap cell (in the source 2D projection); comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; and selecting a best match target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set;applying the transformation (matrix) (to the source 3D point cloud) to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud.

[0005] According to one embodiment of the second aspect, a computer-implemented method is disclosed herein, the computer-implemented method comprising: estimating a 3D overlap volume between a source 3D (three-dimensional) point cloud and a target 3D point cloud based on an overlap region between a source geographic area and a target geographic area, wherein the source 3D point cloud and the target 3D point cloud represent the source geographic area and the target geographic area, respectively; dividing the 3D overlap volume into a plurality of cells; determining at least one overlap cell including at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud (or determining at least one cell including at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud as at least one overlap cell); performing a comparison process for the (or the) overlap cell (for the overlap cell); the comparison process comprises: determining a plurality of target sets of points, each having a first (target) set of points of the target 3D point cloud that are within the overlap cell and a second (target) set of points and a further (target) set of points of the target 3D point cloud that are within cells adjacent / surrounding the overlap cell; determining a source set of points having points of the source 3D point cloud that are within the overlap cell; comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; selecting a best match target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set; and applying the transformation (matrix) (to the source 3D point cloud);registering the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud.

[0006] Features relating to any aspect / embodiment may be applied to any other aspect / embodiment. [Brief explanation of the drawings]

[0007] By way of example only, reference may be made to the drawings in which: [Figure 1] 1 is a diagram useful for understanding point cloud registration. [Figure 2] FIG. 10 is a diagram showing a comparative method. [Figure 3] 1 is a flowchart illustrating a method. [Figure 4] FIG. [Figure 5] FIG. [Figure 6] 1 is a flowchart illustrating a method. [Figure 7] 1 is a flowchart illustrating a method. [Figure 8] 1 is a flowchart illustrating a method. [Figure 9] 1 is a flowchart illustrating a method. [Figure 10] 1 is a flowchart illustrating a method. [Figure 11] 1 is a flowchart illustrating a method. [Figure 12] 1 is a flowchart illustrating a method. [Figure 13] 1 is a flowchart illustrating a method. [Figure 14] FIG. 1 shows an apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0008] As mentioned above, the task of point cloud registration is the process of aligning multiple point clouds in a common coordinate system. The point cloud to be aligned can be called the source point cloud, and the other point cloud (with which the source point cloud will be aligned) can be called the reference or target point cloud.

[0009] Point cloud registration involves estimating the transformation required for the source point cloud to align with the target point cloud, and applying the transformation to the source point cloud to bring the source point cloud into the coordinate system of the target point cloud, i.e., for alignment with the target point cloud.

[0010] In one example application, by registering successive point cloud frames acquired from sensors such as LiDAR (Light Detection and Ranging), robots / vehicles can accurately estimate their position and orientation and navigate autonomously. Further applications of 3D point cloud registration include autonomous driving, general 3D mapping and modeling, robotics, unmanned aerial vehicles, and 3D map generation and / or real-time updates (e.g., on edge devices).

[0011] Figure 1 shows a 2D representation of point clouds S, T, and R. Figure 1 illustrates the concept of point cloud registration, where a source point cloud S is registered with a target point cloud T to obtain a resulting point cloud R.

[0012] FIG. 2 illustrates a comparative method. Elements (labeled 10, 20, 50, and 60) can be considered as method steps and / or modules for performing operations. In feature extraction element 10, a source point cloud and a target point cloud are received and features are extracted therefrom. A voxelization element 20 segments the point clouds. A matching element 50 performs matching between portions of the source point cloud and points of the target point cloud. A registration element 60 registers the source point cloud with the target point cloud based on the matching performed by the matching step. The output of the registration step is a stitched point cloud having points from the source point cloud and points from the target point cloud. Comparative Method 2 described below can be considered a more specific implementation of the general comparative method shown in FIG. 2.

[0013] To aid in understanding the present disclosure and its context, further comparative methods 1-4 are briefly described below, and are only briefly described as comparative methods 1-4 are intended only to highlight examples of comparable techniques, such as the use of deep neural networks, supervised learning, and application-specific techniques.

[0014] Comparative Method 1 has a two-stage approach. In the first stage, a point-wise feature extractor is used to extract features from each point in each point cloud (source and target). These features are then used to construct a patch-wise representation of each point cloud. In the second stage, a patch-wise registration algorithm is used to register the two point clouds.

[0015] More specifically, the comparative method 1 utilizes a point-wise learner called an equivariant fully convolutional network (EFCN) to predict rotation-invariant keypoints and rotation-equal orientations. The detected keypoints are then sent to a patch-wise embedder designed to learn features specific to the selected keypoints. Feature matching is performed using the features generated by the patch-wise embedder. An inlier generator module is used to search for inliers among the initial matches. Finally, a 3D cylindrical convolutional network (3DCCN) is used to estimate a transformation matrix. This transformation matrix is applied to the source point cloud to register the source point cloud with the target point cloud.

[0016] In short, comparative method 1 focuses on learning the transformation matrix from the full point cloud using supervision on a deep neural network (DNN).

[0017] In comparative method 2, source and target point clouds serve as inputs. Feature descriptors are computed for both point clouds, and then both point clouds are partitioned into 3D grids. A two-step optimization is then used to find matching cells between the grids. First, a voting strategy is used to measure the similarity between two given cells based on the feature descriptors. Second, a graph matching algorithm is used to determine corresponding points. That is, the graph matching algorithm hierarchically refines corresponding cells until a point-to-point correspondence is obtained. Outliers are filtered out to identify putative matches. Finally, three points are randomly sampled from the putative matches to compute the final transformation matrix that can be used to register the source point cloud with the target point cloud.

[0018] In essence, Comparative Method 2 divides the point cloud into a grid and uses complex optimization to match the grids. As noted above, Comparative Method 2 can be considered an example of the more general Comparative Method shown in FIG.

[0019] Comparative Method 3 involves learning the posterior probability distribution of Gaussian mixture models (GMMs) from the source and target point clouds. This distribution is then used to predict the distribution-level correspondence between the two point clouds. Comparative Method 3 also handles partial point cloud registration using the Sinkhorn algorithm.

[0020] More specifically, comparative method 3 uses a shared weight network to extract point-level features and overlap scores of the source and target point clouds, and a cluster head module uses these features to calculate a probability matrix, which is then used to estimate parameters of Gaussian mixture models (GMMs). Subsequently, cluster-level and point-level matching modules are used to estimate correspondences and facilitate the calculation of a transformation matrix, which can be used to register the source point cloud with the target point cloud. The shared weight network is trained using local contrastive loss, self-consistency loss, and cross-consistency loss.

[0021] In summary, comparative method 3 focuses on unsupervised learning and performs full point cloud processing.

[0022] Comparative Method 4 is directed to aligning a point cloud obtained by imaging a forest-type landscape using a UAV (unmanned aerial vehicle) with a point cloud obtained using terrestrial LiDAR imaging the forest-type landscape. This method uses a divide-and-conquer strategy to split the registration problem into two subproblems: vertical alignment (ground alignment) and horizontal alignment (canopy alignment). Vertical alignment is achieved based on ground alignment, which is achieved by finding the transformation relationship between the normal vectors of the ground point cloud and the horizontal plane. Horizontal alignment is achieved by canopy projection image matching, which matches two binary images of projected plant points using canopy shape context features. Canopy alignment includes canopy point projection, canopy binary image preprocessing, canopy contour and keypoint extraction, and image matching.

[0023] In summary, comparative method 4 focuses on a divide-and-conquer strategy for vertical (ground) alignment and horizontal (canopy) alignment.

[0024] 3 is a flowchart of a method having steps S10, S20, S30, S40, S50, and S60 for registering a source 3D point cloud to a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area.

[0025] Step S10 includes determining a 2D overlap (sometimes referred to as an overlap) between a source 2D projection of the source 3D point cloud and a target 2D projection of the target 3D point cloud. The 2D overlap corresponds to an overlap region between the source geographic area and the target geographic area. The target 3D point cloud and the source 3D point cloud may be referred to as a target point cloud and a source point cloud.

[0026] The method may further comprise, in a preceding step, determining source and target 2D projections of the source and target 3D point clouds. The 2D projection of the 3D point cloud may be found, for example, by setting the z coordinate values for all points to zero. For example, an orthogonal projection may be used to determine the 2D projections.

[0027] A 2D projection is a 2D projection determined by "removing" the z coordinate from all points, where the "z" coordinate corresponds to height and the other coordinates correspond to the plane of the geographic area. For example, the coordinates / dimensions of a 2D projection correspond to latitude and longitude.

[0028] Step S20 includes dividing the overlapping portion into a plurality of cells.

[0029] Step S30 includes determining at least one "overlap cell" among the cells that includes at least a threshold number of points from the source 2D projection and at least a threshold number of points from the target 2D projection. The threshold number of points is 1, 2, or 3 in some implementations of the method. For example, it may be considered that at least three point pairs in the two point clouds are required to determine the rigid transformation matrix. Different threshold numbers, e.g., a first threshold number and a second threshold number, may be used for the points of the source point cloud and the points of the target point cloud.

[0030] Step S40 includes performing a comparison process on the overlap cell to determine the set of points of the target point cloud that best matches the set of points of the source point cloud within the overlap cell.

[0031] Step S50 includes determining a transformation (e.g., a transformation matrix) between the set of source cloud points and the best matching set of target cloud points in the overlap cell, or vice versa. Step S60 includes applying the transformation (e.g., a transformation matrix) for registration.

[0032] In some implementations of the method, step S50 includes determining a transformation between the set of source point clouds within the overlap cell and the best-match set of target point clouds, and step S60 includes applying the transformation to the source point cloud for registration of the source point cloud to the target point cloud. In some implementations of the method, step S50 includes determining a transformation between the best-match set of target cloud points and the set of source cloud points within the overlap cell, and step S60 includes applying the transformation to the target point cloud for registration of the target point cloud to the source point cloud. That is, regardless of the convention that the source point cloud is the point cloud to be registered to the target point cloud, the transformation may be determined and / or applied inversely. A transformation matrix may be determined for the transformation between one of the point clouds and the other, and its inverse may be applied to register the other point cloud to one of the point clouds.

[0033] The comparison process in step S40 is as follows: determining a plurality of target set points, each having a first set of points of the target 3D point cloud that fall within the overlapping cell (which may be referred to as a first target set), and a second and further set of points of the target 3D point cloud that fall within cells that surround / adjacent to the overlapping cell (which may be referred to as a second target set and a further target set); determining points of the source set that have points of the source 3D point cloud that fall within overlap cells; comparing the source set with each of a plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; Selecting a best-match target set based on the similarity scores; and It has.

[0034] Each similarity score quantifies the similarity between a source set and an associated target set.

[0035] In the above, when the concept of a point of a 3D point cloud falling within a particular cell is used, this refers to a point in the 3D point cloud that falls within a particular cell in a 2D projection of the 3D point cloud, in other words, a point in the 3D point cloud whose representing point in the 2D projection of the 3D point cloud falls within a particular cell, or a point in the 3D point cloud whose 2D projection falls within a particular cell.

[0036] In some implementations of the method, the cells surrounding the overlapping cell used to determine the points of the second and further target sets are determined by finding the Moore neighborhood of the overlapping cell, i.e., the overlapping cell and its surrounding cells form a Moore neighborhood with the overlapping cell as its center cell. In this case, the target set is determined as the set of points of the target 3D point cloud that fall within each cell in the Moore neighborhood with the overlapping cell as its center cell (which includes the overlapping cell only, and the points of this target set correspond to the "first target set" referred to above).

[0037] The method may include determining a plurality of overlapping cells and performing a comparison process for each overlapping cell (or a plurality of overlapping cells). In such a case, step S50 includes determining points of a final target set having points of at least one best-match target set among the best-match target sets, determining points of a final source set having points of at least one corresponding source set (i.e., corresponding to the at least one best-match target set), and determining a transformation matrix between the points of the final source set and the points of the final target set. The final target set may have points from multiple (or the multiple) best-match target sets, and thus the final source set may have points of multiple corresponding source sets. The best-match target set from which points to be added to the final target set are taken may be selected from among the multiple best-match sets based on their similarity scores; for example, a certain number of best-match sets with the highest similarity scores may be selected, and / or any (or some) best-match sets above a threshold similarity score may be selected.

[0038] In some implementations, comparing the source set to each of the multiple target sets includes a shape comparison and / or a color comparison. The shape comparison includes determining a source convex hull of the points of the source set, determining multiple target convex hulls of the points of each of the multiple target sets, and comparing the source convex hull to each target convex hull. The color comparison includes comparing the color of the points of the source set to the color of the points of the target set.

[0039] Comparing the source convex hull to the target convex hull may include determining a first distribution of dihedral angles of the source convex hull of points in the source set and a second distribution of dihedral angles of the target convex hull of points in the target set, and calculating a difference (e.g., a Kullback-Leibler divergence) between the first and second distributions of dihedral angles. Then, a contribution may be added to the similarity score for the target set based on the calculated difference.

[0040] Comparing the colors of the points of the source set with the colors of the points of the target set may include determining a first distribution of color intensities of the points of the source set (e.g., as a histogram of RGB or CYMK colors or other intensities) and a second distribution of color intensities of the points of the target set (e.g., as a histogram of RGB or CYMK colors or other intensities), and calculating a distance / similarity measure (e.g., Bhattacharyya distance) between the first and second distributions. Then, a contribution may be added to a similarity score for the target set based on the calculated distance / similarity.

[0041] 4 illustrates a method of the present disclosure. Elements labeled 110, 120, 140, 150, and 160 in FIG. 4 can be considered method steps or modules configured to perform actions. The steps / actions in the method of FIG. 4 can be considered to correspond to the method steps of FIG. 3.

[0042] The overlap finder 110 receives the source point cloud and the target point cloud and finds the overlap between their 2D projections, which can be thought of as corresponding to step S10. The overlap voxelizer 120 divides the overlap into cells, which can be thought of as corresponding to step S20. The similarity score determiner 140 determines overlap cells and, for each overlap cell, compares the set of points from the source point cloud that fall within the overlap cell with each set of points from the target point cloud that fall within a Moore neighborhood centered on the overlap cell, and determines a similarity score corresponding to each point in the target set (and each corresponding Moore neighborhood cell).

[0043] The matching component 150 determines, for each overlap cell, the cell in the corresponding Moore neighborhood with the highest similarity score. Components 140 and 150 together can be considered to correspond to steps S30 and S40. The registration component determines a transformation between the points of the source point cloud and the points of the target point cloud that fall within the overlap cell and the best match cell, respectively, and applies the transformation to the source point cloud to register the source point cloud with the target point cloud. The registration component 160 can be considered to correspond to steps S50 and S60. A stitched point cloud comprising the source point cloud and the target point cloud is output from the registration component 160.

[0044] Figure 5 is another diagram illustrating the method of the present disclosure. The elements labeled 220, 240, 250, and 260 in Figure 5 can be considered method steps or modules configured to perform operations. It is understood that Figure 5 is similar to Figure 4, with the difference being that Figure 5 includes dashed boxes with additional information, which will be described below, and that Figure 5 includes a division element 220 rather than elements 110 and 120 in Figure 4. Additionally, the remaining elements in Figure 5 have similar reference numbers to elements 140, 150, and 160 in Figure 4, and descriptions of these elements will be omitted for the sake of brevity.

[0045] Further information shown in Figure 5 by dashed boxes represents how data at certain steps may be stored according to some implementations. The source point cloud and target point cloud may be stored as LAS files. The LAS (LASer) format is a file format designed for the exchange and archiving of lidar point cloud data. For example, point cloud data may have x, y, z coordinates, RGB values, scale, offset, etc. An array of points and colors for each point cloud (Points:Array(N) for the target point cloud) T ,3) and color: array (N T ,3), as well as the point array (N S ,3) and color: array (N S ,3)) can be extracted / constructed from the LAS data and used in subsequent stages. Instead of the input being LAS data, the input can also be in the format of an array.

[0046] After the partitioning step 220 (which involves, for example, determining the overlap between the 2D projections and partitioning the overlap, as described for elements 110 and 120 in FIG. 4), the output is in the form of a data structure with grid-point mappings for each point cloud. For example, the length of the list for "points" and the length of the list for "colors" are both the number of cells (# cells) into which the overlap was partitioned. Each element of the list has a size (nj ,3), where j=S,T (S denotes a point in the source point cloud and T denotes a point in the target point cloud). That is, the entire list has a size of (n i j ,3) is an array form, where i = 1 to # cells.

[0047] The determination of overlapping cells (e.g., including at least one point from the source point cloud and at least one point from the target point cloud) may be performed in the segmentation element 220 or the similarity score determination element 240. The condition for determining the set of overlapping cells C is C={k:n k S >0 & n k T >0}.

[0048] The output of the similarity score determination element 240 in this implementation is an array that holds similarity score data for each overlapping cell. For example, assuming there are K overlapping cells, the array would be of size (K,2). For example, k ,1) = maximum match score for overlapping cell k, (cell k ,2) = the index of the best matching Moore neighbor cell for overlap cell k.

[0049] The output of matching element 250 in this implementation is an array of points from both point clouds. That is, points from the source point cloud that fall within an overlap cell are added to array M, and points from the target point cloud that fall within the corresponding best match cell are added to array M'. Points may be added for all overlap cells, or for a subset that have a similarity score above a threshold or selected in other ways as described above. These arrays are then combined to determine the number of points from the source point cloud and the target point cloud that are added to arrays M and M', respectively. S and P T As, M=Array(P S,3) M'=Array(P T ,3) where P S ≪N S , and P T ≪N T That is, the number of points in arrays M and M' can generally be much less than the total number of points in the point cloud.

[0050] The output of the registration component 260 is a stitched point cloud, which, like the input above, is in the form of an array of point positions and colors, i.e., points:array(N R ,3) and color: array (N R ,3), where the index R denotes the stitched point cloud.

[0051] FIG. 6 is a flowchart illustrating an implementation of how to divide the overlapping portion into cells. The method illustrated in FIG. 6 includes steps S10, S15, S20′, and S30. Steps S10 and S30 are the same as the steps of the same labels in FIG. 3 and are included here for context, so redundant description will be omitted. Step S20′ corresponds to step S20 in FIG. 3, except that here the division into cells is based on a grid size (this is not necessarily the case in step S20, but may be). Step S15 includes setting a grid size. Then, step S20′ uses the grid size to divide the overlapping portion into cells, i.e., step S20′ includes dividing the overlapping portion into uniform 2D orthogonal cells of the grid size. The method of FIG. 6 can be employed in the method of FIG. 3 or any other method / methodology described herein to divide the overlapping portion into cells.

[0052] The grid size is set so that the area of the resulting cell is greater than the area of the overlap divided by the total number of points in the overlap. The grid size may be selected taking into account the average number of points per cell. For example, the desired average number of points per cell may be used to determine the total number of cells, and the grid size may be determined based on that number. Alternatively, the desired average number of points per cell may be used to determine the area of each cell, and the grid size may be determined based on that area.

[0053] 7 is a flowchart illustrating the comparison process in one specific implementation of step S40 (or any similarity score determination step / element described herein), i.e., the method illustrated in FIG. 7 may be employed in step S40.

[0054] The method of Figure 7 has multiple threads 0 to N-1, where N is the number of overlapping cells, i.e., each thread corresponds to a comparison for a given overlapping cell.

[0055] Thread 0 has steps S41 through S46. The other threads also have these steps, but for different overlapping cells. Thread 0 has operations performed on each overlapping cell c0 (specifically, its source cloud point) and corresponding Moore neighborhood cell c0' (specifically, its target cloud point, where the Moore neighborhood cell includes the overlapping cell as the center cell in the Moore neighborhood), and thread N-1 has operations performed on overlapping cell c0'. N-1 and the corresponding Moore neighborhood cell c N-1 For simplicity, we will describe these steps using the reference c for the overlapping cells and c' for each Moore neighboring cell.

[0056] Step S41 includes finding the convex hull H of the source cloud points within the overlap cell c. Step S42 includes finding the Moore neighborhood cell c' of the overlap cell c (which includes the overlap cell as the center cell of that Moore neighborhood). Step S43 includes, for each Moore neighborhood cell c', finding the convex hull H' of the target cloud points within the Moore neighborhood cell c'. Step S44 includes comparing the convex hull H to each convex hull H' to determine a shape score for each Moore neighborhood cell c'.

[0057] Step S45 includes determining a color score for each Moore neighborhood cell c' by comparing the color of the target cloud point in that Moore neighborhood cell c' with the color of the source cloud point in the overlap cell c. The color score may alternatively be determined by comparing intensity (e.g., LiDAR intensity) rather than color, and may be specifically referred to as an intensity score.

[0058] Step S46 comprises calculating a similarity score for each Moore neighbourhood cell by combining the shape score and the colour score, which may comprise summing, and the shape score and the colour score may be weighted in the summation.

[0059] In the above description, the phrase "source cloud point within an overlap cell" means, for example, a point in the source point cloud whose 2D projection is within the overlap cell, or a point in the source point cloud that corresponds to a point in the 2D projection of the source point cloud that falls within the overlap cell. Similar descriptions apply to similar phrases (e.g., regarding Moore neighboring cells and target point clouds).

[0060] It will be appreciated that there are similarities between steps S41 through S46 and the description of possible implementations of step S40 described with reference to Figure 3. Mechanisms may be exchanged between the description of Figure 7 and the description of step S40. For example, the Kullback-Leibler divergence may be calculated as part of step S44 and / or a histogram may be used in step S45.

[0061] FIG. 8 is a flowchart illustrating an implementation of steps S50 and S60 of FIG. 3, which includes steps S352, S354, and S360.

[0062] Step S352 involves collecting points from the best-match pair of cells c and c' and adding them to sets M and M', respectively. That is, following a comparison for overlap cell c, points in the source point cloud that fall within overlap cell c are added to set M, and points in the target point cloud that fall within the best-match Moore neighbor cell c' are added to set M'. This is done for all overlap cells, or for a subset (or a subset may be selected in another manner, as described above) whose similarity score with the corresponding best-match cell exceeds a threshold.

[0063] Step S354 comprises finding a correspondence and a transformation matrix between sets M and M' (or, more simply, finding a transformation matrix) using an ICP (iterative closest point) algorithm. Step S360 comprises transforming the source point cloud using the transformation matrix.

[0064] As indicated by the reference "A" in FIGS. 7 and 8, the method of FIG. 8 may be a continuation of the method of FIG. 7, but this is not required.

[0065] 9 is a flowchart showing an implementation of step S10, which includes steps S12, S14, and S16. The implementation shown here is not required. Step S12 includes finding the minimum and maximum latitude and longitude of the source point cloud. For example, the source 2D projection can be used in this step. The minimum and maximum latitude and longitude of the source point cloud are found by min S lat , max S lat , min S lon , and max S lon It is written as follows.

[0066] Step S14 comprises finding the minimum and maximum latitude and longitude of the target point cloud. For example, the target 2D projection can be used in this step. The minimum and maximum latitude and longitude of the target point cloud are found by min T lat , max T lat , min T lon , and max T lon It is written as follows.

[0067] Step S16 consists in finding the maximum and minimum values among the following pairs of values determined in steps S12 and S14: max(min S lat ,min T lat ) max(min S lon ,min T lon ) min(max S lat ,max T lat ) min(max S lon ,max T lon ).

[0068] That is, step S16 comprises finding the maximum of a pair of values having the minimum latitude of the source point cloud and the minimum latitude of the target point cloud, the maximum of a pair of values having the minimum longitude of the source point cloud and the minimum longitude of the target point cloud, the minimum of a pair of values having the maximum latitude of the source point cloud and the maximum latitude of the target point cloud, and the minimum of a pair of values having the maximum longitude of the source point cloud and the maximum longitude of the target point cloud.

[0069] The values determined in step S16 correspond to the overlap boundary, where the overlap is an estimate of the overlap between the 2D representations. As described above, an overlap cell is determined as a cell that has at least a threshold number of points in the source point cloud and at least a threshold number of points in the target point cloud.

[0070] FIG. 10 is a flowchart illustrating an implementation of step S44 of FIG. 7 for a given overlap cell and a given Moore neighborhood cell, including steps S441 and S442. The implementation shown here is not required. Step S441 includes determining the distribution of dihedral angles of the convex hull H of points in the source point cloud whose 2D projections fall within the overlap cell and the distribution of dihedral angles of the convex hull H' of points in the target point cloud whose 2D projections fall within the given Moore neighborhood cell. Step S442 includes calculating the Kullback-Leibler divergence between these two distributions. The Kullback-Leibler divergence is then used to calculate a shape score (which may also be used as the shape score).

[0071] FIG. 11 is a flowchart illustrating an implementation of step S45 of FIG. 7 for a given overlap cell and a given Moore neighborhood cell, and includes steps S451 and S452. The implementation shown here is not required. Step S451 includes determining a density distribution as a histogram of RGB (or Cyan, Magenta, or White) colors or color intensities of points of the source point cloud within the overlap cell c, and determining the density distribution as a histogram of RGB (or Cyan, Magenta, or White) colors or color intensities of points of the target point cloud within the Moore neighborhood cell c′. That is, this step includes determining a histogram of colors of points in the source point cloud corresponding to the overlap cell c by binning the data values (color intensities) into continuous class intervals and normalizing the histogram (which provides a probability distribution of values (color intensities)). The same is done for points in the target point cloud corresponding to a given Moore neighborhood cell c′. Specifically, intensity (e.g., LiDAR intensity) may be used instead of color intensity. Color intensity comprises the intensity in a given color channel (eg, RGB or CYMK).

[0072] Step S452 involves calculating the Bhattacharyya distance between the two distributions. Then, the Bhattacharyya distance is used to calculate the color score (the Bhattacharyya distance may be used as the color score). To calculate the Bhattacharyya distance, first, the Bhattacharyya coefficient BC is calculated using the following formula:

number

[0073] Figure 12 is a flow chart illustrating the implementation of step S20' of Figure 6 for overlapping portions. The implementation shown here may be, but is not required to be, included in step S20' and / or other corresponding steps / elements.

[0074] Step S22' comprises dividing the overlap into a number of cells using a grid size (grid_size, which may be predetermined as described elsewhere) and the boundaries of the overlap in terms of x and y coordinates (e.g., latitude and longitude). The 2D overlap is divided into 2D rectangular (e.g., square) cells. Number of cells #cells = max x ind ×max y ind where max x ind is the maximum index for the x coordinate, and max y ind is the maximum value of the index for the y coordinate, and these indices are, for example, the index of cell c xy to label the cells (and their corresponding points) as

[0075] Step S24' comprises identifying the cell to which each point in the overlapping area corresponds / belongs, i.e., this step comprises identifying the cell into which each point falls in its 2D projection, and each point in the overlapping area is stored and labeled with its corresponding cell, for example in a data structure for use in a later step.

[0076] The method described below is now disclosed. It is understood that the method described below shares similarities with the above-described methods / methodologies, and features may be shared therebetween. The method described below can be considered, for example, as a specific implementation of the method of FIG. 3.

[0077] The goal of this method is to solve the problem of handling large point clouds for faster runtime in point cloud registration. An approach inspired by a divide-and-conquer strategy is proposed. The goal of the proposed approach is to align a source point cloud to a target point cloud. When multiple source point clouds are to be aligned, after alignment to the target point cloud, the source point clouds can all be stitched together to obtain, for example, a 3D global map of an area.

[0078] 1. Input. A point cloud identified as a target point cloud and another point cloud identified as a source point cloud are input.

[0079] Segmentation - This step has two steps: overlap and voxelization.

[0080] 2. In the overlap step, the boundaries of the overlap region between the source point cloud and the target point cloud in 2D are determined using the GPS data (x_min, y_min) and (x_max, y_max), where x and y can be thought of as latitude and longitude.

[0081] 3. In the voxelization step, the overlap region is divided into uniform, orthogonal 2D cells of size grid_size×grid_size. Cells are identified by their cell index, where grid_size is determined as a function of the number of points within the boundary and the boundary value itself. Due to the approximate nature of the overlap region / portion determination, each of these cells can have either zero or multiple points from either or both point clouds. The set of cells "C" (with cell c) that have overlap (i.e., have points from both point clouds) are considered overlap cells. For example, overlap cells can be determined as cells that have at least a threshold number of points from the source point cloud and at least a threshold number of points from the target point cloud, where the threshold number of points may be 1.

[0082] Similarity score determination 4. To find candidate points for correspondence between the source and target point clouds, overlapping cells c in set C are considered. For each cell c in C, the cells in the Moore neighborhood (in 2D) of cell c as the center cell are determined. These cells are denoted as cells c' in set C' for a given overlapping cell c. Set C' for an overlapping cell c also includes the center cell c (but points from the target point cloud are considered for Moore neighborhood cells c' in C', and points from the source point cloud are considered for one or more overlapping cells c in C).

[0083] 5. For a given overlap cell c, the convex hull H of the points of the source point cloud whose 2D projections lie within cell c is determined. For each cell c' corresponding to overlap cell c, the convex hull H' of the points of the target point cloud whose 2D projections lie within cell c' is determined. For convex hull determination / evaluation, a subsampling of points is selected using a farthest point sampling method such that the shape information of the point cloud is preserved. Subsampling is not required, but it helps reduce computation time. The convex hull can be determined by any suitable algorithm for determining a convex hull (e.g., Graham scan, monotone chain, incremental convex hull algorithm, etc.). For each cell c', a shape similarity score (or shape score) is determined based on comparing the convex hull H with the convex hull H'. This convex hull determination and comparison is performed for each overlap cell c in C. The result is a number of shape scores for each overlap cell c corresponding to Moore neighboring cells c' in C' to that overlap cell c.

[0084] 6. For each overlap cell c, multiple color similarity scores (or color scores) corresponding to Moore neighboring cells c' for that overlap cell c are determined. That is, the color score for a given cell c' is calculated by comparing the point of the target point cloud whose 2D projection falls within cell c' with the point of the source point cloud whose 2D projection falls within the overlap cell c corresponding to cell c'. The comparison may include comparing color intensities (e.g., RGB or CYMK) using, for example, a histogram-based measure.

[0085] governance 7. The candidate cell c' with the best "net match score" (e.g., a combination of shape and color scores, defined as a linear combination of the two similarity scores) is selected as the best match cell for the given cell c. For each overlap cell c, a best match cell c' is selected. The sets of points from the source and target point clouds, respectively, for each match pair of cells c and c' are appended to sets M and M', respectively. That is, set M has points from the source point cloud whose 2D projection falls within the overlap cell c of set C, and set M' has points from the target point cloud whose 2D projection falls within the best match cell c' corresponding to the overlap cell c.

[0086] A threshold is applied to the net match score so that not all paired cells c, c' are selected for inclusion in sets M and M', i.e., only "good" matches (and corresponding points) are considered for registration. A transformation matrix is learned between sets M and M', e.g., using a suitable registration algorithm. Note that instead of loading the entire point cloud (e.g., into device memory) for the calculation of the transformation matrix, only the points in sets M and M' need to be loaded for the calculation.

[0087] 8. Once the transformation matrix is obtained (point 7 above), it is applied to the entire source point cloud. The resulting point cloud points P new is P new= P1(not elsewhere) + P1(matching) + P2(not elsewhere), where P1 and P2 are points in the target and source point clouds, respectively. More specifically, P1(matching) refers to points in the target point cloud that match points in the source point cloud (i.e., these points exist in both point clouds and represent the same physical location in both point clouds), and P1(not elsewhere) and P2(not elsewhere) are the remaining points in the target and source point clouds, respectively, that do not have corresponding matching points. Thus, redundant points are not included. If any occluded or missing points are in the target point cloud but not in the source point cloud (or vice versa), these points may be included in the resulting point cloud, for example, by point P2(not elsewhere).

[0088] The shape score and / or color score may be determined as described with reference to other methods and figures disclosed herein.

[0089] The above-described implementations and examples can be considered implementations of the first group. Implementations of the second group are also disclosed herein. The implementations of the second group are the same as the implementations of the first group, except that instead of dividing the 2D overlapping portion into 2D cells, a 3D overlap volume is determined and divided into 3D cells. Furthermore, in the implementations of the second group, the source set and target set corresponding to the (3D) overlapping cell are determined to be different sets of points corresponding to the (2D) overlapping cell in the implementations of the first group (however, the comparison between the sets is the same). That is, in the comparison process in the implementations of the second group, the points of the source set and target set are determined as sets of points of the relevant point cloud that are within the relevant 3D cell, rather than points that fall within the 2D cell in the 2D projection. In other respects, the implementations of the second group are the same as the implementations of the first group, and a redundant description will be omitted.

[0090] For example, Figure 13 is a flowchart showing a method according to one implementation of the second group, which includes steps S101-S601. The similarity with the method of Figure 3, which includes steps S10-S60, will be understood.

[0091] Step S101 includes estimating an overlap volume, i.e., estimating a 3D overlap volume between a source 3D point cloud and a target 3D point cloud based on an overlap region between the source geographical area and the target geographical area, where the source 3D point cloud and the target 3D point cloud represent the source geographical area and the target geographical area.

[0092] Step S201 includes dividing the 3D overlap volume into a plurality of 3D cells.

[0093] Step S301 includes determining at least one overlap cell that includes at least a threshold number of points from the source 3D point cloud and at least a threshold number of points from the target 3D point cloud. It is understood that this overlap cell is a 3D cell. The threshold is the same as the threshold in step S30, and can be, for example, 1 or 2 or 3 (or a different number).

[0094] Step S401 comprises performing a comparison process on the overlap cell to determine a set of points of the target point cloud that best match a set of points of the source point cloud in the overlap cell, i.e., this step comprises performing a comparison process on the overlap cell, the comparison process comprising: determining a plurality of target sets of points, each of the target 3D point cloud having a first target set of points that are within an overlapping cell, and a second target set of points and a further target set of points that are within cells that are adjacent to / surrounding the overlapping cell; determining points of the source set that have points of the source 3D point cloud that are within overlap cells; comparing the source set to each of a plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; Selecting the best matching target set based on similarity scores; It has the following.

[0095] Step S501 comprises determining a transformation between points in the source set and points in the best match target set (or vice versa).

[0096] Step S601 comprises applying a transformation to register a source 3D point cloud to a target 3D point cloud (or vice versa).

[0097] Steps S501 and S601 are the same as steps S50 and S60. The comparison between the source set and the target set in step S401 is the same as the described comparison between sets in the first group of implementations (e.g., step S40), but the sets themselves are determined differently. In the first group of implementations, the cells are 2D (because the cells are divided from the 2D overlapping portions), while in the second group of implementations, the cells are 3D (because the cells are divided from the 3D overlapping volumes), so in the first group of implementations, the sets are determined based on which points fall within the 2D cells in the 2D projection, while in the second group of implementations, the sets are determined based on which points are within the 3D cells.

[0098] Estimating the 3D overlap volume in step S101 may include determining a 3D overlap volume defined in first and second dimensions by a 2D overlap between a source 2D projection of the source 3D point cloud and a target 2D projection of the target 3D point cloud, and in a third dimension by upper and lower bounds of the source and target 3D point clouds in the third dimension, where the 2D overlap corresponds to an overlap region between the source geographic area and the target geographic area. The first and second dimensions are, for example, latitude and longitude (as in the first group of implementations).

[0099] Estimating the 3D overlap volume in step S101 may include determining a 2D overlap between a source 2D projection of the source 3D point cloud and a target 2D projection of the target 3D point cloud (in the same way as the 2D overlap is determined in the first group of implementations), the 2D overlap corresponding to the overlap region between the source geographical area and the target geographical area, and determining a 3D overlap volume defined by the 2D overlap in the first and second dimensions and defined in the third dimension by upper and lower boundaries of the source 3D point cloud and the target 3D point cloud in the third dimension.

[0100] The overlapping cell and its surrounding cells may form a (3D) Moore neighborhood with the overlapping cell as the center cell.

[0101] A target set (for an overlapping cell) may be determined as the (target) set of points of the target 3D point cloud that are within each cell in the (3D) Moore neighborhood, with the overlapping cell as the center cell (of the (3D) Moore neighborhood).

[0102] The cells surrounding the overlapping cell may have orthogonal and diagonal adjacent cells to the overlapping cell.

[0103] As already indicated, the description of the first group of implementations applies to the second group of implementations, except for the parts related to splitting the overlapping portion and determining which points are used for the 2D cell-based comparison.

[0104] The division of the 3D overlap volume (sometimes referred to as an overlap volume or overlap volume) may be based on a grid size. In some implementations, the grid size is set so that the volume of the resulting cell is greater than the volume of the overlap volume divided by the total number of points in the overlap volume. The grid size may be selected taking into account the average number of points per cell. For example, the desired average number of points per cell may be used to determine the total number of cells, and the grid size may be determined based on that number. Alternatively, the desired average number of points per cell may be used to determine the volume of each cell, and the grid size may be determined based on that volume. The overlap volume is, in some implementations, divided into uniform orthogonal 3D cells (e.g., cuboids / cubes).

[0105] Considering the implementation of the first and second groups, the following points are noted:

[0106] The method disclosed herein involves dividing the point clouds to be registered into multiple cells (specifically, overlaps / volumes) and finding the convex hull of these cells, which allows for parallelization and thereby reduces time. Furthermore, separate threads of processing (e.g., for color scores and / or shape scores, as in FIG. 7) can be performed on different processors of a computing system / device. The best match of the candidate points in the overlap / volume is determined by a matching algorithm (e.g., as described above in the comparison process and in FIG. 7, etc.) that can ensure that the source and target point clouds are matched by shape and color. Full point cloud alignment can be performed using only a portion of the points in the match sets of the source and target point clouds found in the matching algorithm stage (the best match set, e.g., highest similarity score).

[0107] One or more methods / methodologies disclosed herein can be considered a framework for improving the efficiency of lidar point cloud registration in terms of time and memory utilization. The methods / methodologies disclosed herein utilize a divide-and-conquer strategy by partitioning the overlap / volume between two point clouds into a grid of cells (2D or 3D) and determining the best set of points for registration from those cells. Point clouds can be stitched together to generate a more complete 3D map. To handle large point cloud sizes, point cloud data is stored in one or more structured databases, and only relevant points can be loaded for calculation rather than loading the entire large point cloud. The divide-and-conquer strategy is an algorithmic technique used to solve complex problems by decomposing them into smaller, more manageable sub-problems, where each sub-problem can be solved independently and the solutions can be combined to obtain a solution to the original problem. Divide-and-conquer strategies are used in fast Fourier transforms, Strassen matrix multiplication, convex hull determination, the Karatsuba algorithm, etc.

[0108] The methods / methodologies disclosed herein can be employed / used in a number of applications, including but not limited to: Lidar mapping. Autonomous Driving: Lidar registration is crucial in the field of autonomous vehicles and other independently mobile units. Through the fusion of multiple lidar scans, such vehicles can build a comprehensive, up-to-date 3D terrain representation / map, enabling safe navigation and on-the-fly decision-making. 3D Mapping and Modeling: Lidar registration is a key process used to generate detailed 3D maps and models of different environments such as urban areas, forests, elevated terrain, and enclosed indoor areas. These 3D representations play a key role in tasks such as urban planning, ecological surveys, and the development of virtual reality experiences. Robotics and UAVs: Lidar registration finds applications in the fields of robotics / UGVs and unmanned aerial vehicles (UAVs) for tasks such as precise positioning, obstacle identification, and comprehensive environment understanding. This technology gives robots and drones the ability to achieve accurate and streamlined awareness of their surroundings.

[0109] The methods / methodologies disclosed herein can be used for registration of any point cloud (not necessarily lidar data).

[0110] One objective of the method / methodology disclosed herein can be considered as how to improve the time and memory utilization of point cloud registration while maintaining alignment accuracy.

[0111] The methods / methodologies disclosed herein can achieve several advantages, including but not limited to: A divide-and-conquer strategy is adopted to solve the problem of handling large point clouds and for faster runtime. For each overlapping cell c (2D or 3D), the respective comparison processes can be performed in parallel, reducing processing time. Furthermore, separate threads of processing (e.g., for color scores and / or shape scores, as in Figure 7) can be performed on different processors of a computing system / device. · The convex hull determination in the comparison process preserves the geometry of the point cloud. The comparison process and optional threshold-based selection of matched pairs allows only a subset of points to be used for registration, rather than all points in the point cloud or all points within the overlap / volume, thereby reducing the processing load and time. This is particularly useful for point clouds of large size, as far fewer points need to be considered when determining the transformation matrix. To handle large point clouds, the point cloud data is stored in one or more structured databases, and only the relevant points can be loaded for calculations, rather than loading the entire large point cloud. This is particularly useful for large point clouds, as far fewer points need to be considered when determining the transformation matrix. The number of points within the overlap / volume is often much smaller than the total number of points in the point cloud. Furthermore, as mentioned above, it is not necessary to consider all points within the overlap / volume when determining the transformation matrix. Previous techniques involve extracting features by applying DNNs to the complete point cloud and / or by using local descriptors across the point cloud, which is memory intensive, especially for large point clouds. As mentioned above, these problems are overcome by the method / methodology disclosed herein.

[0112] Comparative Methods 1-4 have the following drawbacks which are overcome by the method / methodology disclosed herein:

[0113] Comparison Method 1: This method involves a deep learning-based approach and faces challenges when dealing with large point clouds due to limitations in creating large neural networks, especially memory-intensive CNNs, on standard GPUs. Furthermore, dealing with millions of points in a point cloud becomes infeasible due to the large number of trainable parameters, making the optimization and training process extremely difficult. Additionally, obtaining such large labeled data for a supervised approach is challenging for large point clouds. These challenges are addressed in the disclosed method / methodology, among other aspects, by considering overlaps / volumes rather than the entire point cloud, the comparison process, and optional threshold-based selection of matching points for use in determining the transformation matrix.

[0114] Comparison Method 2: This method involves computing a descriptor for the entire point cloud and then dividing it into a grid. This approach poses challenges when dealing with point clouds containing millions of points, as loading and processing the entire cloud becomes a challenging task. Furthermore, the use of descriptors shows limited performance when applied to point clouds with repetitive structures, such as buildings and roads, making descriptor-based approaches susceptible to noise. Additionally, in the grid-matching process after creating the grid, this method relies on a voting algorithm and does not utilize GPS information for effective grid pairing. Furthermore, the presence of multiple sequential steps in the method, such as descriptor discovery, voting module, graph matching, and outlier filtering, prevents this method from being efficiently parallelized. These issues are addressed in the disclosed method / methodology by, among other aspects, considering overlaps / volumes rather than the entire point cloud, the comparison process, and optional threshold-based selection of matching points for use in determining the transformation matrix.

[0115] Comparison Method 3: This method involves an unsupervised, attention-based encoder-decoder approach that involves processing the entire point cloud to estimate the transformation matrix. Therefore, this approach suffers from computational inefficiencies when dealing with large point clouds, leading to unrealistic memory demands on standard GPUs. Furthermore, incorporating a Gaussian mixture model adds to the overall overhead and exacerbates the computational challenges. These issues are addressed in the disclosed method / methodology by, among other aspects, considering the overlap / volume rather than the entire point cloud, the comparison process, and the optional threshold-based selection of matching points to use in determining the transformation matrix.

[0116] Comparison Method 4: The accuracy of this method is highly dependent on the canopy shape, making it a specialized technique tailored to aligning forest point clouds. This approach becomes less effective in scenarios with dense point clouds. This limitation stems from the method's transformation of 3D points into a 2D image, which causes multiple 3D points to map to the same 2D point. As a result, identifying corresponding points becomes difficult, introducing inaccuracies in such situations. These issues are addressed in the disclosed method / methodology by, among other aspects, considering overlap / volume and the comparison process (and optional threshold-based selection of matching points to use in determining the transformation matrix may improve efficiency and throughput).

[0117] One objective of the method / methodology disclosed herein can be considered to be how to improve the memory and time requirements of LIDAR registration to create a 3D map of an entire area. Another objective can be considered to be how to achieve good accuracy for 3D map generation while using less time and memory than other methods.

[0118] 14 is a block diagram of an information processing apparatus 900 or computing device 900, such as a data storage server, embodying the present invention and which may be used to implement some or all of the operations of a method embodying the present invention and to perform some or all of the tasks of an apparatus of an embodiment. The computing device 900 may be used to perform any of the method steps described above, such as any of steps S10-S60 and / or any of steps S101-S601 and / or any of the steps of FIGS. 4-13.

[0119] Computing device 900 has a processor 993 and memory 994. Optionally, the computing device also includes a network interface 997 for communicating with other such computing devices, such as other computing devices of invention embodiments. Optionally, the computing device also includes one or more input mechanisms 996, such as a keyboard and mouse, and a display unit 995, such as one or more monitors. These elements may facilitate user interaction. These components may be connected to one another via a bus 992.

[0120] The memory 994 may include a computer-readable medium, which term may refer to a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) configured to carry computer-executable instructions. Computer-executable instructions may include, for example, instructions and data that are accessible by a computer (e.g., one or more processors) and cause the computer to perform one or more functions or operations. For example, computer-executable instructions may include instructions for implementing a method disclosed herein or any method step disclosed herein, e.g., any of steps S10-S60 and / or any of steps S101-S601 and / or any of the steps of Figures 4-13. Accordingly, the term "computer-readable storage medium" may also include any medium capable of storing, encoding, or carrying a set of instructions for execution by a machine, causing the machine to perform any one or more of the method steps of the present disclosure. Accordingly, the term "computer-readable storage medium" may be interpreted to include, but is not limited to, solid-state memory, optical media, and magnetic media. By way of example, and not limitation, such computer-readable media may include non-transitory computer-readable storage media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid-state memory devices).

[0121] The processor 993 is configured to control the computing device and perform processing operations, for example, executing computer program code stored in the memory 994 to perform any of the method steps described herein. The memory 994 stores data to be read and written by the processor 993, and may store information / data of points of the point cloud, and / or the point cloud, and / or sets of points, and / or threshold information, and / or cell data, and / or grid data, and / or color information, and / or convex hull information, and / or formulas for calculating similarity scores, and / or similarity scores, and / or transformation matrices, and / or the resulting point cloud, and / or a 2D projection of the point cloud, and / or data regarding geographical areas, and / or latitude and longitude information, and / or GPS data, and / or distributions used to determine the similarity scores, and / or input data, and / or other data as described above, and / or programs for performing any of the method steps as described above.

[0122] As referred to herein, a processor may include one or more general-purpose processing devices, such as, for example, a microprocessor, a central processing unit, or the like. A processor may include a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or a combination of instruction sets. A processor may also include one or more special-purpose processing devices, such as, for example, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. In one or more embodiments, a processor is configured to execute instructions to perform the processes and operations described herein. The processor 993 may be considered to include any of the modules described above. Any process described as being implemented by a module may also be implemented as a computational method, such as by the processor 993. The parallel processes referred to above may be performed by separate processors operating in parallel, whether these separate processors are physically separate processors or logical divisions. That is, device 10 may have multiple processors, each capable of implementing one or more threads of the processing described above.

[0123] The display unit 995 may display representations of data stored by the computing device, such as, for example, representations of information / data of points in a point cloud, and / or a point cloud, and / or a set of points, and / or threshold information, and / or cell data, and / or grid data, and / or color information, and / or convex hull information, and / or a formula for calculating a similarity score, and / or a similarity score, and / or a transformation matrix, and / or a resulting point cloud, and / or a 2D projection of the point cloud, and / or data regarding a geographical area, and / or latitude and longitude information, and / or GPS data, and / or a distribution used to determine a similarity score, and / or input data, and / or other data, and / or a GUI window, and / or an interactive representation that allows a user to interact with the apparatus 900, for example by drag-and-drop or selection interaction, and / or any other output described above, as well as cursors and dialog boxes and screens that allow interaction between a user and programs and data stored on the computing device. The input mechanism 996 may allow a user to input data and instructions into the computing device, such as by allowing a user to input any of the user inputs described above.

[0124] The network interface (network I / F) 997 can be connected to a network, such as the Internet, and can be connected to other such computing devices via the network. The network I / F 997 can control data input / output from / to other devices via the network. Other peripheral devices, such as a microphone, speaker, printer, power supply unit, fan, case, scanner, tracker ball, etc., may also be included in the computing device.

[0125] A method embodying the present invention may be performed on a computing device / apparatus 900, such as that shown in FIG. 14. Such a computing device need not have all of the components shown in FIG. 14, but may be configured with a subset of these components. For example, apparatus 900 may have a processor 993 and a memory 994 connected to the processor 993. Alternatively, apparatus 900 may have a processor 993, a memory 994 connected to the processor 993, and a display 995. A method embodying the present invention may be performed by a single computing device that communicates with one or more data storage servers over a network. The computing device may also be the data storage itself, storing at least a portion of the data.

[0126] Methods embodying the present invention may be performed by a plurality of computing devices operating in cooperation with one another, one or more of which may be data storage servers that store at least a portion of the data.

[0127] The invention may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The invention may also be implemented as a computer program or computer program product, that is, a computer program tangibly embodied in a non-transitory information carrier, such as a machine-readable storage device, or in a propagated signal, for execution by, or to control the operation of, one or more hardware modules.

[0128] A computer program may be in the form of a stand-alone program, a computer program portion, or two or more computer programs, may be written in any type of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a data processing environment. A computer program may be deployed to be executed on one module or on multiple modules at one location or distributed across multiple locations and interconnected by a communication network.

[0129] The method steps of the present invention can be performed by one or more programmable processors that execute computer programs to perform the functions of the present invention by processing input data and generating output. Apparatus of the present invention can be implemented as programmed hardware or as special purpose logic circuitry including, for example, an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0130] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from a read-only memory or a random-access memory or both. The essential elements of a computer are a processor for executing instructions coupled to one or more memory devices for storing instructions and data.

[0131] The above-described embodiments of the invention may be used to advantage independently of any other of these embodiments or in any feasible combination with one or more other of these embodiments.

[0132] This disclosure extends to the following appendices:

[0133] APPENDIX X1. A computer-implemented method comprising: determining a 2D overlap between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target set points, each of which includes a first target set point of the target 3D point cloud that falls within the overlap cell, and a second target set point and a further target set point of the target 3D point cloud that fall within cells adjacent to the overlap cell; determining a source set of points having points of the source 3D point cloud that fall within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer-implemented method comprising:

[0134] Appendix X2. The computer-implemented method of Appendix X1, further comprising determining the overlap region between the source geographic area and the target geographic area based on GPS data.

[0135] Annex X3. The computer-implemented method of Annex X1 or X2, wherein the target set for the overlapping cell is determined as points of the target set of the target 3D point cloud that fall within each cell in the Moore neighborhood, with the overlapping cell as the center cell of the Moore neighborhood.

[0136] Appendix X4. The computer-implemented method of any of Appendixes X1 to X3, wherein comparing the source set with each of the plurality of target sets comprises determining a source convex hull of points in the source set, determining a plurality of target convex hulls of points in the plurality of target sets, respectively, and comparing the source convex hull to each target convex hull.

[0137] Appendix X5. Comparing the source set with each of the plurality of target sets comprises, for each target set: performing a first comparison between the shapes of the points of the source set and the shapes of the points of the target set to determine a first contribution to the similarity score for the target set; and / or performing a second comparison between the intensities of the points of the source set and the intensities of the points of the target set to determine a second contribution to the similarity score for the target set; 10. The computer-implemented method of claim 1, further comprising:

[0138] Appendix X6. Comparing the source set to each of the plurality of target sets comprises performing a first and / or second comparison process; The first comparison process determines a convex hull of the points of the source set, determines a first distribution of dihedral angles of the convex hull of the points of the source set, and for each target set: determining a convex hull of the points in the target set; determining a second distribution of dihedral angles of the convex hull of the target set of points; calculating the difference between the first distribution and the second distribution of dihedral angles; adding a contribution to the similarity score for the target set based on the calculated difference; The second comparison process determines a first distribution of intensities of points in the source set, and for each target set: determining a second distribution of intensities of points in the target set; calculating a distance measure between the first and second distributions of intensities; adding a contribution to the similarity score for the target set based on the calculated distance measure. 10. A computer-implemented method according to any one of claims 1 to 3.

[0139] Note X7. When multiple overlapping cells are determined, performing said comparison process for each overlapping cell; determining a final target set point having at least one best match target set point of the best match target sets, and determining a final source set point having at least one corresponding source set point; determining the transformation between the final source set of points and the final target set of points; 10. The computer-implemented method of claim 1, further comprising:

[0140] Appendix X8. The computer-implemented method of Appendix X7, wherein determining the points of the final target set includes selecting at least one of the best match target sets having a similarity score above a threshold score or having the highest similarity score, and determining the points of the final target set as including points of the selected at least one best match target set.

[0141] Appendix X9. A computer program product which, when executed on a computer, causes the computer to perform a method, the method comprising: determining a 2D overlap between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target set points, each of which includes a first target set point of the target 3D point cloud that falls within the overlap cell, and a second target set point and a further target set point of the target 3D point cloud that fall within cells adjacent to the overlap cell; determining a source set of points having points of the source 3D point cloud that fall within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer program comprising:

[0142] Appendix X10. An information processing device having a memory and a processor connected to the memory, wherein the processor: determining a 2D overlap between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target set points, each of which includes a first target set point of the target 3D point cloud that falls within the overlap cell, and a second target set point and a further target set point of the target 3D point cloud that fall within cells adjacent to the overlap cell; determining a source set of points having points of the source 3D point cloud that fall within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; An information processing device configured as follows.

[0143] A computer-implemented method comprising: estimating a 3D overlap volume between a source 3D point cloud and a target 3D point cloud based on an overlap region between the source geographic area and the target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; Dividing the 3D overlap volume into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of the first target set of points of the target 3D point cloud being within the overlapping cell, and a second target set of points and a further target set of points of the target 3D point cloud being within cells adjacent to the overlapping cell; determining a source set of points that have points of the source 3D point cloud that are within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer-implemented method comprising:

[0144] Appendix Y2. The computer-implemented method of Appendix Y1, wherein estimating the 3D overlap volume comprises determining the 3D overlap volume defined in a first dimension and a second dimension by a 2D overlap between a source 2D projection of the source 3D point cloud and a target 2D projection of the target 3D point cloud, and in a third dimension by upper and lower boundaries of the source 3D point cloud and the target 3D point cloud in the third dimension, wherein the 2D overlap corresponds to the overlap region between the source geographic area and the target geographic area.

[0145] Appendix Y3. The computer-implemented method of any one of Appendixes Y1 and Y2, wherein the target set for the overlapping cell is determined as points of the target set of the target 3D point cloud that are within each cell in the Moore neighborhood, with the overlapping cell as the center cell of the Moore neighborhood.

[0146] Addendum Y4. The computer-implemented method of any of Addendums Y1 to Y3, wherein comparing the source set with each of the plurality of target sets comprises determining a source convex hull of points in the source set, determining a plurality of target convex hulls of points in the plurality of target sets, respectively, and comparing the source convex hull with each target convex hull.

[0147] Appendix Y5. Comparing the source set with each of the plurality of target sets comprises, for each target set: performing a first comparison between the shapes of the points of the source set and the shapes of the points of the target set to determine a first contribution to the similarity score for the target set; and / or performing a second comparison between the intensities of the points of the source set and the intensities of the points of the target set to determine a second contribution to the similarity score for the target set; 10. The computer-implemented method of any one of claims Y1 to Y3, comprising:

[0148] Addendum Y6. Comparing the source set to each of the plurality of target sets comprises performing a first and / or second comparison process; The first comparison process determines a convex hull of the points of the source set, determines a first distribution of dihedral angles of the convex hull of the points of the source set, and for each target set: determining a convex hull of the points in the target set; determining a second distribution of dihedral angles of the convex hull of the target set of points; calculating the difference between the first distribution and the second distribution of dihedral angles; adding a contribution to the similarity score for the target set based on the calculated difference; The second comparison process determines a first distribution of intensities of points in the source set, and for each target set: determining a second distribution of intensities of points in the target set; calculating a distance measure between the first and second distributions of intensities; adding a contribution to the similarity score for the target set based on the calculated distance measure. A computer-implemented method according to any one of appendices Y1 to Y3.

[0149] Appendix Y7. When multiple overlapping cells are determined, performing said comparison process for each overlapping cell; determining a final target set point having at least one best match target set point of the best match target sets, and determining a final source set point having at least one corresponding source set point; determining the transformation between the final source set of points and the final target set of points; 10. The computer-implemented method of any one of claims Y1 to Y6, comprising:

[0150] Appendix Y8. The computer-implemented method of Appendix Y7, wherein determining the points of the final target set includes selecting at least one of the best match target sets having a similarity score above a threshold score or having the highest similarity score, and determining the points of the final target set as including points of the selected at least one best match target set.

[0151] A computer program product that, when executed on a computer, causes the computer to perform a method, the method comprising: estimating a 3D overlap volume between a source 3D point cloud and a target 3D point cloud based on an overlap region between the source geographic area and the target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; Dividing the 3D overlap volume into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of the first target set of points of the target 3D point cloud being within the overlapping cell, and a second target set of points and a further target set of points of the target 3D point cloud being within cells adjacent to the overlapping cell; determining a source set of points that have points of the source 3D point cloud that are within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer program comprising:

[0152] Appendix Y10. An information processing device having a memory and a processor connected to the memory, wherein the processor: estimating a 3D overlap volume between a source 3D point cloud and a target 3D point cloud based on an overlap region between the source geographic area and the target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; Dividing the 3D overlap volume into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of the first target set of points of the target 3D point cloud being within the overlapping cell, and a second target set of points and a further target set of points of the target 3D point cloud being within cells adjacent to the overlapping cell; determining a source set of points that have points of the source 3D point cloud that are within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; An information processing device configured as follows.

[0153] This disclosure extends to the following appendix (S):

[0154] S1. A computer-implemented method comprising: determining a 2D overlap between a source 2D (two-dimensional) projection of a source 3D (three-dimensional) point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; Dividing the 2D overlap portion into a plurality of cells, and determining at least one overlap cell including at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection (or determining at least one cell including at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection as at least one overlap cell); performing a comparison process for the (or the) overlapping cells, said comparison process (for the overlapping cells) comprising: determining a plurality of target sets of points, each of which comprises a first (target) set of points of the target 3D point cloud that fall within the overlapping cell (in the target 2D projection), and a second target set of points and a further (target) set of points of the target 3D point cloud that fall within cells adjacent / surrounding the overlapping cell (in the target 2D projection); determining points of a source set that have points of the source 3D point cloud that fall within the overlap cell (in the source 2D projection); comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; selecting a best matching target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set; applying the transformation (matrix) (to the source 3D point cloud) to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer-implemented method comprising:

[0155] S2. The computer-implemented method of S1, comprising determining the source 2D projection of the source 3D point cloud and the target 2D projection of the target 3D point cloud.

[0156] S3. The computer-implemented method of S1 or S2, wherein the first number and / or threshold number is 1, 2, or 3.

[0157] Appendix S4. The computer-implemented method of any of Appendixes S1 to S3, wherein the first and second threshold numbers are the same.

[0158] Clause S5. The computer-implemented method of any of clauses S1-S4, further comprising determining the overlap region between the source geographic area and the target geographic area based on GPS data.

[0159] Appendix S6. The computer-implemented method of any of Appendixes S1 to S5, wherein the overlapping cell and its surrounding cells form a (2D) Moore neighborhood with the overlapping cell as a center cell.

[0160] Appendix S7. A computer-implemented method according to any of Appendixes S1 to S6, wherein the target set (for the overlapping cell) is determined as the points of the (target) set of the target 3D point cloud that fall within each cell in the (2D) Moore neighborhood, with the overlapping cell as the center cell (of the (2D) Moore neighborhood).

[0161] Appendix S8. The computer-implemented method of any of Appendixes S1 to S7, wherein comparing the source set with each of the plurality of target sets includes determining a source convex hull of points in the source set, determining a plurality of target convex hulls of points in the plurality of target sets, respectively, and comparing the source convex hull to each target convex hull.

[0162] Appendix S9. The computer-implemented method of any of Appendixes S1 to S8, comprising determining a contribution to a similarity score for each target set based on a comparison of the source convex hull with the associated target convex hull.

[0163] Appendix S10. A computer-implemented method according to any of Appendixes S1 to S9, wherein comparing the source set with each of the plurality of target sets comprises, for each target set, performing a first comparison between the shapes of the points of the source set and the shapes of the points of the target set to determine a first contribution to the similarity score for the target set, and / or performing a second comparison between the (color) intensities of the points of the source set and the (color) intensities of the points of the target set to determine a second contribution to the similarity score for the target set.

[0164] Appendix S11. The computer-implemented method of any of Appendixes S1 to S10, wherein comparing the source set to each of the plurality of target sets includes performing a first and a second comparison for each target set and combining the first and second contributions to determine a similarity score for the target set.

[0165] Appendix S12. The computer-implemented method of any of Appendixes S1 to S11, wherein comparing the source set to each of the plurality of target sets includes, for each target set, performing a first comparison between the shape of the points of the source set and the shape of the points of the target set to determine a first contribution to the similarity score of the target set, performing a second comparison between the (color) intensities of the points of the source set and the (color) intensities of the points of the target set to determine a second contribution to the similarity score of the target set, and combining the first and second contributions to determine the similarity score of the target set.

[0166] S13. The computer-implemented method of any of S1 to S12, wherein comparing the source set with each of the plurality of target sets comprises performing a first and / or second comparison process, wherein the first comparison process comprises determining a convex hull of points of the source set, determining a first distribution of dihedral angles of the convex hull of points of the source set, and for each target set, determining a convex hull of points of the target set, determining a second distribution of dihedral angles of the convex hull of points of the target set, calculating a difference between the first and second distributions of dihedral angles, and adding a contribution to the similarity score for the target set based on the calculated difference; and the second comparison process comprises determining a first distribution of (color) intensities of points in the source set, and for each target set, determining a second distribution of (color) intensities of points in the target set, calculating a distance / similarity measure between the first and second distributions of intensities, and adding a contribution to the similarity score for the target set based on the calculated difference.

[0167] Appendix S14. The computer-implemented method of any of Appendixes S1 to S13, wherein comparing the source set with each of the plurality of target sets comprises determining a convex hull of points of the source set, determining a first distribution of dihedral angles of the convex hull of points of the source set, for each target set, determining a convex hull of points of the target set, determining a second distribution of dihedral angles of the convex hull of points of the target set, calculating a difference between the first and second distributions of dihedral angles, and adding a contribution to the similarity score for the target set based on the calculated difference.

[0168] Appendix S15. The computer-implemented method of any of Appendixes S13-S14, wherein the variance is the Kullback-Leibler divergence.

[0169] Clause S16. The computer-implemented method of any of clauses S8-S15, wherein determining the convex hull comprises using farthest point sampling for a set of related points.

[0170] Appendix S17. A computer-implemented method according to any of Appendixes S1 to S16, wherein comparing the source set with each of the plurality of target sets comprises, for each target set, comparing the (color) intensity of points in the source set with the (color) intensity of points in the target set.

[0171] Appendix S18. The computer-implemented method of Appendix S17, wherein comparing the (color) intensities of the points of the source set with the (color) intensities of the points of the target set comprises using a histogram-based measure.

[0172] Appendix S19. The computer-implemented method of Appendix S17 or S18, wherein comparing the colors of the points of the source set with the colors of the points of the target set comprises determining a first distribution of (color) intensities of the points of the source set and a second distribution of (color) intensities of the points of the target set, and comparing the first and second distributions of intensities.

[0173] Appendix S20. The computer-implemented method of Appendix S19, comprising determining a contribution to the similarity score for each target set based on a comparison of the first distribution of intensities with the second distribution of related intensities.

[0174] Appendix S21. The computer-implemented method of Appendix S19 or S20, wherein comparing the first and second distributions of intensities comprises determining a distance / similarity measure between the distributions.

[0175] Appendix S22. The computer-implemented method of any of Appendixes S13 to S21, comprising determining a contribution to the similarity score for each target set based on the distance / similarity measure corresponding to that target set.

[0176] Clause S23. The computer-implemented method of any of clauses S19 to S22, wherein the first and second distributions of intensities are determined as histograms of intensities of points in the relevant sets.

[0177] Appendix S24. The computer-implemented method of any of Appendixes S19 to S23, wherein the first and second distributions of intensity are distributions of RGB or CYMK colors of points in the related set, or distributions of LiDAR intensity of points in the related set.

[0178] Appendix S25. The computer-implemented method of any of Appendixes S19 to S24, wherein the first and second distributions of intensities are determined as histograms of RGB or CYMK colors or LiDAR intensities of points in the relevant sets.

[0179] Appendix S26. The computer-implemented method of any of Appendixes S13 to S25, wherein the distance / similarity measure comprises a Bhattacharyya distance.

[0180] Appendix S27. A computer-implemented method according to any of Appendixes S1 to S26, wherein comparing the source set with each of the plurality of target sets comprises determining a first distribution of (color) intensities of points in the source set, determining for each target set a second distribution of (color) intensities of points in the target set, calculating a distance / similarity measure between the first and second distributions of intensities, and adding a contribution to a similarity score for the target set based on the calculated difference.

[0181] Clause S28. The computer-implemented method of clause S27, wherein the distance / similarity measure comprises a Bhattacharya distance.

[0182] Appendix S29. The computer-implemented method of Appendix S27 or S28, wherein the first and second distributions of intensities are determined as histograms of RGB or CYMK colors or LiDAR intensities of points in the relevant sets.

[0183] Appendix S30. A computer-implemented method described in any of Appendixes S9 to S29, wherein comparing the source set to each of the plurality of target sets includes, for each target set, combining the first and second contributions to determine the similarity score for the target set.

[0184] Clause S31. The computer-implemented method of any of clauses S1-S30, wherein determining the transformation matrix comprises determining the transformation matrix using an iterative closest point (ICP) algorithm.

[0185] Appendix S32. A computer-implemented method described in any of Appendixes S1 to S31, comprising, when multiple overlapping cells are determined, performing the comparison process for each overlapping cell (to determine multiple best match target sets corresponding to multiple source sets, each corresponding to an overlapping cell), determining a final target set point having a best match target set point of at least one of the best match target sets, determining a final source set point having a corresponding at least one source set point (corresponding to the at least one best match target set), and determining the transformation matrix between the final source set point and the final target set point.

[0186] Appendix S33. The computer-implemented method of Appendix S32, wherein determining the points of the final target set includes selecting at least one of the best match target sets having a similarity score above a threshold score or having the highest similarity score, and determining the points of the final target set as including the points of the selected at least one best match target set.

[0187] Appendix S34. The computer-implemented method of any of Appendixes S1 to S33, wherein the source 2D projection has points that represent points of the source 3D point cloud and the target 2D projection has points that represent points of the target 3D point cloud.

[0188] Appendix S35. The computer-implemented method of any of Appendixes S1 to S34, wherein points in the source 3D point cloud that fall within the overlap cell are points whose representing points in the source 2D projection are within the overlap cell, and points in the target 3D point cloud that fall within the overlap cell are points whose representing points in the target 2D projection are within the overlap cell.

[0189] Appendix S36. The computer-implemented method of any of Appendixes S1 to S35, wherein the source 3D point cloud and the target 3D point cloud comprise LiDAR scan data.

[0190] Appendix S37. A computer-implemented method, comprising: determining a source 2D projection of a source 3D point cloud representing a source geographic area; determining a target 2D projection of a target 3D point cloud representing a target geographic area; determining a 2D overlap portion between the source 2D projection and the target 2D projection corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells; determining at least one overlap cell comprising at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection (or determining at least one cell comprising at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection as at least one overlap cell); and performing a comparison process for the (or) overlap cell, wherein the comparison process compares a first (or a second) of the target 3D point cloud that falls within the overlap cell (in the target 2D projection). determining a plurality of target sets of points, each having points of a first (target) set of points of the target 3D point cloud and a second (target) set of points of the target 3D point cloud that fall within cells adjacent / surrounding (in the target 2D projection) to the overlap cell; determining a source set of points having points of the source 3D point cloud that fall within the overlap cell (in the source 2D projection); comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; selecting a best match target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set; and applying the transformation (matrix) (to the source 3D point cloud),and registering the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud.

[0191] Appendix S38. A computer-implemented method, comprising: determining a source 2D projection of a source 3D point cloud representing a source geographic area; determining a target 2D projection of a target 3D point cloud representing a target geographic area; determining a 2D overlap portion between the source 2D projection and the target 2D projection corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells; determining at least one overlap cell comprising at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection (or determining at least one cell comprising at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection as at least one overlap cell); and for each (or the) overlap cell, determining a first (target) set of points of the target 3D point cloud that fall within the overlap cell and a second (target) set of points of the target 3D point cloud that fall within cells adjacent / surrounding the overlap cell. determining a plurality of target set points, each having a target set point and a further (target) set point; determining a source set point having points of the source 3D point cloud that fall within the overlap cell; comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; and selecting a best match target set based on the similarity scores; determining a final target set point having at least one best match target set point among the best match target sets; determining a final source set point having at least one corresponding source set point (corresponding to the at least one best match target set); and determining a transformation (matrix) between the final source set points and the final target set points or between the best match target set points and the source set points;applying the transformation (matrix) (to the source 3D point cloud) to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud.

[0192] Appendix S39. A computer program product that, when executed on a computer, causes the computer to perform a method, the method comprising: determining a 2D overlap between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively; the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap into a plurality of cells; and determining at least one overlap cell that includes at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection (or determining at least one cell comprising at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection as at least one overlap cell), and performing a comparison process for the (or the) overlap cell, said comparison process (for the overlap cell) determining a plurality of target sets of points, each of which has a first (target) set of points of the target 3D point cloud that fall within the overlap cell (in the target 2D projection), and a second target set of points and a further (target) set of points of the target 3D point cloud that fall within cells adjacent / surrounding the overlap cell (in the target 2D projection); determining points of a source set that have points of the source 3D point cloud that fall within the overlap cell (in the source 2D projection); comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; selecting a best match target set based on the similarity scores; determining a transformation (matrix) between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; and applying the transformation (matrix) (to the source 3D point cloud) to register the source 3D point cloud to the target 3D point cloud or vice versa.

[0193] Appendix S40. An information processing device having a memory and a processor connected to the memory, wherein the processor determines a 2D overlap portion between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap portion corresponding to an overlap region between the source geographic area and the target geographic area, divides the 2D overlap portion into a plurality of cells, determines at least one overlap cell including at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection (or determines at least one cell including at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection as at least one overlap cell), and performs a comparison process for the (or the) overlap cell, wherein the comparison process (for the overlap cell) is performed by comparing the (or the) overlap cell with the (or the) target 2D projection. determining a plurality of target sets of points, each having a first (target) set of points of the target 3D point cloud that fall within the overlap cell (in the target 2D projection) and a second target set of points and a further (target) set of points of the target 3D point cloud that fall within cells adjacent / surrounding the overlap cell (in the target 2D projection); determining a source set of points having points of the source 3D point cloud that fall within the overlap cell (in the source 2D projection); comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; and selecting a best match target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set;an information processing apparatus configured to apply the transformation (matrix) (to the source 3D point cloud) to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud;

[0194] This disclosure extends to Appendix (B) below.

[0195] Appendix B1. A computer-implemented method comprising: estimating a 3D overlap volume between a source 3D (three-dimensional) point cloud and a target 3D point cloud based on an overlap region between the source geographic area and the target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; dividing the 3D overlap volume into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud (or determining at least one cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud as at least one overlap cell); performing a comparison process for the (or the) overlapping cells, said comparison process (for the overlapping cells) comprising: determining a plurality of target sets of points, each of which comprises a first (target) set of points of the target 3D point cloud that are within the overlapping cell, and a second (target) set of points and a further (target) set of points of the target 3D point cloud that are within cells adjacent / surrounding the overlapping cell; determining a source set of points that have points of the source 3D point cloud that are within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; selecting a best matching target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set; applying the transformation (matrix) (to the source 3D point cloud) to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud.

[0196] Appendix B2. The computer-implemented method of Appendix B1, wherein estimating the 3D overlap volume comprises determining the 3D overlap volume defined in a first dimension and a second dimension by a 2D overlap between a source 2D projection of the source 3D point cloud and a target 2D projection of the target 3D point cloud, and in a third dimension by upper and lower boundaries of the source 3D point cloud and the target 3D point cloud in the third dimension, wherein the 2D overlap corresponds to the overlap region between the source geographical area and the target geographical area.

[0197] Appendix B3. The computer-implemented method of Appendix B1, wherein estimating the 3D overlap volume comprises determining a 2D overlap between a source 2D projection of the source 3D point cloud and a target 2D projection of the target 3D point cloud, the 2D overlap corresponding to the overlap region between the source geographical area and the target geographical area, and determining the 3D overlap volume defined by the 2D overlap in a first dimension and a second dimension, and defined in a third dimension by upper and lower boundaries of the source 3D point cloud and the target 3D point cloud in the third dimension.

[0198] Appendix B4. The computer-implemented method of any of Appendixes B1-B3, wherein the overlapping cell and its surrounding cells form a (3D) Moore neighborhood with the overlapping cell as a center cell.

[0199] Appendix B5. A computer-implemented method according to any of Appendixes B1 to B4, wherein the target set (for the overlapping cell) is determined as the points of the (target) set of the target 3D point cloud that are within each cell in the (3D) Moore neighborhood, with the overlapping cell as the center cell (of the (3D) Moore neighborhood).

[0200] Appendix B6. The computer-implemented method of any one of Appendix B4 to B5, wherein the surrounding cells of the overlapping cell include orthogonal and diagonal adjacent cells to the overlapping cell.

[0201] Appendix B7. The computer-implemented method of any of Appendixes B2 to B6, comprising determining the source 2D projection of the source 3D point cloud and the target 2D projection of the target 3D point cloud.

[0202] Appendix B8. The computer-implemented method of any of Appendixes B1-B7, wherein the first and / or second threshold number is 1 or 2 or 3.

[0203] Appendix B9. The computer-implemented method of any of Appendixes B1-B8, wherein the first and second threshold numbers are the same.

[0204] Appendix B10. The computer-implemented method of any of Appendixes B2-B9, further comprising determining the overlap region between the source geographic area and the target geographic area based on GPS data.

[0205] Appendix B11. The computer-implemented method of any of Appendixes B1 to B10, wherein comparing the source set with each of the plurality of target sets comprises determining a source convex hull of points in the source set, determining a plurality of target convex hulls of points in the plurality of target sets, respectively, and comparing the source convex hull to each target convex hull.

[0206] Appendix B12. The computer-implemented method of any of Appendixes B1-B11, comprising determining a contribution to a similarity score for each target set based on a comparison of the source convex hull and the associated target convex hull.

[0207] Appendix B13. A computer-implemented method according to any of Appendixes B1 to B12, wherein comparing the source set with each of the plurality of target sets comprises, for each target set, performing a first comparison between the shape of the points of the source set and the shape of the points of the target set to determine a first contribution to the similarity score for the target set, and / or performing a second comparison between the (color) intensities of the points of the source set and the (color) intensities of the points of the target set to determine a second contribution to the similarity score for the target set.

[0208] Appendix B14. The computer-implemented method of any of Appendixes B1 to B13, wherein comparing the source set to each of the plurality of target sets includes performing a first and a second comparison for each target set and combining the first and second contributions to determine a similarity score for the target set.

[0209] Appendix B15. The computer-implemented method of any of Appendixes B1 to B14, wherein comparing the source set to each of the plurality of target sets includes, for each target set, performing a first comparison between the shape of the points of the source set and the shape of the points of the target set to determine a first contribution to the similarity score of the target set, performing a second comparison between the (color) intensities of the points of the source set and the (color) intensities of the points of the target set to determine a second contribution to the similarity score of the target set, and combining the first and second contributions to determine the similarity score of the target set.

[0210] Appendix B16. The computer-implemented method of any of Appendix B1 to B15, wherein comparing the source set to each of the plurality of target sets comprises performing a first and / or second comparison process, wherein the first comparison process comprises determining a convex hull of points of the source set and a first distribution of dihedral angles of the convex hull of points of the source set, and for each target set, determining a convex hull of points of the target set and a second distribution of dihedral angles of the convex hull of points of the target set, calculating a difference between the first and second distributions of dihedral angles, and adding a contribution to the similarity score for the target set based on the calculated difference, and the second comparison process comprises determining a first distribution of (color) intensities of points in the source set and for each target set, determining a second distribution of (color) intensities of points in the target set, calculating a distance / similarity measure between the first and second distributions of intensities, and adding a contribution to the similarity score for the target set based on the calculated difference.

[0211] Appendix B17. The computer-implemented method of any of Appendixes B1 to B16, wherein comparing the source set with each of the plurality of target sets comprises determining a convex hull of points of the source set, determining a first distribution of dihedral angles of the convex hull of points of the source set, for each target set, determining a convex hull of points of the target set, determining a second distribution of dihedral angles of the convex hull of points of the target set, calculating a difference between the first and second distributions of dihedral angles, and adding a contribution to the similarity score for the target set based on the calculated difference.

[0212] Appendix B18. The computer-implemented method of Appendix B16 or B17, wherein the variance is the Kullback-Leibler divergence.

[0213] Appendix B19. The computer-implemented method of any of Appendixes B11-B18, wherein determining the convex hull comprises using farthest point sampling for a set of related points.

[0214] Appendix B20. A computer-implemented method according to any of Appendixes B1 to B19, wherein comparing the source set with each of the plurality of target sets comprises, for each target set, comparing the (color) intensity of points in the source set with the (color) intensity of points in the target set.

[0215] Appendix B21. The computer-implemented method of Appendix B20, wherein comparing the (color) intensities of the points of the source set with the (color) intensities of the points of the target set comprises using a histogram-based measure.

[0216] Appendix B22. The computer-implemented method of Appendix B20 or B21, wherein comparing the colors of the points of the source set with the colors of the points of the target set comprises determining a first distribution of (color) intensities of the points of the source set and a second distribution of (color) intensities of the points of the target set, and comparing the first and second distributions of intensities.

[0217] Appendix B23. The computer-implemented method of Appendix B22, comprising determining a contribution to the similarity score for each target set based on a comparison of the first distribution of intensities with the second distribution of related intensities.

[0218] Appendix B24. The computer-implemented method of Appendix B22 or B23, wherein comparing the first and second distributions of intensities comprises determining a distance / similarity measure between the distributions.

[0219] Appendix B25. The computer-implemented method of any of Appendixes B16-B24, comprising determining a contribution to the similarity score for each target set based on the distance / similarity measure corresponding to that target set.

[0220] Appendix B26. The computer-implemented method of any of Appendixes B22-B25, wherein the first and second distributions of intensities are determined as histograms of intensities of points in the relevant sets.

[0221] Appendix B27. The computer-implemented method of any of Appendixes B22 to B26, wherein the first and second distributions of intensity are distributions of RGB or CYMK colors of points in the related set or distributions of LiDAR intensity of points in the related set.

[0222] Appendix B28. The computer-implemented method of any of Appendixes B22 to B27, wherein the first and second distributions of intensities are determined as histograms of RGB or CYMK colors or LiDAR intensities of points in the relevant sets.

[0223] Appendix B29. The computer-implemented method of any of Appendix B16-B28, wherein the distance / similarity measure comprises a Bhattacharyya distance.

[0224] Appendix B30. A computer-implemented method according to any of Appendixes B1 to B29, wherein comparing the source set with each of the plurality of target sets comprises determining a first distribution of (color) intensities of points in the source set, determining for each target set a second distribution of (color) intensities of points in the target set, calculating a distance / similarity measure between the first and second distributions of intensities, and adding a contribution to a similarity score for the target set based on the calculated difference.

[0225] Appendix B31. The computer-implemented method of Appendix B30, wherein the distance / similarity measure comprises a Bhattacharya distance.

[0226] Appendix B32. The computer-implemented method of Appendix B30 or B31, wherein the first and second distributions of intensities are determined as histograms of RGB or CYMK colors or LiDAR intensities of points in the relevant sets.

[0227] Appendix B33. The computer-implemented method of any of Appendixes B12 to B32, wherein comparing the source set to each of the plurality of target sets includes, for each target set, combining the first and second contributions to determine the similarity score for the target set.

[0228] Clause B34. The computer-implemented method of any of Clauses B1-B33, wherein determining the transformation matrix comprises determining the transformation matrix using an iterative closest point (ICP) algorithm.

[0229] Appendix B35. A computer-implemented method described in any of Appendixes B1 to B34, comprising, when multiple overlapping cells are determined, performing the comparison process for each overlapping cell (to determine multiple best match target sets corresponding to multiple source sets, each corresponding to an overlapping cell), determining a final target set point having a best match target set point of at least one of the best match target sets, determining a final source set point having a corresponding at least one source set point (corresponding to the at least one best match target set), and determining the transformation matrix between the final source set point and the final target set point.

[0230] Appendix B36. The computer-implemented method of Appendix B35, wherein determining the points of the final target set includes selecting at least one of the best match target sets having a similarity score above a threshold score or having the highest similarity score, and determining the points of the final target set as including the points of the selected at least one best match target set.

[0231] Appendix B37. The computer-implemented method of any of Appendixes B1 to B36, wherein the source 2D projection has points that represent points of the source 3D point cloud and the target 2D projection has points that represent points of the target 3D point cloud.

[0232] Appendix B38. The computer-implemented method of any of Appendixes B1-B37, wherein the source 3D point cloud and the target 3D point cloud comprise LiDAR scan data.

[0233] Appendix B39. A computer-implemented method for estimating a 3D overlap volume between a source 3D (three-dimensional) point cloud and a target 3D point cloud based on an overlap region between a source geographic area and a target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; dividing the 3D overlap volume into a plurality of cells; determining at least one overlap cell including at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud (or determining at least one cell including at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud as at least one overlap cell); and for each (or the) overlap cell, determining a first (or the) target 3D point cloud of the target 3D point cloud that is within the overlap cell. determining a plurality of target set points, each having points of a first (target) set of the target 3D point cloud and a second (target) set of points and a further (target) set of points of the target 3D point cloud that are within cells adjacent / surrounding the overlapping cells; determining a source set point having points of the source 3D point cloud that are within the overlapping cells; comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; and selecting a best match target set based on the similarity scores; determining a final target set point having at least one best match target set point from the best match target sets; and determining a final source set point having at least one corresponding source set point (corresponding to the at least one best match target set);determining a transformation (matrix) between the points of the final source set and the points of the final target set, or between the points of the best match target set and the points of the source set, and applying the transformation (matrix) (to the source 3D point cloud) to register the source 3D point cloud to the target 3D point cloud, or the target 3D point cloud to the source 3D point cloud.

[0234] Appendix B40. A computer program product that, when executed on a computer, causes the computer to perform a method, the method comprising: estimating a 3D overlap volume between a source 3D (three-dimensional) point cloud and a target 3D point cloud based on an overlap region between a source geographic area and a target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; dividing the 3D overlap volume into a plurality of cells; determining at least one overlap cell comprising at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud (or determining at least one cell comprising at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud as at least one overlap cell); performing a comparison process for the (or) overlap cell; ) the comparison process comprises: determining a plurality of target sets of points, each having points of a first (target) set of the target 3D point cloud that are within the overlap cell, and points of a second (target) set and further (target) sets of points of the target 3D point cloud that are within cells adjacent / surrounding the overlap cell; determining a source set of points having points of the source 3D point cloud that are within the overlap cell; comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; selecting a best match target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set; and applying the transformation (matrix) (to the source 3D point cloud);and registering the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud.

[0235] Appendix B41. An information processing device having a memory and a processor connected to the memory, wherein the processor estimates a 3D overlap volume between a source 3D (three-dimensional) point cloud and a target 3D point cloud based on an overlap region between a source geographic area and a target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; divides the 3D overlap volume into a plurality of cells; determines at least one overlap cell including at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud (or determines at least one cell including at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud as at least one overlap cell); performs a comparison process for the (or) overlap cell; and calculates the ratio (for the overlap cell). the comparison process comprises: determining a plurality of target sets of points, each having a first (target) set of points of the target 3D point cloud that are within the overlap cell, and a second (target) set of points and a further (target) set of points of the target 3D point cloud that are within cells adjacent / surrounding the overlap cell; determining a source set of points having points of the source 3D point cloud that are within the overlap cell; comparing the source set with each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets, each similarity score quantifying a similarity between the source set and the target set; selecting a best match target set based on the similarity scores; determining a transformation (matrix) between the points of the source set and the points of the best match target set or between the points of the best match target set and the points of the source set; and applying the transformation (matrix) (to the source 3D point cloud);an information processing device configured to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud;

Claims

1. 1. A computer-implemented method comprising: determining a 2D overlap between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of which includes a first target set of points of the target 3D point cloud that fall within the overlap cell, and a second target set of points and a further target set of points of the target 3D point cloud that fall within cells adjacent to the overlap cell; determining a source set of points having points of the source 3D point cloud that fall within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer-implemented method comprising:

2. 1. A computer-implemented method comprising: estimating a 3D overlap volume between a source 3D point cloud and a target 3D point cloud based on an overlap region between the source geographic area and the target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; dividing the 3D overlap volume into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of the target 3D point cloud having a first target set of points that are within the overlapping cell, and a second target set of points and a further target set of points that are within cells adjacent to the overlapping cell; determining a source set of points having points of the source 3D point cloud that are within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer-implemented method comprising:

3. The computer-implemented method of claim 1 , further comprising determining the overlap area between the source geographic area and the target geographic area based on GPS data.

4. 3. The computer-implemented method of claim 2, wherein estimating the 3D overlap volume comprises determining the 3D overlap volume defined in a first dimension and a second dimension by a 2D overlap between a source 2D projection of the source 3D point cloud and a target 2D projection of the target 3D point cloud, and in a third dimension by upper and lower boundaries of the source 3D point cloud and the target 3D point cloud in the third dimension, wherein the 2D overlap corresponds to the overlap region between the source geographical area and the target geographical area.

5. 5. The computer-implemented method of claim 1, wherein the target set for the overlapping cell is determined as points of the target set of the target 3D point cloud that fall within each cell in the Moore neighborhood, with the overlapping cell as the center cell of the Moore neighborhood.

6. 5. The computer-implemented method of claim 1, wherein comparing the source set with each of the plurality of target sets comprises determining a source convex hull of points in the source set, determining a plurality of target convex hulls of points in the plurality of target sets, respectively, and comparing the source convex hull to each target convex hull.

7. Comparing the source set with each of the plurality of target sets includes, for each target set: performing a first comparison between the shapes of the points of the source set and the shapes of the points of the target set to determine a first contribution to the similarity score for the target set; and / or performing a second comparison between the intensities of the points of the source set and the intensities of the points of the target set to determine a second contribution to the similarity score for the target set; 5. A computer-implemented method according to any one of claims 1 to 4, comprising:

8. If multiple overlapping cells are determined, performing said comparison process for each overlapping cell; determining a final target set point having at least one best match target set point of the best match target sets, and determining a final source set point having at least one corresponding source set point; determining the transformation between the final source set of points and the final target set of points; 5. A computer-implemented method according to any one of claims 1 to 4, comprising:

9. 9. The computer-implemented method of claim 8, wherein determining the points of the final target set comprises selecting at least one of the best match target sets having a similarity score above a threshold score or having the highest similarity score, and determining the points of the final target set as including points of the selected at least one best match target set.

10. 1. A computer program which, when executed on a computer, causes the computer to carry out a method, the method comprising: determining a 2D overlap between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of which includes a first target set of points of the target 3D point cloud that fall within the overlap cell, and a second target set of points and a further target set of points of the target 3D point cloud that fall within cells adjacent to the overlap cell; determining a source set of points having points of the source 3D point cloud that fall within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer program comprising:

11. 1. A computer program which, when executed on a computer, causes the computer to carry out a method, the method comprising: estimating a 3D overlap volume between a source 3D point cloud and a target 3D point cloud based on an overlap region between the source geographic area and the target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; dividing the 3D overlap volume into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of the target 3D point cloud having a first target set of points that are within the overlapping cell, and a second target set of points and a further target set of points that are within cells adjacent to the overlapping cell; determining a source set of points having points of the source 3D point cloud that are within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; A computer program comprising:

12. An information processing device having a memory and a processor connected to the memory, wherein the processor: determining a 2D overlap between a source 2D projection of a source 3D point cloud and a target 2D projection of a target 3D point cloud, the source 3D point cloud and the target 3D point cloud representing a source geographic area and a target geographic area, respectively, the 2D overlap corresponding to an overlap region between the source geographic area and the target geographic area; dividing the 2D overlap portion into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 2D projection and at least a second threshold number of points from the target 2D projection; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of which includes a first target set of points of the target 3D point cloud that fall within the overlap cell, and a second target set of points and a further target set of points of the target 3D point cloud that fall within cells adjacent to the overlap cell; determining a source set of points having points of the source 3D point cloud that fall within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; An information processing device configured as follows.

13. An information processing device having a memory and a processor connected to the memory, wherein the processor: estimating a 3D overlap volume between a source 3D point cloud and a target 3D point cloud based on an overlap region between the source geographic area and the target geographic area, the source 3D point cloud and the target 3D point cloud representing the source geographic area and the target geographic area, respectively; dividing the 3D overlap volume into a plurality of cells and determining at least one overlap cell that includes at least a first threshold number of points from the source 3D point cloud and at least a second threshold number of points from the target 3D point cloud; performing a comparison process on the overlapping cells, the comparison process comprising: determining a plurality of target sets of points, each of the target 3D point cloud having a first target set of points that are within the overlapping cell, and a second target set of points and a further target set of points that are within cells adjacent to the overlapping cell; determining a source set of points having points of the source 3D point cloud that are within the overlap cell; comparing the source set to each of the plurality of target sets to determine a plurality of similarity scores for each of the plurality of target sets; selecting a best matching target set based on the similarity scores; determining a transformation between points of the source set and points of the best match target set or between points of the best match target set and points of the source set; applying the transformation to register the source 3D point cloud to the target 3D point cloud or the target 3D point cloud to the source 3D point cloud; An information processing device configured as follows.

Images