Method and apparatus for generating a point cloud histogram

Abstract

The technology described herein relates to methods, apparatuses, and computer readable media configured to generate a point cloud histogram. A one-dimensional histogram can be generated by determining, for each 3D point of a 3D point cloud, a distance to a reference. For each histogram entry, distances within the distance range of the entry are added to generate the one-dimensional histogram. A two-dimensional histogram can be generated by determining, for each 3D point, an orientation having at least a first value of a first component and a second value of a second component to determine a set of orientations. A two-dimensional histogram can be generated based on the set of orientations. Each bin can be associated with a range of values of the first component and the second component. Orientations having a first value within the range of first values and a second value within the range of second values of the bin can be added for each bin.

Description

[0001] Divisional application

[0002] This application is a divisional application of Chinese invention patent application number 202180061425.1, which was filed on May 10, 2021, and is entitled "Method and apparatus for generating point cloud histograms".

[0003] Related applications

[0004] This application claims priority to U.S. Provisional Application No. 63 / 023,163, filed May 11, 2020, entitled “METHODS AND APPARATUS FOR GENERATING POINT CLOUD HISTOGRAMS,” and U.S. Provisional Application No. 63 / 065,456, filed August 13, 2020, entitled “METHODS AND APPARATUS FOR GENERATING POINT CLOUD HISTOGRAMS,” the entire contents of which are incorporated herein by reference. Technical Field

[0005] The techniques described in this article generally relate to methods and apparatuses for machine vision, including techniques for generating histograms of point cloud data. Background Technology

[0006] Machine vision systems can include robust imaging capabilities, including three-dimensional (3D) imaging devices. For example, a 3D sensor can image a scene to generate a set of 3D points, each containing a (x, y, z) position in a 3D coordinate system (e.g., where the z-axis represents the distance from the 3D imaging device). Such a 3D imaging device can generate a 3D point cloud, which includes a set of 3D points captured during the 3D imaging process. However, the absolute number of 3D points in a 3D point cloud can be enormous (e.g., compared to the 2D data of the scene). Furthermore, a 3D point cloud may only contain pure 3D data points and therefore may not include data indicating relationships between / among the 3D points, or other information (such as surface normal information). Processing 3D points without data indicating relationships between other points can be complex. Therefore, while 3D point clouds can provide a wealth of 3D data, performing machine vision tasks on 3D point cloud data can be complex, time-consuming, and resource-intensive. Summary of the Invention

[0007] According to the disclosed subject matter, apparatus, systems, and methods are provided for improved machine vision techniques, particularly for providing improved machine vision techniques for summarizing point cloud data (e.g., which can be used to compare objects in the point cloud data). In some embodiments, these techniques are used to generate histograms of point cloud data. The histograms can be of various dimensions, such as one-dimensional and / or two-dimensional histograms. Histograms can be generated using various metrics, including those based on a reference to the point cloud data. For example, a one-dimensional histogram can be generated based on the distances of 3D points to a reference plane, to representative points of the 3D point cloud (e.g., centroids), etc. As another example, a two-dimensional histogram can be generated based on information determined from the 3D point cloud (e.g., based on surface normals, vectors, etc.).

[0008] Some aspects relate to a computerized method for generating a histogram of a three-dimensional (3D) point cloud. The method includes: receiving data indicating a 3D point cloud comprising a plurality of 3D points; determining a reference spatially related to the 3D point cloud; determining, for each of the plurality of 3D points, a distance to the reference to generate a distance group of the plurality of 3D points; and generating a histogram based on the distance group, comprising, for each entry in the entry group, inserting distances from the distance group that are within a distance range associated with that entry.

[0009] According to some examples, the method includes: generating a 3D voxel grid for at least a portion of a 3D point cloud, wherein each voxel in the 3D voxel grid includes the same group of dimensions; determining, for each voxel in the 3D voxel grid, whether one or more of a plurality of 3D data points are within the voxel to generate an associated group of 3D points for the voxel; determining, for each voxel in the 3D voxel grid with the associated group of 3D points, a single 3D data point for the voxel based on the associated group of 3D data points; and storing the single 3D data point in the voxel. Determining the distance group may include, for each voxel in the 3D voxel grid, determining the distance from the single 3D data point to a reference to generate the distance group.

[0010] According to some examples, the reference is a two-dimensional (2D) reference plane, and determining the distance to each 3D point to generate this set of distances includes determining the shortest distance from each 3D point to the reference plane.

[0011] Based on some examples, the reference is a reference line, and determining the distance to each 3D point to generate this distance set includes determining the shortest distance from each 3D point to the reference line.

[0012] According to some examples, the method includes determining an estimated centroid of a 3D point cloud, where a reference is the estimated centroid. Determining the distance to each 3D point to generate this set of distances may include determining the distance from each 3D point to the estimated centroid.

[0013] According to some examples, the method includes comparing a histogram with a second histogram generated for a second 3D point cloud to determine a measure of similarity between the 3D point cloud and the second 3D point cloud.

[0014] Some aspects relate to a computerized method for generating a histogram of a three-dimensional (3D) point cloud. The method includes receiving data indicating a 3D point cloud comprising a plurality of 3D points. The method includes generating an orientation group, including determining an orientation for each 3D point in the 3D point cloud, wherein the orientation includes at least a first value of a first component and a second value of a second component. The method includes generating a histogram comprising a group of intervals based on the orientation group, wherein each interval in the interval group is associated with a range of first values ​​of the first component and a range of second values ​​of the second component; and generating the histogram includes, for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value respectively within the range of first and second values ​​associated with the interval.

[0015] According to some examples, the interval group is arranged in two dimensions, where the first dimension is associated with the first component and the second dimension is associated with the second component.

[0016] According to some examples, the first component includes the tilt angle, and the second component includes the azimuth angle.

[0017] According to some examples, the method further includes: generating a 3D voxel mesh for at least a portion of a 3D point cloud, wherein each voxel in the 3D voxel mesh comprises the same group of dimensions; determining, for each voxel in the 3D voxel mesh, whether one or more of a plurality of 3D data points are within the voxel to generate an associated group of 3D points for the voxel; determining, based on the associated group of 3D data points, a single 3D data point of the voxel for each voxel in the 3D voxel mesh with the associated group of 3D points; and storing the single 3D data point in the voxel. Generating the group of orientations includes determining the orientation of the single 3D data point for each voxel in the 3D voxel mesh to generate the group of orientations.

[0018] According to some examples, generating this orientation group involves determining the orientation of each 3D point in the 3D point cloud based on a fixed coordinate system associated with the 3D point cloud.

[0019] According to some examples, generating this orientation group involves determining the orientation of each 3D point in the 3D point cloud based on the local coordinate system associated with the 3D point in the 3D point cloud.

[0020] According to some examples, the method includes comparing a histogram with a second histogram associated with a second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud. Comparing the histogram with the second histogram includes determining a first set of peaks in the histogram and a second set of peaks in the second histogram, and determining a correspondence between at least a portion of the first set of peaks and at least a portion of the second set of peaks.

[0021] Some embodiments relate to a non-transitory computer-readable medium including instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to perform any of the methods described herein.

[0022] Some embodiments relate to a system including a memory storing instructions and a processor configured to execute instructions to perform any of the methods described herein.

[0023] Therefore, the features of the disclosed subject matter have been outlined quite broadly to facilitate a better understanding of the subsequent detailed embodiments and to facilitate a better understanding of the invention's contribution to the prior art. Additional features of the disclosed subject matter will, of course, be described below and will form the subject matter of the appended claims. It should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. Attached Figure Description

[0024] In the accompanying drawings, each identical or nearly identical component illustrated in the various figures is indicated by the same reference numerals. For clarity, not every component is labeled in every figure. The drawings are not necessarily drawn to scale, but rather focus on illustrating various aspects of the technology and equipment described herein.

[0025] Figure 1 An exemplary machine vision system according to some embodiments is shown.

[0026] Figure 2 This is a flowchart illustrating an exemplary computerized method for generating a histogram of a three-dimensional point cloud according to some embodiments.

[0027] Figure 3A and Figure 3AC Two exemplary point-to-plane histograms according to some embodiments are shown.

[0028] Figure 3B and Figure 3BC Two exemplary point-to-line histograms according to some embodiments are shown.

[0029] Figure 4 , Figure 4A , Figure 4B , Figure 4C Histograms of distances from two exemplary points to the centroid are shown according to some embodiments.

[0030] Figure 5 This is a flowchart of an exemplary computerized method for generating a histogram of a 3D point cloud, according to some embodiments.

[0031] Figure 6 Two exemplary normal direction histograms of point cloud data of a frustum according to some embodiments are shown.

[0032] Figure 7 Two exemplary normal direction histograms of point cloud data for a cylindrical object according to some embodiments are shown.

[0033] Figure 8 Two exemplary normal direction histograms of a hemispherical object according to some embodiments are shown.

[0034] Figure 9 Two exemplary normal direction histograms of urban landscape objects according to some embodiments are shown.

[0035] Figure 10 This is based on the use of some embodiments in combination Figures 6 to 9 An exemplary table of similarity scores calculated from a 2D oriented histogram. Detailed Implementation

[0036] The techniques described herein provide data simplification techniques that can be used to analyze 3D point cloud images. The inventors have recognized that conventional machine vision techniques can be significantly inefficient when processing 3D point cloud data (e.g., to determine the presence of objects within 3D point cloud data). 3D point clouds typically comprise hundreds of thousands or millions of (x, y, z) points. Therefore, the inventors have realized that directly interpreting such a large number of 3D points in space can be quite challenging. For example, because 3D point clouds contain such a large number of 3D points and typically do not include information about the spatial relationships between these points, attempting to interpret a pure 3D point cloud is infeasible for many machine vision applications that perform such interpretations with limited time, limited hardware resources, etc.

[0037] In particular, it may be desirable to determine whether an object is captured in a 3D point cloud image. Conventional techniques address this problem by searching for objects in the point cloud using computationally intensive methods. For example, some methods process the set of 3D points to compute edge features indicating abrupt changes (e.g., wrinkles or tears in one or more surfaces in the field of view), compute a measure of the prevalence of such edges, and determine the presence (or absence) of an object if the prevalence exceeds a predetermined threshold. Computing such edge features requires significant computation, involving points in the neighborhood of each point in the 3D cloud. Additionally or alternatively, it may be desirable to compare objects detected in different 3D point cloud images. Like object detection, such methods may require computing edge features. For example, edge features can be computed in each point in the point cloud, the position and other properties of the edges can be compared, and finally, the comparison results can be aggregated into a similarity measure of the objects.

[0038] As another example, some methods for object detection, classification, and / or registration compute point meshes or surface patches to determine object features (e.g., surface curvature). Computing such meshes or surface patches alone can be time-consuming, and the techniques require further processing to determine object features and build a surface model. For example, computing the surface features of a mesh or patch and building a parametric model often involves iterative optimization processes, which typically limits the use of these techniques to non-real-time applications.

[0039] The inventors have made technological improvements to machine vision technology to address these and other inefficiencies. The technique described herein simplifies 3D point clouds into 1D signals (e.g., 1D histograms) and / or 2D images (e.g., 2D histograms), which can be used to easily interpret 3D point clouds for a variety of machine vision applications. The technique also provides for the interpretation of 1D and 2D signals, such as comparing different 3D point clouds to determine if they are similar. For example, 1D or 2D histograms can be calculated for two different 3D point cloud images, and the histograms can be compared to determine if the 3D point cloud images are likely to contain the same objects. Since the large amounts of data in traditional 3D point clouds can be simplified to one or two dimensions, the technique described herein significantly improves performance and allows for the interpretation of 3D point clouds for various types of machine vision tasks. Furthermore, compared to conventional techniques, the resulting histograms can be represented using a small amount of data and therefore require minimal memory or disk space to store.

[0040] The described histogram-based techniques overcome various processing inefficiencies of conventional techniques because they do not require computationally intensive aspects such as calculating edge features, point meshes, and / or surface patches (and do not require generating surface models). Instead, the techniques described herein provide the direct determination of statistical characteristics of objects from 3D points. For example, 1D histograms can be generated based on 3D point locations, and / or 2D histograms can be generated based on point normal vectors. As an illustrative example, consider a manufacturing application where 3D sensors are mounted above a conveyor belt through which objects (e.g., cardboard boxes of different sizes, mail envelopes, plastic bags, etc.) pass. Each class of objects has different characteristics, including one or more heights above the conveyor belt, surface normal orientation, etc. Such characteristics can be represented by histograms based on 1D distance and / or histograms based on 2D orientation to detect and classify these objects in real time as they travel down the conveyor belt. For example, given a histogram of distances through objects, the height of objects above the conveyor belt can be quickly estimated using statistical measures such as median, specific percentile, pattern, etc. Additionally or alternatively, the techniques described herein can reduce noise and / or enhance the robustness of measurement results by using metrics derived from histograms.

[0041] In the following description, numerous specific details are set forth regarding the systems and methods of the disclosed subject matter, as well as the environments in which such systems and methods may operate, in order to provide a thorough understanding of the disclosed subject matter. Furthermore, it will be understood that the examples provided below are exemplary, and other systems and methods within the scope of the disclosed subject matter can be envisioned.

[0042] Figure 1 An exemplary machine vision system 100 according to some embodiments is shown. The exemplary machine vision system 100 includes a camera 102 (or other imaging acquisition device) and a computer 104. Although in Figure 1 Only one camera 102 is shown, but it should be understood that multiple cameras can be used in a machine vision system (e.g., where the point cloud is derived from point clouds from multiple cameras). Computer 104 includes one or more processors and a human-machine interface in the form of a computer monitor, and optionally one or more input devices (e.g., keyboard, mouse, trackball, etc.). Camera 102 includes a lens 106 and a camera sensor element (not shown), as well as other components. Lens 106 includes a field of view 108 and focuses light from the field of view 108 onto the sensor element. The sensor element generates a digital image of the camera field of view 108 and provides this image to the processor, which is part of computer 104. Figure 1As shown in the example, object 112 travels along conveyor 110 into the field of view 108 of camera 102. While object 112 is in the field of view 108, camera 102 can generate one or more digital images of the object for processing, as discussed further herein. In operation, the conveyor can contain multiple objects. Such objects can pass sequentially through the field of view 108 of camera 102, for example, during inspection. In this way, camera 102 can acquire at least one image of each observed object 112.

[0043] In some embodiments, camera 102 is a three-dimensional (3D) imaging device. As an example, camera 102 may be a 3D sensor that scans a scene line by line, such as a DS-line of a laser profilometer 3D displacement sensor available from the assignee of this application, Cognex Corp. According to some embodiments, the 3D imaging device may generate a set of (x, y, z) points (e.g., where the z-axis adds a third dimension, such as distance from the 3D imaging device). The 3D imaging device may use various 3D image generation techniques, such as shape-from-shading, stereo imaging, time-of-flight techniques, projector-based techniques, and / or other 3D generation techniques. In some embodiments, machine vision system 100 includes a two-dimensional imaging device, such as a two-dimensional (2D) CCD or CMOS imaging array. In some embodiments, the two-dimensional imaging device generates a 2D array of brightness values.

[0044] In some embodiments, the machine vision system processes 3D data from camera 102. The 3D data received from camera 102 may include, for example, point clouds and / or distance images. A point cloud may include a set of 3D points on or near the surface of a physical object. For example, these points may be represented by their coordinates in a straight line or other coordinate system. In some embodiments, other information may optionally be present, such as a grid or raster structure indicating which points are neighbors on the object surface. In some embodiments, information about surface features (including curvature, surface normals, edges, and / or color and albedo information), whether derived from sensor measurements or previously calculated, may be included in the input point cloud. In some embodiments, 2D and / or 3D data may be obtained from 2D and / or 3D sensors, from CAD or other solid models, and / or by preprocessing distance images, 2D images, and / or other images.

[0045] According to some embodiments, the set of 3D points may be part of a 3D point cloud within a user-specified region of interest, and / or include data specifying the region of interest within the 3D point cloud. For example, since a 3D point cloud may include so many points, it may be desirable to specify and / or define one or more regions of interest (e.g., to limit the space on which the techniques described herein are applied).

[0046] Examples of computer 104 may include, but are not limited to, a single server computer, a series of server computers, a single personal computer, a series of personal computers, a minicomputer, a mainframe computer, and / or a computing cloud. Various components of computer 104 may execute one or more operating systems, examples of which may include, but are not limited to, Microsoft Windows Server™; Novell Netware™; Red Hat Linux™, Unix, and / or custom operating systems. One or more processors of computer 104 may be configured to process operations stored in memory connected to one or more processors. Memory may include, but is not limited to, hard disk drives, flash drives, tape drives, optical drives, RAID arrays, random access memory (RAM), and read-only memory (ROM).

[0047] The techniques described herein relate to generating histograms of 3D point clouds. Histograms can be generated based on geometric aspects of a 3D point cloud, such as the position of 3D points and / or the normal direction of those points (e.g., on the surface of an object captured by the 3D point cloud). Some embodiments relate to one-dimensional (1D) histograms. 1D histograms can be generated relative to a reference (such as a reference plane, line, and / or other points spatially oriented relative to the point cloud). For example, a point-to-plane distance histogram is a 1D histogram generated based on the distance from each point (or other representation, such as a voxel) in the 3D point cloud to a selected or estimated plane. Similarly, a point-to-line distance histogram is a 1D histogram generated based on the distance from each point (or other representation, such as a voxel) in the 3D point cloud to a selected or estimated line. As another example, a point-to-centroid distance histogram is a 1D histogram generated based on the distance from each point (or other representation, such as a voxel) in the 3D point cloud to the centroid of the 3D point cloud. Some embodiments involve two-dimensional (2D) histograms. For example, a normal direction projection histogram is a 2D histogram of the unit normal direction of a 3D point cloud.

[0048] For various 3D point cloud-based applications, histograms can serve as useful feature descriptors for scenes. The techniques described in this paper (including descriptors and interpretations) are robust to variations in resolution, noise, and pose, maintaining geometric invariance. Assuming reduced dimensionality and data points, the computational cost of these techniques can be lower than conventional techniques used for comparing point clouds. Histograms can be used (e.g., directly) as input to various image processing techniques and computer vision tools, including classification, measurement, object detection, object registration, and / or deep learning.

[0049] Some embodiments of the technology involve generating 1D histograms. Figure 2This is a flowchart illustrating an exemplary computerized method 200 for generating a histogram of a three-dimensional (3D) point cloud according to some embodiments. At step 202, a machine vision system (e.g., Figure 1 The machine vision system 100 receives data indicating a 3D point cloud comprising multiple 3D points. At step 204, the machine vision system determines a reference (e.g., a reference plane, line, centroid, etc.) configured to have some spatial relationship with the 3D point cloud (e.g., selected based on the point cloud and / or based on the point cloud itself). At step 206, the machine vision system determines the distance to the reference for each of the multiple 3D points to generate a set of distances for the multiple 3D points. At step 208, the machine vision system generates a histogram based on this set of distances.

[0050] Referring to step 202, according to some embodiments, the 3D point cloud can be processed before generating the histogram. For example, techniques may include voxelizing the 3D point cloud. The machine vision system can generate a 3D voxel grid for at least a portion of the 3D point cloud (e.g., the portion of interest for which a histogram is to be calculated), wherein each voxel in the 3D voxel grid has the same dimensions (e.g., the same length, width, and height). The machine vision system can determine for each voxel in the 3D voxel grid whether the location of one or more of a plurality of 3D data points falls within that voxel to generate an associated set of 3D points for that voxel. It should be understood that some voxels may be empty (e.g., where the locations of 3D points do not fall within those voxels).

[0051] According to some embodiments, a voxel can store its associated set of 3D points (e.g., for subsequent processing, such as calculating the average and / or median). According to some embodiments, the set of 3D points can be processed before being stored in the voxel to reduce the number of data points stored in the voxel grid. For example, for each voxel in a 3D voxel grid, the machine vision system can determine a single 3D data point for that voxel based on its associated set of 3D data points and store that single 3D data point in the voxel (e.g., by determining the centroid, average point value, etc.). Histograms can be generated based on the voxel data. For example, suppose a set of 3D points is stored in voxels such that each voxel includes zero 3D points (e.g., if no 3D points are located in the voxel) or one 3D point (e.g., if a single 3D point falls in the voxel, or if multiple points exist, then representative 3D data points are generated for the multiple points). A machine vision system can generate a set of distances by determining the distance from a single 3D data point to a reference for each voxel in a 3D voxel grid that stores 3D data points.

[0052] Referring to steps 204 to 208, according to some embodiments, when a voxel mesh is used to represent the 3D point cloud at step 202, it can be determined that the reference in step 204 is spatially related to the voxel mesh. For step 206, the distance from a representative point of each voxel in the mesh to the reference can be measured to generate a set of distances for generating a histogram at step 208.

[0053] Referring to step 204, various references can be used to compute the point cloud. According to some embodiments, the reference is a plane, such as a 2D reference plane. The reference plane can be determined using various techniques. For example, a user can specify a 2D reference plane as input. As another example, a user can specify a region of interest (ROI) in the point cloud, where the points contained can be used to extract a plane as the reference plane. The reference plane can be extracted from the ROI using various techniques. For example, the reference plane can be extracted from the ROI by fitting a plane using some and / or all points contained within the ROI using the least squares method, and / or by fitting a plane with the maximum number of inlays on the contained points using the Random Sample Consensus (RANSAC) technique, and so on.

[0054] According to some embodiments, the reference can be a line estimated based on a 3D point cloud. The reference line can be determined based on the region of interest using various techniques as described herein. For example, the reference line can be determined using the least squares method, fitting a line using some and / or all points contained within the region of interest. As another example, the RANSAC technique can be used to fit a line with the maximum number of inlays on some and / or all points contained within the region of interest, thereby determining the reference line. In some embodiments, the reference line can be determined based on one or more 3D shapes extracted from points in the region of interest (e.g., the axis of a cylinder, the intersection of two non-parallel planes, etc.).

[0055] According to some embodiments, the reference can be a point estimated based on a 3D point cloud, such as the estimated centroid or centroid of the 3D point cloud. A machine vision system can process a 3D point cloud (e.g., 3D points and / or voxels) to determine the estimated centroid and use it as a reference point. While calculating the centroid is an example of a technique that can be used to calculate a reference point, it should be understood that other methods can be used in conjunction with the techniques described herein. In particular, the reference point used to create a histogram can be determined as needed and should not be limited to the centroid of the object being examined. For example, a reference point can be created by determining a 3D shape and extracting the centroid from that 3D shape, and / or calculating the centroid from a subgroup of 3D points, etc.

[0056] Referring to step 206, if the reference is a plane, the machine vision system can determine the distance of each 3D point by determining the distance of each 3D point to the reference plane. This distance can be determined by calculation (e.g., the shortest distance from each point to the reference plane, and / or the distance of each point along the projection, etc.). Such a 1D point-to-plane histogram can represent the distribution of distances from points to the reference plane. By eliminating the influence from noise points and outliers, the point-to-plane histogram can be used to robustly measure the distance / height from (e.g., noisy) 3D points to the reference plane. In some embodiments, if the reference is a line, the machine vision system can determine the distance of each 3D point by determining the distance of each 3D point to an estimated line (e.g., the shortest distance to the estimated line). Such a 1D point-to-line histogram can represent the identification of surface points of an object.

[0057] Referring to step 208, in some embodiments, the histogram includes a set of entries (e.g., a one-dimensional bar chart and / or a dotted line chart, etc.). For example, each entry may be associated with a distance and / or a range of distances. According to some embodiments, when calculating the histogram, the histogram entries may represent signed or unsigned distances from each element of the 3D point cloud to a reference. Generating the histogram may include: for each entry in the histogram, adding a set of distances that satisfy the range of distances associated with the entry and / or distances within the range of distances associated with the entry. For example, adding may include determining a count of the distances for each entry in the histogram, for example, by discretizing / quantizing their values ​​such that the distances are considered to belong to an entry in the histogram.

[0058] Referring to steps 206 to 208, in some embodiments, if the reference is a point (e.g., a centroid), the machine vision system can determine the distance to each 3D point by determining the distance from each 3D point to the estimated centroid. Such a 1D point-to-centroid histogram can represent the identifiers of surface points of an object. The point-to-centroid histogram can be used as a valid feature for point cloud-based object recognition and / or classification, etc.

[0059] Figure 3A Two exemplary point-to-planar histograms of an urban landscape object according to some embodiments are shown. Figure 3A An exemplary histogram 300 is shown, illustrating the point-to-plane distances of the point cloud 302 of a cylindrical object within box 304 to plane 306 (which is located at the base of box 304). The X-axis of histogram 300 represents the distance from the reference plane 306, and the Y-axis represents the number of points at a specific distance. Figure 3ACAn exemplary histogram 350 is also shown, illustrating the point-to-plane distances of the point cloud 352 of the three planar surfaces within box 354 to plane 356 (which is also located at the base of box 354). Similar to histogram 300, the X-axis of histogram 350 represents the distance to reference plane 356, and the Y-axis represents the number of points at a specific distance. As described herein, the histogram distances in these examples are calculated by determining the shortest distance from each point to the reference plane.

[0060] In some embodiments, as described herein, histograms 300 and 350 can be compared (e.g., to determine whether objects captured by a 3D image are similar). The similarity score between the two histograms 300 and 350 is 0.296643, where the calculated score value can range from 0 to 1.0, where 1.0 indicates that the two compared histograms are identical, and a value of 0 indicates minimum similarity. Therefore, in this example, a score of 0.296643 indicates that the objects within regions 304 and 354 are not similar. The similarity score for this example is calculated using a histogram intersection metric by comparing the sum of the minimum normalized frequencies across all bins between the two histograms.

[0061] Figure 3B Two exemplary point-to-line histograms of urban landscape objects according to some embodiments are shown. Figure 3B An exemplary histogram 370 shows the point-to-line distance from the point cloud 372 of the cylindrical object within box 374 to line 376 (the axis of the cylinder). Figure 3BC An exemplary histogram 380 is also shown, illustrating the point-to-line distances from the point cloud 382 to line 386 of the three planar surfaces within box 384. The X-axis of histograms 370 and 380 represents the distance to the corresponding reference lines 376 and 386, and the Y-axis represents the number of points with a specific shortest distance. The similarity score between the two histograms 300 and 380 is 0.191645, where the calculated score value can range from 0 to 1.0, where 1.0 indicates that the two compared histograms are identical, and a value of 0 indicates minimum similarity. Figure 3A Similarly, the score is calculated by summing the minimum normalized frequencies. Therefore, in this example, a score of 0.191645 indicates that the objects in regions 374 and 384 are dissimilar.

[0062] Figure 4 Histograms of distances from two exemplary points to the centroid are shown according to some embodiments. Figure 4A It shows Figure 4 An exemplary histogram 400 of the distance from the point to the centroid of the point cloud 402 on the surface of the frustum inside the box 404 is shown. Figure 4C It also shows Figure 4BAn exemplary histogram 450 shows the distance from points to the centroid of the point cloud 452 of the sphere within box 454. Since the centroid is within the object, the centroids of the frustum and the sphere are not located within it. Figure 4B As shown in the diagram. The X-axis of histograms 400 and 450 represents the distance from the associated centroid, and the Y-axis represents the number of points at a specific distance. The similarity score of the two histograms, 400 and 450, is 0.835731, indicating a higher similarity than... Figure 3A , Figure 3AC Histograms 300 and 350 show higher similarity.

[0063] According to some embodiments, the technology can generate 2D histograms representing 2D information (e.g., information associated with multiple directions, such as the x, y, and z directions). For example, a 2D normal direction histogram can be generated to represent the distribution of unit normal directions of points in 3D space. Figure 5 This is a diagram of an exemplary computerized method 500 for generating a histogram of a 3D point cloud according to some embodiments. At step 502, a machine vision system receives a 3D point cloud having a plurality of 3D points. At step 504, the machine vision system generates a set of orientations. The machine vision system determines the orientation of each 3D point in the 3D point cloud.

[0064] In some embodiments, orientation includes a first value of a first component (e.g., tilt angle) and a second value of a second component (e.g., azimuth angle). According to some embodiments, for direction or orientation, the tilt angle can be an angle ranging from 0 degrees to 180 degrees relative to the direction of the positive Z-axis. The azimuth angle can be an angle projected onto the positive X-axis in the XY plane ranging from 0 degrees to 360 degrees. The azimuth angle can be periodic with a period of 360 degrees.

[0065] At step 506, the machine vision system generates a histogram based on the set of orientations. The histogram can be intervalized for the orientations and / or values ​​determined at step 504. In some embodiments, the histogram comprises a set of two-dimensional oriented intervals. For example, each interval can be associated with a first value range of a first component (e.g., a tilt index of the interval) and a second value range of a second component (e.g., an orientation index of the interval). The machine vision system generates the histogram by adding orientations for each interval, respectively, to the first and second values ​​within the first and second value ranges, which are associated with the intervals, to quantize each 3D point to an interval. As described herein, for example, adding may include determining a count of points having values ​​within a particular component range to generate each entry of the histogram, such that the point is considered to belong to an entry of the histogram by discretizing / quantizing the value of that point.

[0066] According to some embodiments, referring to steps 504 to 506, the unit normal at each point can be an ordered pair of tilt angles and azimuth angles. The normal direction histogram can be a 2D image, where the two dimensions are tilt angles and azimuth angles. Each row and each column of the 2D histogram image can correspond to intervals of tilt angles and azimuth angles, respectively. The number of rows and columns can be determined by the tilt angle range and azimuth angle range, as well as the interval size. Each value at a given pixel or interval conveys a count (e.g., frequency) of the normal directions that its tilt angle and azimuth angle fall into that interval.

[0067] According to some embodiments, a 2D histogram can be computed by performing a set of steps on each 3D point within the region of interest. For each 3D point, the machine vision system can compute its tilt angle and azimuth angle, quantize each tilt angle and azimuth angle by interval size to determine a tilt angle interval (e.g., tilt index) and an azimuth angle interval (e.g., azimuth index), and increment the pixel / interval value indexed at the aforementioned tilt index and azimuth index by 1.

[0068] According to some embodiments, optional sub-images (such as centroid images) can be determined. For example, a centroid image may be a triplet image having a direction histogram. Figure 1 A variety of pixels or intervals. The value in each interval with a sub-image index (i, j) can be the 3D centroid of the location of a point in the point cloud, where the normal direction lies in the interval (i, j) of the orientation histogram.

[0069] As described herein, according to some embodiments, 3D point clouds can be processed (e.g., voxelizing the 3D point cloud) before generating a histogram. A machine vision system can generate a 3D voxel mesh for at least a portion of the 3D point cloud. For each voxel in the 3D voxel mesh, the machine vision system can determine whether the location of one or more of a plurality of 3D data points falls within that voxel to generate an associated set of 3D points for that voxel.

[0070] According to some embodiments, a voxel may store a set of 3D points, representative points, or a set of points, normals, or other vectors associated with it. According to some embodiments, a machine vision system may determine the representative normals or vectors of a voxel. For example, for each voxel in a 3D voxel mesh, the machine vision system may determine the surface normals and / or orientations of representative 3D data points (e.g., median points) of that voxel to generate that set of orientations. As another example, the machine vision system may determine the surface normal vector using an associated set of 3D point locations, neighboring 3D data point locations, and / or information from a 3D sensor. In some embodiments, if a voxel is not associated with any 3D point, the voxel may be set to zero. In some embodiments, the technique may include determining the representative normal or vector of each voxel based on an associated set of 3D data points. For example, the representative vector may be determined by calculating component averages and / or by extracting eigenvectors from a cumulative matrix (e.g., formed by accumulating the outer product of each vector with itself, vvT), etc. As a further example, the technique may include determining the vector of each of the associated 3D points and storing a set of vectors.

[0071] Referring to computerized method 500, according to some embodiments, when a 3D point cloud is represented using a voxel mesh at step 502, the orientation at step 504 is determined using the surface normals and / or orientations of representative 3D data points for each voxel to generate a set of orientations for generating a histogram at step 506.

[0072] A 2D histogram can be generated to represent various aspects of a 3D point cloud, including global and / or local aspects. According to some embodiments, the histogram can be a global feature descriptor. For example, the set of orientations can be determined based on a fixed coordinate system associated with the 3D point cloud. For example, consider point clouds of multiple scenes, all represented in a fixed coordinate system (e.g., a client coordinate system). Because the orientation histogram from each point cloud conveys information about the desired subgroups and / or the entire 3D point cloud as a whole, it can be considered a global descriptor of the scene (e.g., in the form of a distribution of the orientations of surface point normals).

[0073] According to some embodiments, a histogram can be a local feature descriptor. For example, the set of orientations can be determined based on a local coordinate system associated with the 3D points of a 3D point cloud. For example, the histogram can be recalculated at one or more points in the 3D point cloud. Given 3D points of a point cloud, a machine vision system can establish a local coordinate system based on the 3D points, where the origin of the local coordinate system is at said points. The tilt angle and azimuth angle of the orientation of the 3D point cloud can be calculated in the local coordinate system, and the tilt angle and azimuth angle, after being intervalized as described herein, are used as local descriptors.

[0074] Various techniques can be used to obtain a local 3D coordinate system at a given point in a point cloud. According to some examples, a machine vision system can select an initial local 3D coordinate frame with its origin at that point and its Z-axis aligned with the normal direction of that point (e.g., where the X and Y axes are arbitrary). The machine vision system can establish the final local coordinate frame by first finding its K nearest neighbors to determine its X-axis, and then using their normals to compute an orientation histogram in the initial local 3D space. The machine vision system can identify the 2D interval location of the first distinguishable highest-frequency peak of the histogram (e.g., along the direction of increasing azimuth) and use the direction of its azimuth as the X-axis identifying the coordinate frame. The final local coordinate system for each point can depend on the geometry of that point's neighborhood (e.g., such that it does not change with point cloud sampling and object pose).

[0075] Figure 6 Two exemplary normal direction histograms of point cloud data of a frustum according to some embodiments are shown. Figure 6 An exemplary histogram 600 of the normals of the point cloud 602 of a frustum object in a first pose is shown. Figure 6 An exemplary histogram 650 of the normals of the point cloud 652 of the frustum object in a second pose is also shown. Both histograms 600 and 650 show five significant peaks (600A-600E and 650A-650E, respectively), each peak corresponding to a planar patch of the frustum object. The horizontal direction of images 600 and 650 represents an azimuth interval covering an azimuth angle range from 0 degrees to 360 degrees, and their vertical direction represents a tilt interval covering a tilt angle range from 0 degrees to 90 degrees. In image 600, the five peaks are 600A: (91, 42), 600B: (183, 44), 600C: (275, 2), 600D: (271, 47), and 600E: (358, 45); in image 650, the five peaks are 650A: (17, 60), 650B: (99, 25), 650C: (240, 38), 650D: (309, 67), and 650E: (331, 24), where the first and second coordinates of the pixel position indicate the azimuth and tilt angles in degrees, respectively.

[0076] Figure 7 Two exemplary normal direction histograms of point cloud data for a cylindrical object according to some embodiments are shown. Figure 7 An exemplary histogram 700 of the normals of the point cloud 702 of a cylindrical object in a first pose is shown. Figure 7An exemplary histogram 750 of the normals of the point cloud 752 of a cylindrical object in a second (different) pose is also shown. The horizontal directions of images 700 and 750 represent azimuth intervals covering an azimuth angle range from 0 degrees to 360 degrees, and their vertical directions represent tilt intervals covering a tilt angle range from 0 degrees to 90 degrees.

[0077] Histograms 700 and 750 both show a significant and strong ridge. As can be seen in images 702 and 752, the part is made of concentric cylindrical surfaces sharing a common axis. For each surface point of the part, its normal is (conceptually) perpendicular to the common axis and points away from it. Therefore, the normals of surface points lie more or less in the same three-dimensional plane perpendicular to the common axis. For example, consider a 3D circle of unit radius perpendicular to the common axis: all unit normals originate at the center of the unit circle and end on the circle, occupying some arc segment of the circle.

[0078] In image 702, the part is positioned such that its axis is roughly aligned with the X direction of the 3D coordinate space of the point cloud; when the normals of the surface points are projected into the XY domain, their projections will lie on a line in the XY domain (e.g., corresponding to two opposite directions with an azimuth difference of 180 degrees). This is reflected by the two (almost) vertical ridges 700A and 700B shown in image 700, whose azimuth distance is approximately half of the entire X dimension (360 degrees). As another example, when the normals of the surface points are projected onto the Z-axis of the coordinate space, the projection will take tilt values ​​from 0 to the maximum tilt angle (<90 degrees), depending on the field of view of the 3D sensor used to capture the point cloud.

[0079] In image 752, the part is rotated from the part in image 700, and the normals of each surface point are no longer perpendicular to the X direction because the common axis of the part is not parallel to the X-axis. This is conceptually analogous to projecting the points representing the 3D circles (for the covered arcs) onto the XY and Z-domains of the 3D coordinate space, respectively. This is shown in image 750 as the connected ridges 750A and 750B. Ridge 750A appears broken because it is a flipped U-shape, separated at 360-degree azimuth angles due to the periodicity of the orientation directions.

[0080] Figure 8 Two exemplary normal direction histograms of a hemispherical object according to some embodiments are shown. Figure 8 An exemplary histogram 800 of the normals of the point cloud 802 of a hemisphere in a first pose is shown. Figure 8An exemplary histogram 850 of the normals of the point cloud 852 of the hemisphere in the second pose is also shown. Both histograms 800 and 850 show a uniform distribution of the normal directions. The horizontal directions of histograms 800 and 850 represent azimuth intervals covering the azimuth angle range from 0 degrees to 360 degrees, and their vertical directions represent tilt intervals covering the tilt angle range from 0 degrees to 90 degrees.

[0081] In both images 800 and 850, there are some “blank” spaces 800A and 850A, which correspond to the range of tilt angles to which no normal to any point on the sphere’s surface can be projected. These “blank” areas appear near the high end of the tilt direction. As an example, consider a setup where a sphere (e.g., a ball) is imaged by a 3D sensor looking down at the sphere from above. A 3D sensor can typically only capture a specific portion of the sphere’s surface near the sensor, and how much it can observe generally depends on the sensor’s field of view (FOV) and its distance / pose from the sphere. Since the 2D histogram image is fixed in its tilt dimension (including 90 degrees) in this example, the sensor cannot capture the surface portion outside the sensor’s FOV but still on the upper hemisphere. Therefore, in this example, there are no points on the sphere’s point cloud with a tilt angle greater than the sensor’s maximum field of view (which is less than 90 degrees due to the perspective sensing model).

[0082] Figure 9 Two exemplary normal direction histograms of urban landscape objects according to some embodiments are shown. Figure 9 An exemplary histogram 900 of the normals of the point cloud 902 of an urban landscape object in a first pose is shown. Figure 9 An exemplary histogram 950 of the normals of the point cloud 952 of an urban landscape object in a second (different) pose is also shown. The horizontal directions of histograms 900 and 950 represent azimuth intervals covering an azimuth angle range from 0 degrees to 360 degrees, and their vertical directions represent tilt intervals covering a tilt angle range from 0 degrees to 90 degrees.

[0083] Histograms 900 and 950 both show similar patterns in the normal direction, but with offsets and rotations between them. Specifically, the U-shaped pattern 900A in histogram 900 is represented by the U-shaped pattern 950A in the lower left of histogram 950. Figure 9 In this example, the same component is used to capture point clouds, but with a different pose relative to the 3D sensor. This component consists of several sub-components: a top surface 902A (characterized by several pairs of planar surfaces forming the apex of a triangle), concentric cylindrical surfaces 902B (and...), and so on. Figure 7 (The same as shown), two rows of box tops 902C and base surfaces 902D. The histograms shown in images 900 and 950 reflect the combination of normal information of the surfaces of these sub-parts. Figure 7As shown, the histogram of the concentric cylindrical surface has a U-shaped ridge, and its tilt and azimuth angles are more dispersed than those of the other sub-parts. For the two rows of box top and base surfaces, their normal directions are similar, and they contribute a single peak in the 2D histogram image. Since the top sub-part consists of several pairs of planar patches, each patch has a small area; therefore, its histogram is characterized by a finite number of peaks. By combining all the histograms of these sub-parts, two different orientations of the overall histogram are shown in images 900 and 950, where the U-shaped 900A, 950A correspond to the concentric cylindrical surface, and the highest peaks 900B, 950B (with the highest frequency, indicated by the darkest pixels) correspond to the normals of the two rows of box top 902C and base surface 902D.

[0084] According to some embodiments, 1D and 2D histograms can be analyzed to interpret 3D point clouds. Techniques may include comparing two (or more) histograms to compute scores indicating similarity between / among the histograms. According to some embodiments, 1D histograms can be used to measure the similarity between two sets of points. As described herein, each distance-based 1D histogram can provide a valid fingerprint for characterizing a 3D point cloud dataset. Different scoring methods (e.g., histogram intersection, Bhattacharyya metric, and normalized cross-correlation, etc.) can be used to compute the similarity between two 1D histograms to assess how similar their represented points appear.

[0085] In some embodiments, one or more rating metrics can be used to compare 1D histograms. For example, to compare two 1D histograms, the machine vision system can use one or more of four rating metrics based on intersection, dot product, Bhattacharyya metric, and normalized cross-correlation, respectively.

[0086] In some embodiments, the machine vision system determines the smaller of two relative frequency values ​​(each histogram) for each interval. Figure 1 (a number of intervals) to determine a score based on intersection. The machine vision system can calculate the score as the sum of the smaller values ​​of these relative frequencies across all intervals.

[0087] In some embodiments, both the dot product-based score and the Bhattacharyya metric score involve calculating the product of two relative frequency values ​​for each interval. The machine vision system can calculate the dot product score by summing these interval products over all intervals. The machine vision system can calculate the Bhattacharyya score by summing the square roots of these interval products over all intervals.

[0088] In some embodiments, the machine vision system can calculate a score based on the Normalized Cross Correlation (NCC) by calculating (NCC + 1) / 2. The machine vision system can then divide the above dot product score by two factors (each histogram). Figure 1 (Number of factors), calculate NCC. For each histogram, these factors can be the root mean square values ​​of the relative frequency distribution.

[0089] In some embodiments, each 1D histogram can first be normalized to produce a relative frequency distribution that sums to 1. A similarity score can then be calculated based on these two relative frequency distributions. Regardless of the metric used by the machine vision system, the resulting score will range from 0 to 1, where a higher score indicates greater similarity between the two histograms.

[0090] In some embodiments, to compare two 2D histograms, a set of high-frequency intervals can be identified for each histogram. These intervals can be represented by frequency peaks to indicate the dominant 3D orientation of the histogram. At each frequency peak interval, its X and Y components indicate the azimuth and sloping intervals where the peak occurs, respectively; its Z component represents the frequency of the peak. In some embodiments, noise removal techniques can be applied to the histograms first to filter out intervals with frequencies below a predetermined significance threshold.

[0091] In some embodiments, a machine vision system can identify frequency peaks by locating 2D blobs (e.g., connected components) in a filtered histogram characterized by compact clusters of adjacent non-zero frequency intervals. The location (X, Y) of the peak can be the centroid of its constituent intervals (e.g., the average of its frequency-weighted azimuth and slant interval locations); the frequency value (Z) of the peak is the sum of the frequency values ​​of its constituent intervals. In some embodiments, if there are no significant high-frequency intervals in the histogram, the average value across all directions can be designated as the dominant direction.

[0092] In some embodiments, the machine vision system may seek an optimal rotation aligned with the principal orientation to minimize the difference in 3D orientation conveyed by the two sets of peak positions. The machine vision system may then obtain a rigid transformation corresponding to this optimal rotation.

[0093] In some embodiments, a measure of rotational goodness (referred to as a rotation score for illustrative purposes) is calculated based on the distance between frequency peaks considered as corresponding peaks. In some embodiments, the machine vision system first calculates the average Euclidean distance between the vertices of the unit vectors of the directions of the corresponding frequency peaks. The rotation score can be calculated as 1 minus this average distance.

[0094] Another example of a computable score is a frequency matching score. For each interval of a first histogram with non-zero frequency values, the machine vision system can rotate the representative direction (e.g., taken as the center of the interval) using the transformation described above to obtain a mapping direction. The mapped direction falls into a set of directions corresponding to one of the intervals of the second histogram. If multiple intervals of the first histogram are mapped to one interval of the second histogram, their frequencies are summed to obtain the corrected frequency. Thus, the first histogram is mapped to the space of the second histogram. For each interval, an overlap metric is calculated for the two frequencies associated with that interval (i.e., the frequencies of the second histogram and the mapped frequencies of the first histogram). This overlap metric is the ratio of the smaller frequency to the larger frequency. Next, the machine vision system calculates the average of these interval overlap metrics for all intervals with non-zero frequency content. Finally, the machine vision system calculates the value of the frequency matching score by applying a piecewise quadratic sigmoid function to the average overlap metric. For example, applying the sigmoid function to push the intermediate values ​​toward 0 or 1 may be beneficial.

[0095] In some embodiments, the overall similarity score between two 2D histograms can be calculated as the product of the rotation score and the frequency matching score. Similar to the scores for 1D histograms, the score can range from 0 to 1, where a higher score indicates greater similarity between the two histograms.

[0096] According to some embodiments, 2D normal orientation histograms can be used to measure the similarity between the surfaces of two objects. In a 2D histogram of orientations, column distances can linearly reflect differences in azimuth angles (depending on a period of 360 degrees), while row distances can linearly reflect differences in tilt angles. According to some embodiments, histograms calculated at different parts of a 3D point cloud and / or different viewpoints of a scene can be correlated by matching a portion of the histogram. For example, a correspondence between at least a portion of a first set of peaks and at least a portion of a second set of peaks can be determined by comparing different histograms by matching one or more peaks of one histogram with one or more peaks of another histogram. For example, consider two point clouds acquired by imaging the same object in two different poses. A machine vision system can identify the peak positions in the two histograms for the two point clouds and establish a correspondence between these frequency peaks. Such a correspondence allows, for example, the machine vision system to estimate the rotation of an object from one view to the next. In a similar manner, transitions between views can also be estimated based on the histogram of the centroid image.

[0097] According to some embodiments, 1D histograms can be generated based on 2D histograms. For example, by summing the pixel values ​​of a 2D image row by row and column by column, two 1D orientation histograms (e.g., related to tilt and azimuth angles) can be derived from the 2D histogram. Each 1D orientation histogram can provide effective features for characterizing the surface of an object. Such 1D histograms can be useful identifiers whenever the surface is fixed (e.g., using 3D-based registration). Like distance-based 1D histograms... Figure 1 Similarly, different scoring methods can be used to calculate the similarity between two 1D orientation histograms representing two surfaces.

[0098] As described herein, techniques can be used to create global descriptors, such as providing a histogram of normal orientations for a global feature descriptor. Global histograms can be used with existing image processing techniques for various applications, such as object recognition, classification, and / or registration. According to some embodiments, the highest frequency peaks can be extracted from the histogram. For example, such peaks can provide useful statistical information about the surface properties of an object. Each peak can correspond to a dominant surface normal orientation in coordinate space, and its frequency can indicate the number of points where the surface normal is located at that orientation. Thus, each peak can be characterized by orientation and frequency. Given a frequency threshold, a machine vision system can identify all peaks with frequencies exceeding that threshold. The resulting peaks can represent the object through the relationship between the number of peaks and the orientations they represent.

[0099] As described herein, techniques can be used to create local descriptors, such as local orientation histograms. According to some embodiments, a local orientation histogram can represent a point cloud using keypoints. For example, keypoints of a point cloud can be characterized by its local orientation histogram, which has peak frequencies on some non-zero rows (e.g., peaks are not on row 0). A point can be considered a keypoint if any qualifying peak is present on any non-zero row of its local orientation histogram. A peak can be qualifying if the corresponding frequency of the peak exceeds a threshold (e.g., a predetermined threshold). Each keypoint can be associated with a list of qualifying peaks. Because the number of keypoints is typically much smaller than the number of original points, a point cloud of an object with a surface of varying curvature can be simplified to a set of keypoints. Keypoints can appear around the boundaries between surface patches with different orientations. Non-keypoints can be located on surfaces with uniform orientation (e.g., indicated by a large sum of frequencies on row 0 of the histogram), slowly changing normal orientations, etc. Keypoints may play a more significant role in point cloud-based machine vision applications compared to non-keypoints.

[0100] According to some embodiments, local orientation histograms can be used to search for objects using correspondences between two sets of keypoints. For example, given an object, keypoints of that object can be extracted from a training-time point cloud acquired in a typical acquisition pose to form a reference model. At runtime, keypoints can be extracted for each point cloud acquisition to form a runtime model based on a set of keypoints. By comparing the corresponding peak values ​​of keypoint pairs, a mathematical score can be calculated between them. For each correspondence of keypoints between the reference model and the runtime model, a total score can be obtained by summing the matching scores of the corresponding pairs of keypoints. Different methods such as RANSAC can be used to correspond to the two sets of keypoints. The correspondence with the highest score can be used to determine the result.

[0101] According to some embodiments, local orientation histograms can be used to search for objects by comparing their models. For example, a model can be built for a point, which may include the point, its identifying coordinate frame, the region of interest (ROI) surrounding the point, and / or a local orientation histogram computed using points within the ROI in the local coordinate frame. Histograms of two involved models can be directly compared (e.g., for a given point, one at training time and the other at runtime). To improve efficiency, models can be built and searched at key points during both training and runtime.

[0102] Figure 10 This illustrates the use of combinations according to some embodiments. Figures 6 to 9 Table 1000 illustrates the example similarity scores calculated from the described 2D oriented histogram. (Source: [Insert Table 1000 here]) Figures 6 to 9 Images 652, 752, 852, and 952 are used to represent fingerprints for determining values ​​in columns and rows, i.e., combined with... Figures 6 to 9Histograms 650, 750, 850, and 950 are discussed separately. As shown along the diagonal, each histogram receives a perfect score of 1.0 when compared to itself. None of them score higher than 0.28, indicating that no histograms have similar objects. For this example, to calculate the similarity of two 2D histograms, for each histogram, the interval with the highest frequency (e.g., a frequency peak exceeding a user-specified threshold) is identified, each interval representing the predominant 3D orientation of the surface it represents. As described herein, the highest peak can be represented as the centroid of the interval identified as belonging to the same blob in the histogram image. If the two histograms contain enough frequency peaks, a fit is employed to associate their predominant orientations, thereby finding the optimal rotation between them as described herein (e.g., since the histograms may have been generated using images capturing objects in different poses). Then, as described herein, a similarity score is calculated by taking into account the error from the rotation estimation and the frequency similarity between the two sets of peak intervals aligned by the rotation. If no significant high-frequency intervals are identified in any 2D histogram, a similarity score is calculated using 1D orientation histograms derived from their respective tilt and azimuth angles based on correlation.

[0103] The embodiments discussed herein can be used in a variety of different applications, some of which may include, but are not limited to, part picking, 3D inspection, automotive assembly, volumetric inspection of molded plastics and cast metals, and assembly inspection in vision-guided robots. Such applications may include searching for and identifying the location and orientation of patterns of interest within an image (e.g., to guide a robot gripper or inspect an object).

[0104] The techniques operating according to the principles described herein can be implemented in any suitable manner. The processing and decision boxes in the flowcharts above represent steps and actions that may be included in algorithms that perform these different processes. The algorithms derived from these processes can be implemented as software integrated with and instructing the operation of one or more single-purpose or multi-purpose processors, can be implemented as functionally equivalent circuits such as digital signal processing (DSP) circuits or application-specific integrated circuits (ASICs), or can be implemented in any other suitable manner. It should be understood that the flowcharts included herein do not describe the syntax or operation of any particular circuit or any particular programming language or type of programming language. Rather, the flowcharts illustrate functional information that those skilled in the art can use to manufacture circuits or implement computer software algorithms for processing specific means of performing the types of techniques described herein. It should also be understood that, unless otherwise indicated herein, the specific order of steps and / or actions described in each flowchart is merely an illustration of possible algorithms and may vary in implementations and embodiments of the principles described herein.

[0105] Accordingly, in some embodiments, the techniques described herein can be embodied in computer-executable instructions implemented as software, including application software, system software, firmware, middleware, embedded code, or any other suitable type of computer code. Such computer-executable instructions can be written using a variety of suitable programming languages ​​and / or any of the programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a framework or virtual machine.

[0106] When the techniques described herein are implemented as computer-executable instructions, these instructions can be implemented in any suitable manner, including as multiple functional facilities, each providing one or more operations to perform the execution of an algorithm based on these techniques. Regardless of instantiation, a "functional facility" is a structural component of a computer system that, when integrated with and executed by one or more computers, causes those computers to perform a specific operational role. A functional facility can be part of or an entire software element. For example, a functional facility can be implemented as the function of a process, or as a discrete process, or as any other suitable processing unit. If the techniques described herein are implemented as multiple functional facilities, each functional facility can be implemented in its own way; not all functional facilities need to be implemented in the same way. Furthermore, these functional facilities can be executed in parallel and / or serially as appropriate, and can exchange information with each other using shared memory on one or more computers on which they are executed, using message passing protocols, or in any other suitable manner.

[0107] Typically, functional facilities include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. The functionality of functional facilities can generally be combined or distributed as needed within the system in which they operate. In some implementations, one or more functional facilities that implement the techniques described herein can together form a complete software package. In alternative embodiments, these functional facilities may be adapted to interact with other unrelated functional facilities and / or processes to enable software program applications.

[0108] This document has described some exemplary functional facilities for performing one or more tasks. However, it should be understood that the described functional facilities and task partitioning are merely illustrative of the types of functional facilities that can implement the exemplary techniques described herein, and embodiments are not limited to implementation in any particular number, partition, or type of functional facilities. In some implementations, all functions may be implemented in a single functional facility. It should also be understood that in some implementations, some of the functional facilities described herein may be implemented together with or separately from other functional facilities (i.e., as a single unit or separate units), or some of these functional facilities may not be implemented.

[0109] In some embodiments, computer-executable instructions implementing the techniques described herein (when implemented as one or more functional facilities or in any other way) may be encoded on one or more computer-readable media to provide functionality to the medium. Computer-readable media include magnetic media such as hard disk drives, optical media such as optical discs (CDs) or digital versatile discs (DVDs), permanent or non-permanent solid-state storage (e.g., flash memory, magnetic RAM, etc.), or any other suitable storage medium. Such computer-readable media may be implemented in any suitable manner. As used herein, a “computer-readable medium” (also referred to as a “computer-readable storage medium”) means a tangible storage medium. Tangible storage media are non-transitory and have at least one physical structural component. In a “computer-readable medium” as used herein, at least one physical structural component has at least one physical characteristic that may be altered in some way during the creation of a medium with embedded information, during the recording of information thereon, or during any other process of encoding the medium with information. For example, the magnetization state of a portion of the physical structure of a computer-readable medium may be altered during the recording process.

[0110] Furthermore, some of the aforementioned technologies include actions of storing information (e.g., data and / or instructions) in a specific manner for use by these technologies. In some implementations of these technologies (e.g., in implementations where the technology is implemented as computer-executable instructions), information may be encoded on a computer-readable storage medium. Where specific structures are described herein as advantageous formats for storing this information, these structures can be used to give the information a physical organization when encoded on the storage medium. These advantageous structures can then provide functionality to the storage medium by influencing the operation of one or more processors interacting with the information; for example, by improving the efficiency of computer operations performed by one or more processors.

[0111] In some, but not all, implementations where the technology can be implemented as computer-executable instructions, these instructions can be executed on one or more suitable computing devices operating in any suitable computer system, or one or more computing devices (or one or more processors of one or more computing devices) can be programmed to execute computer-executable instructions. A computing device or processor can be programmed to execute these instructions when they are stored in a manner accessible to the computing device or processor, such as a data storage device (e.g., an on-chip cache or instruction register, a computer-readable storage medium accessible via a bus, a computer-readable storage medium accessible via one or more networks and accessible by the device / processor, etc.). Functional facilities including these computer-executable instructions can be integrated with and direct the operation of a single multipurpose programmable digital computing device, a coordinated system of two or more multipurpose computing devices sharing processing power and jointly executing the technologies described herein, a single computing device dedicated to executing the technologies described herein, or a coordinated system of computing devices (located in one location or geographically distributed), one or more field-programmable gate arrays (FPGAs) for executing the technologies described herein, or any other suitable system.

[0112] A computing device may include at least one processor, a network adapter, and a computer-readable storage medium. For example, a computing device may be a desktop or laptop computer, a personal digital assistant (PDA), a smartphone, a server, or any other suitable computing device. A network adapter may be any suitable hardware and / or software that enables the computing device to communicate wired and / or wirelessly with any other suitable computing device over any suitable computing network. A computing network may include wireless access points, switches, routers, gateways, and / or other network devices, as well as any suitable wired and / or wireless communication media, including the Internet, for exchanging data between two or more computers. A computer-readable medium may be adapted to store data to be processed by a processor and / or instructions to be executed by a processor. A processor is capable of processing data and executing instructions. Data and instructions may be stored on a computer-readable storage medium.

[0113] Computing devices may additionally include one or more components and peripherals, including input devices and output devices. These devices are particularly useful for presenting user interfaces. Examples of output devices that can be used to provide a user interface include printers or displays for visual presentation of output, and speakers or other sound-generating devices for auditory presentation of output. Examples of input devices that can be used for a user interface include keyboards and pointing devices (such as mice, touchpads, and digitizers). As another example, computing devices may receive input information via speech recognition or in other auditory formats.

[0114] Embodiments of implementing the technology in circuits and / or computer-executable instructions have been described. It should be understood that some embodiments may be in the form of methods, at least one example of which has been provided. Actions performed as part of a method may be ordered in any suitable manner. Accordingly, embodiments may be constructed in which actions are performed in a different order than illustrated, which may include performing several actions simultaneously, even if they are shown as sequential actions in the illustrative embodiments.

[0115] The various aspects of the above embodiments can be used individually, in combination, or in various arrangements not specifically discussed in the foregoing embodiments, and therefore their application is not limited to the details and arrangements of the components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment can be combined with aspects described in other embodiments in any way.

[0116] The use of ordinal numbers such as “first,” “second,” and “third” to modify claim elements in claims does not imply any priority, order of precedence, or sequence of actions of one claim element relative to another, or the temporal order of the actions of the method. Rather, it serves merely as a label to distinguish one claim element with a certain name from another element with the same name (but using an ordinal number), thus differentiating claim elements.

[0117] Furthermore, the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The use of "including," "comprising," "having," "containing," "involving," and variations thereof in this document means to cover the items listed thereafter and their equivalents, as well as additional items.

[0118] The term “exemplary” is used herein to mean that it is used as an example, instance, or illustration. Any embodiments, implementations, processes, features, etc., described herein as exemplary should therefore be understood as illustrative examples and not as preferred or advantageous examples, unless otherwise stated.

[0119] Therefore, having described several aspects of at least one embodiment, it should be understood that various changes, modifications, and improvements will readily occur to those skilled in the art. Such changes, modifications, and improvements are intended to be part of this disclosure and are intended to remain within the spirit and scope of the principles described herein. Accordingly, the foregoing description and figures are merely exemplary.

[0120] Various aspects are described in this disclosure, including but not limited to the following:

[0121] 1. A computerized method for generating histograms of three-dimensional (3D) point clouds, the method comprising:

[0122] The receiving instruction includes 3D point cloud data containing multiple 3D points;

[0123] Determine a reference that has a spatial relationship with the 3D point cloud;

[0124] For each of the plurality of 3D points, determine the distance to the reference to generate a distance set for the plurality of 3D points; and

[0125] Generate a histogram based on the distance group, including the insertion of a distance from the distance group within the distance range associated with the entry for each entry in the entry group.

[0126] 2. The method according to 1 further includes:

[0127] A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions;

[0128] For each voxel in the 3D voxel mesh, determine whether one or more of the plurality of 3D data points are within the voxel to generate a group of 3D points associated with the voxel.

[0129] For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and

[0130] The single 3D data point is stored in the voxel.

[0131] 3. The method according to 2, wherein determining the distance group includes:

[0132] For each voxel in the 3D voxel mesh, the distance from the individual 3D data point to the reference is determined to generate the distance set.

[0133] 4. The method according to any one of 1 to 3, wherein:

[0134] The reference is a two-dimensional (2D) reference plane; and

[0135] Determining the distance to each 3D point to generate the distance set includes: determining the shortest distance from each 3D point to the reference plane.

[0136] 5. The method according to any one of 1 to 4, wherein:

[0137] The reference is a reference line; and

[0138] Determining the distance to each 3D point to generate the distance set includes: determining the shortest distance from each 3D point to the reference line.

[0139] 6. The method according to any one of 1 to 5, further comprising:

[0140] Determine the estimated centroid of the 3D point cloud, wherein the reference is the estimated centroid.

[0141] 7. The method according to 6, wherein determining the distance of each 3D point to generate the distance set comprises: determining the distance of each 3D point to the estimated centroid.

[0142] 8. The method according to any one of 1 to 7, further comprising: comparing the histogram with a second histogram generated for the second 3D point cloud to determine a measure of similarity between the 3D point cloud and the second 3D point cloud.

[0143] 9. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to...

[0144] Generate a histogram of a 3D point cloud, including:

[0145] The receiving instruction includes 3D point cloud data containing multiple 3D points;

[0146] Determine a reference that has a spatial relationship with the 3D point cloud;

[0147] For each of the plurality of 3D points, determine the distance to the reference point to generate a distance group for the plurality of 3D points; and

[0148] Generate a histogram based on the distance group, including the insertion of a distance from the distance group within the distance range associated with the entry for each entry in the entry group.

[0149] 10. The non-transitory computer-readable medium according to claim 9, wherein the instructions are further operable to cause the one or more processors to execute:

[0150] A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions;

[0151] For each voxel in the 3D voxel mesh, determine whether one or more of the plurality of 3D data points are within the voxel to generate a group of 3D points associated with the voxel.

[0152] For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and

[0153] The single 3D data point is stored in the voxel.

[0154] 11. The non-transitory computer-readable medium according to claim 10, wherein determining the distance set comprises:

[0155] For each voxel in the 3D voxel mesh, the distance from the individual 3D data point to the reference is determined to generate the distance set.

[0156] 12. The non-transitory computer-readable medium according to any one of claims 9 to 11, wherein:

[0157] The reference is a two-dimensional (2D) reference plane; and

[0158] Determining the distance to each 3D point to generate the distance set includes: determining the shortest distance from each 3D point to the reference plane.

[0159] 13. The non-transitory computer-readable medium according to any one of claims 9 to 12, wherein:

[0160] The reference is a reference line; and

[0161] Determining the distance to each 3D point to generate the distance set includes: determining the shortest distance from each 3D point to the reference line.

[0162] 14. The non-transitory computer-readable medium according to any one of claims 9 to 13, wherein the instructions are further operable to cause the one or more processors to execute:

[0163] Determining the estimated centroid of the 3D point cloud includes determining the distance from each 3D point to the estimated centroid, wherein the reference is the estimated centroid.

[0164] 15. A system comprising a memory storing instructions and at least one processor configured to execute the instructions to generate a histogram of a three-dimensional (3D) point cloud, comprising:

[0165] The receiving instruction includes 3D point cloud data containing multiple 3D points;

[0166] Determine a reference that has a spatial relationship with the 3D point cloud;

[0167] For each of the plurality of 3D points, determine the distance to the reference to generate a distance set for the plurality of 3D points; and

[0168] Generate a histogram based on the distance group, including the insertion of a distance from the distance group within the distance range associated with the entry for each entry in the entry group.

[0169] 16. The system of claim 15, wherein the instructions are further operable to cause the at least one processor to perform:

[0170] A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions;

[0171] For each voxel in the 3D voxel mesh, determine whether one or more of the plurality of 3D data points are within the voxel to generate a group of 3D points associated with the voxel.

[0172] For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and

[0173] The single 3D data point is stored in the voxel.

[0174] 17. The system according to 16, wherein determining the distance group comprises:

[0175] For each voxel in the 3D voxel mesh, the distance from the individual 3D data point to the reference is determined to generate the distance set.

[0176] 18. The system according to any one of 15 to 17, wherein:

[0177] The reference is a two-dimensional (2D) reference plane; and

[0178] Determining the distance to each 3D point to generate the distance set includes: determining the shortest distance from each 3D point to the reference plane.

[0179] 19. The system according to any one of 15 to 18, wherein:

[0180] The reference is a reference line; and

[0181] Determining the distance to each 3D point to generate the distance set includes: determining the shortest distance from each 3D point to the reference line.

[0182] 20. The system according to any one of 15 to 20, wherein the instructions are further operable to cause the at least one processor to perform:

[0183] Determining the estimated centroid of the 3D point cloud includes determining the distance from each 3D point to the estimated centroid, wherein the reference is the estimated centroid.

[0184] 21. A computerized method for generating histograms of three-dimensional (3D) point clouds, the method comprising:

[0185] The receiving instruction includes 3D point cloud data containing multiple 3D points;

[0186] Generating an orientation group includes determining the orientation of each 3D point in the 3D point cloud, wherein the orientation includes at least a first value of a first component and a second value of a second component.

[0187] Based on the orientation group, a histogram including interval groups is generated, wherein:

[0188] Each interval in the interval group is associated with a first value range of the first component and a second value range of the second component; and

[0189] Generating the histogram includes: for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value respectively within the first value range and the second value range associated with the interval.

[0190] 22. The method according to 21, wherein the group intervals are arranged in two dimensions, wherein a first dimension is associated with the first component and a second dimension is associated with the second component.

[0191] 23. The method according to any one of 21 to 22, wherein the first component includes a tilt angle and the second component includes an azimuth angle.

[0192] 24. The method according to any one of 21 to 23, further comprising:

[0193] A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions;

[0194] For each voxel in the 3D voxel mesh, determine whether one or more of the plurality of 3D data points are within the voxel to generate a group of 3D points associated with the voxel.

[0195] For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and

[0196] The single 3D data point is stored in the voxel.

[0197] 25. The method according to 24, wherein generating the orientation group comprises:

[0198] For each voxel in the 3D voxel mesh, the orientation of the individual 3D data point is determined to generate the orientation group.

[0199] 26. The method according to any one of 21 to 25, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

[0200] 27. The method according to any one of 21 to 26, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

[0201] 28. The method according to any one of 21 to 27 further includes comparing the histogram with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud.

[0202] 29. The method according to 28, wherein comparing the histogram with the second histogram includes:

[0203] Determine the first set of peak values ​​of the histogram and the second set of peak values ​​of the second histogram;

[0204] Determine the correspondence between at least a portion of the first set of peak values ​​and at least a portion of the second set of peak values.

[0205] 30. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to generate a histogram of a three-dimensional (3D) point cloud, comprising:

[0206] The receiving instruction includes 3D point cloud data containing multiple 3D points;

[0207] Generating an orientation group includes determining the orientation of each 3D point in the 3D point cloud, wherein the orientation includes at least a first value of a first component and a second value of a second component.

[0208] Based on the orientation group, a histogram including interval groups is generated, wherein:

[0209] Each interval in the interval group is associated with a first value range of the first component and a second value range of the second component; and

[0210] Generating the histogram includes: for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value respectively within the first value range and the second value range associated with the interval.

[0211] 31. The non-transitory computer-readable medium according to claim 30, wherein the instructions are further operable to cause the one or more processors to execute:

[0212] A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions;

[0213] For each voxel in the 3D voxel mesh, determine whether one or more of the plurality of 3D data points are within the voxel to generate a group of 3D points associated with the voxel.

[0214] For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and

[0215] The single 3D data point is stored in the voxel.

[0216] 32. The non-transitory computer-readable medium according to 31, wherein generating the orientation group comprises:

[0217] For each voxel in the 3D voxel mesh, the orientation of the individual 3D data point is determined to generate the orientation group.

[0218] 33. The non-transitory computer-readable medium according to any one of 30 to 32, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

[0219] 34. The non-transitory computer-readable medium according to any one of 30 to 33, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

[0220] 35. The non-transitory computer-readable medium according to any one of 30 to 34, wherein the instructions are further operable to cause the one or more processors to execute:

[0221] The histogram is compared with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud, including:

[0222] Determine the first set of peak values ​​of the histogram and the second set of peak values ​​of the second histogram; and

[0223] Determine the correspondence between at least a portion of the first set of peak values ​​and at least a portion of the second set of peak values.

[0224] 36. A system comprising a memory storing instructions and at least one processor configured to execute the instructions to generate a histogram of a three-dimensional (3D) point cloud, comprising:

[0225] The receiving instruction includes 3D point cloud data containing multiple 3D points;

[0226] Generating an orientation group includes determining the orientation of each 3D point in the 3D point cloud, wherein the orientation includes at least a first value of a first component and a second value of a second component.

[0227] Based on the orientation group, a histogram including interval groups is generated, wherein:

[0228] Each interval in the interval group is associated with a first value range of the first component and a second value range of the second component; and

[0229] Generating the histogram includes: for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value respectively within the first value range and the second value range associated with the interval.

[0230] 37. The system according to claim 36, wherein the instructions are further operable to cause the at least one processor to perform:

[0231] A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions;

[0232] For each voxel in the 3D voxel mesh, determine whether one or more of the plurality of 3D data points are within the voxel to generate a group of 3D points associated with the voxel.

[0233] For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and

[0234] The single 3D data point is stored in the voxel.

[0235] 38. The system according to any one of 36 to 37, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

[0236] 39. The system according to any one of 36 to 38, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

[0237] 40. The system according to any one of 36 to 39, wherein the instructions are further operable to cause the at least one processor to perform:

[0238] The histogram is compared with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud, including:

[0239] Determine the first set of peak values ​​of the histogram and the second set of peak values ​​of the second histogram; and

[0240] Determine the correspondence between at least a portion of the first set of peak values ​​and at least a portion of the second set of peak values.

Claims

1. A computerized method for generating a histogram of a three-dimensional point cloud to represent the 3D point cloud for comparison with other point clouds, the method comprising: The receiving instruction includes 3D point cloud data containing multiple 3D points; Generating an orientation group includes determining the orientation of each 3D point in the 3D point cloud, wherein the orientation includes at least a first value of a first component and a second value of a second component. Based on the orientation groups of the 3D points, a histogram is generated including interval groups representing the orientation groups of the 3D points, wherein the histogram is a global feature descriptor, and: Each interval in the interval group is associated with a first value range of the first component and a second value range of the second component; and Generating the histogram includes, for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value respectively within the first value range and the second value range associated with the interval to determine a representation of multiple orientations of the 3D points common to each interval; and The histogram is stored for comparison with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud.

2. The method according to claim 1, wherein, The interval group is arranged in two dimensions, wherein the first dimension is associated with the first component and the second dimension is associated with the second component.

3. The method according to claim 1, wherein, The first component includes the tilt angle, and the second component includes the azimuth angle.

4. The method according to claim 1, further comprising: A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions; For each voxel in the 3D voxel mesh, determine whether one or more of a plurality of 3D data points are within the voxel to generate an associated 3D point group for the voxel. For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and The single 3D data point is stored in the voxel.

5. The method according to claim 4, wherein, Generating the orientation group includes: For each voxel in the 3D voxel mesh, the orientation of the individual 3D data point is determined to generate the orientation group.

6. The method according to claim 1, wherein, Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

7. The method according to claim 1, wherein, Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on the local coordinate system associated with the 3D point in the 3D point cloud.

8. The method of claim 1, further comprising comparing the histogram with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud.

9. The method according to claim 8, wherein, Comparing the histogram with the second histogram includes: Determine the first set of peak values ​​of the histogram and the second set of peak values ​​of the second histogram; Determine the correspondence between at least a portion of the first set of peak values ​​and at least a portion of the second set of peak values.

10. A non-transitory computer-readable medium comprising instructions, when executed by one or more processors on a computing device, operable to cause the one or more processors to generate a histogram of a three-dimensional 3D point cloud to represent the 3D point cloud for comparison with other point clouds, comprising: The receiving instruction includes 3D point cloud data containing multiple 3D points; Generating an orientation group includes determining the orientation of each 3D point in the 3D point cloud, wherein the orientation includes at least a first value of a first component and a second value of a second component. Based on the orientation groups of the 3D points, a histogram is generated including interval groups representing the orientation groups of the 3D points, wherein the histogram is a global feature descriptor, and: Each interval in the interval group is associated with a first value range of the first component and a second value range of the second component; and Generating the histogram includes, for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value respectively within the first value range and the second value range associated with the interval to determine a representation of multiple orientations of the 3D points common to each interval; and The histogram is stored for comparison with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud.

11. The non-transitory computer-readable medium according to claim 10, wherein, The instructions are further operable to cause the one or more processors to execute: A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions; For each voxel in the 3D voxel mesh, determine whether one or more of a plurality of 3D data points are within the voxel to generate an associated 3D point group for the voxel. For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and The single 3D data point is stored in the voxel.

12. The non-transitory computer-readable medium according to claim 11, wherein, Generating the orientation group includes: For each voxel in the 3D voxel mesh, the orientation of the individual 3D data point is determined to generate the orientation group.

13. The non-transitory computer-readable medium according to claim 10, wherein, Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

14. The non-transitory computer-readable medium according to claim 10, wherein, Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on the local coordinate system associated with the 3D point in the 3D point cloud.

15. The non-transitory computer-readable medium according to claim 10, wherein, The instructions are further operable to cause the one or more processors to perform: comparing the histogram with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud, including: determining a first set of peaks in the histogram and a second set of peaks in the second histogram; And determine the correspondence between at least a portion of the first set of peaks and at least a portion of the second set of peaks.

16. A system comprising a memory storing instructions and at least one processor configured to execute the instructions to generate a histogram of a three-dimensional 3D point cloud for comparison with other point clouds, comprising: The receiving instruction includes 3D point cloud data containing multiple 3D points; Generating an orientation group includes determining the orientation of each 3D point in the 3D point cloud, wherein the orientation includes at least a first value of a first component and a second value of a second component. Based on the orientation groups of the 3D points, a histogram is generated including interval groups representing the orientation groups of the 3D points, wherein the histogram is a global feature descriptor, and: Each interval in the interval group is associated with a first value range of the first component and a second value range of the second component; and Generating the histogram includes, for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value respectively within the first value range and the second value range associated with the interval to determine a representation of multiple orientations of the 3D points common to each interval; and The histogram is stored for comparison with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud.

17. The system according to claim 16, wherein, The instructions are further operable to cause the at least one processor to execute: A 3D voxel mesh is generated for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel mesh includes the same group of dimensions; For each voxel in the 3D voxel mesh, determine whether one or more of a plurality of 3D data points are within the voxel to generate an associated 3D point group for the voxel. For each voxel in the 3D voxel mesh with associated 3D point groups, a single 3D data point of the voxel is determined based on the associated 3D data point groups; and The single 3D data point is stored in the voxel.

18. The system according to claim 16, wherein, Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

19. The system according to claim 16, wherein, Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on the local coordinate system associated with the 3D point in the 3D point cloud.

20. The system according to claim 16, wherein, The instructions are further operable to cause the at least one processor to perform: comparing the histogram with a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud, including: determining a first set of peaks in the histogram and a second set of peaks in the second histogram; and determining a correspondence between at least a portion of the first set of peaks and at least a portion of the second set of peaks.

Images