Method and device for generating point cloud histogram

Abstract

The techniques described herein relate to methods, apparatuses, and computer-readable media configured to generate a point cloud histogram. A one-dimensional histogram may be generated by determining a distance to a reference for each 3D point of the 3D point cloud. For each histogram entry, a distance within the range of distance of the entry is added to generate a one-dimensional histogram. A two-dimensional histogram may be determined by determining, for each 3D point, an orientation having at least a first value of a first component and a second value of a second component to generate an orientation group. A two-dimensional histogram may be generated based on the set of orientations. Each interval may be associated with a range of values of the first component and the second component. An orientation may be added for each interval having a first value and a second value within a first range of values and a second range of values, respectively, of the interval.

Description

[0001] Divisional application This application is a divisional application of the Chinese invention patent application with the parent application number 202180061425.1. The application date of the parent application is May 10, 2021, and the name of the invention is "Method and device for generating point cloud histogram".

[0002] Related Applications This application claims priority under 35 USC § 119(e) to U.S. Provisional Application No. 63 / 023,163, filed on May 11, 2020, entitled “METHODS AND APPARATUS FOR GENERATING POINT CLOUD HISTOGRAMS,” and U.S. Provisional Application No. 63 / 065,456, filed on August 13, 2020, entitled “METHODS AND APPARATUS FOR GENERATING POINT CLOUD HISTOGRAMS,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The techniques described herein generally relate to methods and apparatus for machine vision, including techniques for generating histograms of point cloud data. Background Art

[0004] Machine vision systems can include robust imaging capabilities, including three-dimensional (3D) imaging devices. For example, a 3D sensor can image a scene to generate a set of 3D points, each of which includes an (x, y, z) position within a 3D coordinate system (e.g., where the z-axis of the coordinate system represents the distance from the 3D imaging device). Such a 3D imaging device can generate a 3D point cloud, which includes a set of 3D points captured during the 3D imaging process. However, the absolute number of 3D points in the 3D point cloud may be huge (e.g., compared to the 2D data of the scene). In addition, the 3D point cloud may only include pure 3D data points, and therefore may not include data indicating the relationship between / among the 3D points, or other information (such as surface normal information), and processing 3D points without data indicating the relationship between other points may be complex. Therefore, although 3D point clouds can provide a large amount of 3D data, performing machine vision tasks on 3D point cloud data may be complex, time-consuming, require a large amount of processing resources, and so on. Summary of the invention

[0005] According to the disclosed subject matter, apparatus, systems and methods are provided for improved machine vision techniques, and in particular for providing improved machine vision techniques for summarizing point cloud data (e.g., which can be used to compare objects in the point cloud data). In some embodiments, these techniques are used to generate a histogram of point cloud data. The histogram can be of various dimensions, such as a one-dimensional histogram and / or a two-dimensional histogram. The histogram can be generated using various metrics, including based on a reference to the point cloud data. For example, a one-dimensional histogram can be generated based on the distance of a 3D point to a reference plane, to a representative point of the 3D point cloud (e.g., a centroid), etc. As another example, a two-dimensional histogram can be generated based on information determined from a 3D point cloud (such as, based on surface normals, vectors, etc.).

[0006] Some aspects relate to a computerized method for generating a histogram of a three-dimensional (3D) point cloud. The method includes: receiving data indicating a 3D point cloud including a plurality of 3D points; determining a reference in spatial relationship with the 3D point cloud; for each 3D point in the plurality of 3D points, determining a distance to the reference to generate a distance group for the plurality of 3D points; and generating a histogram including an entry group based on the distance group, including, for each entry in the entry group, inserting a distance from the distance group and within a distance range associated with the entry.

[0007] According to some examples, the method includes: generating a 3D voxel grid for at least a portion of a 3D point cloud, wherein each voxel in the 3D voxel grid includes the same dimension group; for each voxel in the 3D voxel grid, determining whether one or more of a plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated 3D point group, determining a single 3D data point for the voxel based on the associated 3D data point group; and storing the single 3D data point in the voxel. Determining the distance group may include determining, for each voxel in the 3D voxel grid, a distance from the single 3D data point to a reference to generate the distance group.

[0008] According to some examples, the reference is a two-dimensional (2D) reference plane, and determining the distance of each 3D point to generate the distance set includes determining a shortest distance of each 3D point to the reference plane.

[0009] According to some examples, the reference is a reference line, and determining the distance of each 3D point to generate the distance set includes determining a shortest distance of each 3D point to the reference line.

[0010] According to some examples, the method includes determining an estimated centroid of the 3D point cloud, wherein the reference is the estimated centroid. Determining a distance of each 3D point to generate the distance set may include determining a distance of each 3D point to the estimated centroid.

[0011] According to some examples, the method includes comparing the histogram to a second histogram generated for a second 3D point cloud to determine a measure of similarity between the 3D point cloud and the second 3D point cloud.

[0012] Some aspects relate to a computerized method for generating a histogram of a three-dimensional (3D) point cloud. The method includes receiving data indicating a 3D point cloud including a plurality of 3D points. The method includes generating an orientation group, including determining, for each 3D point in the 3D point cloud, an orientation of the 3D point, wherein the orientation includes at least a first value of a first component and a second value of a second component. The method includes generating a histogram including an interval group based on the orientation group, wherein each interval in the interval group is associated with a first value range of the first component and a second value range of the second component; and generating the histogram includes, for each interval in the interval group, adding an orientation from the orientation group having a first value and a second value within a first value range and a second value range, respectively, associated with the interval.

[0013] According to some examples, the group of intervals is arranged in two dimensions, wherein a first dimension is associated with the first component and a second dimension is associated with the second component.

[0014] According to some examples, the first component comprises a tilt angle and the second component comprises an azimuth angle.

[0015] According to some examples, the method further includes: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid includes the same dimension group; for each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated 3D point group, determining a single 3D data point for the voxel based on the associated 3D data point group; and storing the single 3D data point in the voxel. Generating the set of orientations includes determining, for each voxel in the 3D voxel grid, an orientation of the single 3D data point to generate the orientation group.

[0016] According to some examples, generating the orientation set includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

[0017] According to some examples, generating the orientation set includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

[0018] According to some examples, the method includes comparing the histogram to a second histogram associated with the second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud. Comparing the histogram to the second histogram includes determining a first set of peaks of the histogram and a second set of peaks of the second histogram, and determining a correspondence between at least a portion of the first set of peaks and at least a portion of the second set of peaks.

[0019] Some embodiments relate to a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to perform a method of any of the techniques described herein.

[0020] Some embodiments are directed to a system comprising a memory storing instructions and a processor configured to execute the instructions to perform a method of any of the techniques described herein.

[0021] Thus, there has been outlined rather broadly the features of the disclosed subject matter in order that the detailed description that follows may be better understood, and in order that the present contribution to the art may be better appreciated. Of course, additional features of the disclosed subject matter will be described hereinafter and will form the subject of the appended claims. It should be understood that the phraseology and terminology used herein is for descriptive purposes and should not be regarded as limiting. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In the accompanying drawings, each identical or nearly identical component illustrated in various figures is represented by the same reference numeral. For clarity, not every component is labeled in every figure. The accompanying drawings are not necessarily drawn to scale, but emphasis is placed on illustrating various aspects of the techniques and equipment described herein.

[0023] Figure 1 An exemplary machine vision system is shown in accordance with some embodiments.

[0024] Figure 2 is a flow chart illustrating an exemplary computerized method for generating a histogram of a three-dimensional point cloud in accordance with some embodiments.

[0025] Figure 3A and Figure 3AC Two exemplary point-to-plane histograms are shown in accordance with some embodiments.

[0026] Figure 3B and Figure 3BC Two exemplary point-to-line histograms are shown in accordance with some embodiments.

[0027] Figure 4 , Figure 4A , Figure 4B , Figure 4C Two exemplary point-to-centroid distance histograms are shown in accordance with some embodiments.

[0028] Figure 5 is a flow chart of an exemplary computerized method for generating a histogram of a 3D point cloud in accordance with some embodiments.

[0029] Figure 6 Two exemplary normal direction histograms of point cloud data for a frustum are shown in accordance with some embodiments.

[0030] Figure 7 Two exemplary normal direction histograms of point cloud data for a cylindrical object are shown in accordance with some embodiments.

[0031] Figure 8 Two exemplary normal direction histograms for a hemispherical object are shown in accordance with some embodiments.

[0032] Fig. 9 Two exemplary normal direction histograms of a cityscape object are shown in accordance with some embodiments.

[0033] Fig.10 According to some embodiments, the use of Figures 6 to 9 Table of exemplary similarity scores for the depicted 2D orientation histogram calculation. DETAILED DESCRIPTION

[0034] The technology described herein provides data simplification techniques that can be used to analyze 3D point cloud images. The inventors have realized that conventional machine vision techniques can be significantly inefficient when processing 3D point cloud data (e.g., to determine whether an object exists in the 3D point cloud data). 3D point clouds typically include hundreds of thousands or millions of (x, y, z) points. Therefore, the inventors have realized that directly interpreting such a large number of 3D points in space can be quite challenging. For example, because a 3D point cloud includes such a large number of 3D points and typically does not include information about the spatial relationship between the 3D points, attempting to interpret a pure 3D point cloud is not feasible for many machine vision applications, which perform such interpretations with limited time, limited hardware resources, etc.

[0035] In particular, it may be desirable to determine whether a 3D point cloud image captures an object. Conventional techniques address such problems by searching for objects in a point cloud using computationally intensive techniques. For example, some methods process the set of 3D points to compute edge features that indicate abrupt changes (e.g., wrinkles or tears in one or more surfaces in the field of view), compute a measure of the prevalence of such edges, and determine the presence (or absence) of an object if the prevalence exceeds a predetermined threshold. Computing such edge features requires a large amount of computation, involving points in the neighborhood of each point in the 3D cloud. Additionally or alternatively, it may be desirable to compare objects detected in different 3D point cloud images. Like object detection, such methods may require computing edge features. For example, edge features may be computed in each of the point clouds, the positions and other properties of the edges may be compared, and finally, the comparison results may be summarized into a similarity measure for the objects.

[0036] As another example, some methods of object detection, classification, and / or registration compute a grid of points or a surface patch to determine object features (e.g., surface curvature). Computing such a grid or surface patch alone can be time consuming, and the technique requires further processing to be performed in order to determine the object features and build a surface model. For example, computing the surface features of the grid or patch and building a parametric model typically requires an iterative process involving optimization, which generally limits the use of these techniques to non-real-time applications.

[0037] The inventors have made technical improvements to machine vision technology to address these and other inefficiencies. The technology described herein can simplify a 3D point cloud into a 1D signal (e.g., a 1D histogram) and / or a 2D image (e.g., a 2D histogram), which can be used to easily interpret the 3D point cloud for various machine vision applications. The technology also provides an interpretation of the 1D signal and the 2D signal, such as comparing different 3D point clouds to determine whether the 3D point clouds are similar. For example, a 1D or 2D histogram can be calculated for two different 3D point cloud images, and the histograms can be compared to determine whether the 3D point cloud images may include the same object. Since a large amount of data in a 3D point cloud can traditionally be simplified to one or two dimensions, the technology described herein can significantly improve performance and allow various types of machine vision tasks to interpret 3D point clouds. Further, compared to conventional techniques, the resulting histogram can be represented using a small amount of data, and therefore only a minimum of memory or disk space is required to store the histogram.

[0038] The described histogram-based techniques can overcome various processing inefficiencies of conventional techniques because they do not require computationally intensive aspects, such as, calculating edge features, point grids and / or surface patches (and also do not require the generation of surface models). In contrast, the techniques described herein provide statistical features for determining objects directly from 3D points. For example, a 1D histogram can be generated based on 3D point positions, and / or a 2D histogram can be generated based on point normal vectors. As an illustrative example, consider a manufacturing application in which a 3D sensor is mounted above a conveyor belt (e.g., cardboard boxes of different sizes, mailing envelopes, plastic bags, etc.) through which an object passes. Each class of objects has different characteristics, including one or more heights above the conveyor belt, surface normal orientation, etc. Such characteristics can be represented by a histogram based on 1D distances and / or a histogram based on 2D directions, so that these objects are detected and classified in real time when the object travels down the conveyor belt. For example, given a distance histogram through an object, a statistical metric (e.g., median, specific percentile, mode, etc.) can be used to quickly estimate the height of the object above the conveyor belt. Additionally or alternatively, by using metrics derived from histograms, the techniques described herein can reduce noise and / or increase the robustness of measurement results, among other things.

[0039] In the following description, many specific details are set forth regarding the systems and methods of the disclosed subject matter and the environments in which such systems and methods can operate, etc., in order to provide a thorough understanding of the disclosed subject matter. In addition, it will be understood that the examples provided below are exemplary, and it is contemplated that there are other systems and methods within the scope of the disclosed subject matter.

[0040] Figure 1 1 shows an exemplary machine vision system 100 according to some embodiments. The exemplary machine vision system 100 includes a camera 102 (or other imaging acquisition device) and a computer 104. Figure 1 Only one camera 102 is shown, but it should be understood that multiple cameras may be used in a machine vision system (e.g., where a point cloud is merged from point clouds from multiple cameras). Computer 104 includes one or more processors and a human-machine interface in the form of a computer display, and optionally one or more input devices (e.g., a keyboard, mouse, trackball, etc.). Camera 102 includes, among other components, a lens 106 and a camera sensor element (not shown). Lens 106 includes a field of view 108, and lens 106 focuses light from field of view 108 onto the sensor element. The sensor element generates a digital image of the camera field of view 108 and provides the image to a processor that forms part of computer 104. As shown in FIG. Figure 1, an object 112 travels along a conveyor 110 into a field of view 108 of a camera 102. When the object 112 is in the field of view 108, the camera 102 may generate one or more digital images of the object for processing, as further discussed herein. In operation, the conveyor may contain a plurality of objects. Such objects may sequentially pass within the field of view 108 of the camera 102, such as during an inspection. In this way, the camera 102 may capture at least one image of each object 112 being viewed.

[0041] In some embodiments, the camera 102 is a three-dimensional (3D) imaging device. As an example, the camera 102 can be a 3D sensor that scans a scene line by line, such as the DS-line of laser profiler 3D displacement sensors available from Cognex Corp., the assignee of the present application. According to some embodiments, the 3D imaging device can generate a set of (x, y, z) points (e.g., where the z-axis adds a third dimension, such as the distance from the 3D imaging device). The 3D imaging device can use various 3D image generation techniques, such as shape-from-shading, stereo imaging, time-of-flight techniques, projector-based techniques, and / or other 3D generation techniques. In some embodiments, the machine vision system 100 includes a two-dimensional imaging device, such as a two-dimensional (2D) CCD or CMOS imaging array. In some embodiments, the two-dimensional imaging device generates a 2D array of brightness values.

[0042] In some embodiments, the machine vision system processes 3D data from camera 102. The 3D data received from camera 102 may include, for example, a point cloud and / or a range image. The point cloud may include a set of 3D points on or near the surface of a solid object. For example, the points may be represented by their coordinates in a rectilinear or other coordinate system. In some embodiments, other information may also be optionally present, such as a grid or grid structure indicating which points are neighbors on the surface of the object. In some embodiments, information about surface features (including curvature, surface normals, edges, and / or color and albedo information), whether derived from sensor measurements or previously calculated, may be included in the input point cloud. In some embodiments, 2D and / or 3D data may be obtained from 2D and / or 3D sensors, from CAD or other solid models, and / or by preprocessing range images, 2D images, and / or other images.

[0043] According to some embodiments, the set of 3D points may be part of a 3D point cloud within a user-specified region of interest and / or include data specifying a region of interest in a 3D point cloud. For example, because a 3D point cloud may include so many points, it may be desirable to specify and / or define one or more regions of interest (e.g., to limit the space to which the techniques described herein apply).

[0044] Examples of computer 104 may include, but are not limited to, a single server computer, a series of server computers, a single personal computer, a series of personal computers, a minicomputer, a mainframe computer, and / or a computing cloud. Various components of computer 104 may execute one or more operating systems, examples of which may include, but are not limited to, for example, Microsoft Windows Server™; Novell Netware™; Redhat Linux™, Unix, and / or custom operating systems. One or more processors of computer 104 may be configured to process operations stored in a memory connected to one or more processors. The memory may include, but is not limited to, a hard drive, a flash drive, a tape drive, an optical drive, a RAID array, a random access memory (RAM), and a read-only memory (ROM).

[0045] The technology described herein relates to generating a histogram of a 3D point cloud. A histogram may be generated based on geometric aspects of a 3D point cloud, such as the position of a 3D point and / or the normal direction of a point (e.g., on the surface of an object captured by a 3D point cloud), etc. Some embodiments relate to a one-dimensional (1D) histogram. A 1D histogram may be generated relative to a reference (such as a reference plane, line, and / or other points spatially oriented relative to a point cloud). For example, a point-to-plane distance histogram is a 1D histogram generated based on the distance from each point (or other representation, such as a voxel) in a 3D point cloud to a selected or estimated plane. For example, a point-to-line distance histogram is a 1D histogram generated based on the distance from each point (or other representation, such as a voxel) in a 3D point cloud to a selected or estimated line. As another example, a point-to-centroid distance histogram is a 1D histogram generated based on the distance from each point (or other representation, such as a voxel) in a 3D point cloud to the centroid of a 3D point cloud. Some embodiments relate to two-dimensional (2D) histograms. For example, a normal direction projection histogram is a 2D histogram of unit normal directions of a 3D point cloud.

[0046] For various 3D point cloud based applications, histograms can be used as useful feature descriptors of scenes. The techniques described herein (including descriptors and interpretations) can be resistant to changes such as resolution, noise, and pose, maintaining geometric invariance. Given the reduced dimensionality and data points, the computational cost of the techniques can be lower than conventional techniques for comparing point clouds. Histograms can be used (e.g., directly) as input to various image processing techniques and computer vision tools (including classification, measurement, object detection, object registration, and / or deep learning, etc.).

[0047] Some embodiments of the techniques involve generating a 1D histogram. Figure 2is a flow chart illustrating an exemplary computerized method 200 for generating a histogram of a three-dimensional (3D) point cloud according to some embodiments. At step 202, a machine vision system (e.g., Figure 1 The machine vision system 100 of the embodiment of the present invention receives data indicating a 3D point cloud including a plurality of 3D points. At step 204, the machine vision system determines a reference (e.g., a reference plane, a line, a centroid, etc.) that is set in a certain spatial relationship with the 3D point cloud (e.g., selected based on the point cloud and / or calculated based on the point cloud, etc.). At step 206, the machine vision system determines a distance to the reference for each 3D point in the plurality of 3D points to generate a set of distances for the plurality of 3D points. At step 208, the machine vision system generates a histogram based on the set of distances.

[0048] Referring to step 202, according to some embodiments, the 3D point cloud may be processed before generating a histogram. For example, the technique may include voxelizing the 3D point cloud. The machine vision system may generate a 3D voxel grid for at least a portion of the 3D point cloud (e.g., a portion of interest for which a histogram is to be calculated), wherein each voxel in the 3D voxel grid has the same size (e.g., the same length, width, and height). The machine vision system may determine for each voxel in the 3D voxel grid whether the position of one or more of the plurality of 3D data points falls within the voxel to generate a set of 3D points associated with the voxel. It should be understood that some voxels may be empty (e.g., where the position of the 3D point does not fall within those voxels).

[0049] According to some embodiments, a voxel may store its associated set of 3D points (e.g., for subsequent processing, such as for calculating an average and / or median, etc.). According to some embodiments, the set of 3D points may be processed before being stored in the voxel to reduce the number of data points stored in the voxel grid. For example, for each voxel in the 3D voxel grid, the machine vision system may determine a single 3D data point for the voxel based on the associated set of 3D data points, and store the single 3D data point in the voxel (e.g., by determining the centroid of the point, the average point value, etc.). A histogram may be generated based on the voxel data. For example, assume that a set of 3D points are stored in the voxel so that each voxel includes zero 3D points (e.g., if no 3D point is located in the voxel) or one 3D point (e.g., if a single 3D point falls in the voxel, or if there are multiple points, representative 3D data points are generated for multiple points). The machine vision system may generate the set of distances by determining, for each voxel in the 3D voxel grid storing the 3D data point, a distance from a single 3D data point to a reference to determine the set of distances.

[0050] Referring to steps 204 to 208, according to some embodiments, when a voxel grid is used to represent the 3D point cloud at step 202, it can be determined that the reference of step 204 is in spatial relationship with the voxel grid. For step 206, the distance from the representative point of each voxel of the grid to the reference can be measured to generate a set of distances for generating a histogram at step 208.

[0051] Referring to step 204, a point cloud may be calculated using various references. According to some embodiments, the reference is a plane, such as a 2D reference plane. The reference plane may be determined according to various techniques. For example, a user may specify a 2D reference plane as input. As another example, a user may specify a region of interest of a point cloud, wherein the contained points may be used to extract a plane as a reference plane. The reference plane may be extracted from the region of interest using various techniques. For example, a reference plane may be extracted from the region of interest by fitting a plane using some and / or all points contained in the region of interest using a least squares method, and / or by fitting a plane with a maximum number of inliers on the contained points using a RANdom SAmple Consensus (RANSAC) technique, and the like.

[0052] According to some embodiments, the reference can be a line estimated based on a 3D point cloud. The reference line can be determined based on the region of interest using various techniques as described herein. For example, the reference line can be determined using a least squares method, using some and / or all points contained in the region of interest to fit a line. As another example, the reference line can be determined using a RANSAC technique to fit a line with a maximum number of inlays on some and / or all points contained in the region of interest. In some embodiments, the reference line can be determined based on one or more 3D shapes extracted from points in the region of interest (e.g., the axis of a cylinder, the intersection of two non-parallel planes, etc.).

[0053] According to some embodiments, the reference can be a point estimated based on a 3D point cloud, such as an estimated centroid or centroid of the 3D point cloud. The machine vision system can process the 3D point cloud (e.g., 3D points and / or voxels) to determine the estimated centroid and use the estimated centroid as a reference point. Although calculating the centroid is an example of a technique that can be used to calculate a reference point, it should be understood that other methods can be used with the techniques described herein. In particular, the reference points used to create the histogram can be determined as needed and should not be limited to the centroid of the inspected object. For example, the reference point can be created by determining a 3D shape and extracting the centroid from the 3D shape, and / or calculating the centroid from a subset of 3D points, and so on.

[0054] Referring to step 206, if the reference is a plane, the machine vision system can determine the distance of each 3D point by determining the distance of each 3D point to the reference plane. The distance can be determined by calculating (e.g., the shortest distance of each point to the reference plane, and / or the distance of each point along the projection, etc.). Such a 1D point-to-plane histogram can represent the distribution of distances from the point to the reference plane. By eliminating the effects of noise points and outliers, the point-to-plane histogram can be used to robustly measure the distance / height from the (e.g., noisy) 3D point to the reference plane. In some embodiments, if the reference is a line, the machine vision system can determine the distance of each 3D point by determining the distance of each 3D point to the estimated line (e.g., the shortest distance to the estimated line). Such a 1D point-to-line histogram can represent the identification of the surface points of the object.

[0055] Referring to step 208, in some embodiments, the histogram includes a set of entries (e.g., a one-dimensional bar graph and / or a point-line graph, etc.). For example, each entry may be associated with a distance and / or a distance range. According to some embodiments, when calculating a histogram, the histogram entry may represent a signed distance or an unsigned distance from each element of the 3D point cloud to a reference. Generating a histogram may include: for each entry of the histogram, adding a set of distances that satisfy a distance range associated with the entry and / or are within a distance range associated with the entry. For example, adding may include determining a count of the distance of each entry of the histogram, for example, so that the distance is considered to belong to an entry of the histogram by discretizing / quantizing its value.

[0056] Referring to steps 206 to 208, in some embodiments, if the reference is a point (e.g., a centroid), the machine vision system can determine the distance of each 3D point by determining the distance of each 3D point to the estimated centroid. Such a 1D point-to-centroid histogram can represent the identity of the surface points of the object. The point-to-centroid histogram can be used as an effective feature for object recognition and / or classification based on point clouds, etc.

[0057] Figure 3A Two exemplary point-to-plane histograms of a cityscape object are shown in accordance with some embodiments. Figure 3A An exemplary histogram 300 of point-to-plane distances of a point cloud 302 of a cylindrical object within a box 304 to a plane 306 disposed at the base of the box 304 is shown. The X-axis of the histogram 300 represents the distance from the reference plane 306, and the Y-axis represents the number of points having a particular distance. Figure 3ACAlso shown is an exemplary histogram 350 of point-to-plane distances of the point cloud 352 of three planar surfaces within the box 354 to a plane 356 (which is also disposed at the base of the box 354). Similar to the histogram 300, the X-axis of the histogram 350 represents the distance from the reference plane 356, and the Y-axis represents the number of points having a particular distance. As described herein, the histogram distances in these examples are calculated by determining the shortest distance of each point to the reference plane.

[0058] In some embodiments, as described herein, the histograms 300 and 350 can be compared (e.g., to determine whether the objects captured by the 3D images are similar). The similarity score between the two histograms 300 and 350 is 0.296643, where the calculated score value can range from 0 to 1.0, where 1.0 indicates that the two compared histograms are the same, and a value of 0 indicates the minimum similarity. Therefore, in this example, the score 0.296643 indicates that the objects within the regions 304, 354 are not similar. The similarity score for this example is calculated using a histogram intersection metric by comparing the sum of the minimum values ​​of the normalized frequencies over all bins between the two histograms.

[0059] Figure 3B Two exemplary point-to-line histograms of a cityscape object are shown in accordance with some embodiments. Figure 3B An exemplary histogram 370 of point-to-line distances from a point cloud 372 of a cylindrical object within a box 374 to a line 376 (the axis of the cylinder) is shown. Figure 3BC Also shown is an exemplary histogram 380 of point-to-line distances from a point cloud 382 to a line 386 for three planar surfaces within a box 384. The X-axis of the histograms 370 and 380 represents the distance from the respective reference lines 376 and 386, and the Y-axis represents the number of points having a particular shortest distance. The similarity score between the two histograms 370 and 380 is 0.191645, where the calculated score value can range from 0 to 1.0, where 1.0 indicates that the two compared histograms are identical, and a value of 0 indicates minimal similarity. Figure 3A Again, the score is calculated using the sum of the minimum values ​​of the normalized frequencies. Thus, in this example, the score 0.191645 indicates that the objects within regions 374, 384 are not similar.

[0060] Figure 4 Two exemplary point-to-centroid distance histograms are shown in accordance with some embodiments. Figure 4A Shows Figure 4 An exemplary histogram 400 of point-to-centroid distances for a point cloud 402 of a frustum surface within a box 404 is shown. Figure 4C It also shows Figure 4BAn exemplary histogram 450 of the distances from the points of the point cloud 452 of the sphere within the box 454 is shown. Since the center of mass is within the object, the center of mass of the frustum and the sphere is not in the Figure 4B The X-axis of the histograms 400 and 450 represents the distance from the associated centroid, and the Y-axis represents the number of points with a specific distance. The similarity score of the two histograms 400 and 450 is 0.835731, indicating that the similarity is better than Figure 3A , Figure 3AC The histogram 300 and the histogram 350 have a higher similarity.

[0061] According to some embodiments, the technique may generate a 2D histogram representing 2D information (e.g., information associated with multiple directions in the x-direction, the y-direction, and the z-direction, etc.). For example, a 2D normal direction histogram may be generated to represent the distribution of unit normal directions of points in 3D space. Figure 5 is a diagram of an exemplary computerized method 500 for generating a histogram of a 3D point cloud according to some embodiments. At step 502, a machine vision system receives a 3D point cloud having a plurality of 3D points. At step 504, the machine vision system generates a set of orientations. The machine vision system determines, for each 3D point in the 3D point cloud, an orientation of the 3D point.

[0062] In some embodiments, the orientation includes a first value of a first component (e.g., an inclination angle) and a second value of a second component (e.g., an azimuth angle). According to some embodiments, for the direction or orientation, the inclination angle may be an angle ranging from 0 to 180 degrees with the direction of the positive Z axis. The azimuth angle may be an angle in the XY plane projected to the positive X axis ranging from 0 to 360 degrees. The azimuth angle may be periodic with a period of 360 degrees.

[0063] At step 506, the machine vision system generates a histogram based on the set of orientations. The histogram can bin the orientations and / or values ​​determined at step 504. In some embodiments, the histogram includes a set of two-dimensionally oriented bins. For example, each bin can be associated with a first value range for a first component (e.g., a tilt index for the bin) and a second value range for a second component (e.g., an orientation index for the bin). The machine vision system generates a histogram by adding, for each bin, the orientations of the first and second values, respectively, within the first and second value ranges, the first and second values ​​being associated with the bins to quantize each 3D point to the bin. As described herein, for example, the addition can include determining a count of points having values ​​within a particular component range to generate each entry of the histogram, for example, so that the point is considered to belong to an entry of the histogram by discretizing / quantizing the value of the point.

[0064] According to some embodiments, with reference to steps 504 to 506, the unit normal at each point may be an ordered pair of tilt angles and azimuth angles. The normal direction histogram may be a 2D image, wherein the two dimensions are tilt angles and azimuth angles. Each row and each column of the 2D histogram image may correspond to intervals of tilt angles and azimuth angles, respectively. The number of rows and columns may be determined by the tilt angle range and azimuth angle range and the interval size, respectively. Each value at a given pixel or interval conveys a count (e.g., frequency) of normal directions whose tilt angles and azimuth angles fall within the interval.

[0065] According to some embodiments, a set of steps may be performed for each 3D point in the region of interest to calculate a 2D histogram. For each 3D point, the machine vision system may calculate its tilt and azimuth, quantize each tilt and azimuth by bin size to determine a tilt bin (e.g., tilt index) and an azimuth bin (e.g., azimuth index), and increase the pixel / bin value indexed at the tilt index and azimuth index by 1.

[0066] According to some embodiments, an optional secondary image (such as a centroid image) may be determined. For example, the centroid image may be a triplet image having a directional histogram. Figure 1 The value in each bin with secondary image index (i, j) may be the 3D centroid of the position of the point of the point cloud, where the normal direction is in bin (i, j) of the direction histogram.

[0067] As described herein, according to some embodiments, a 3D point cloud may be processed (such as voxelizing the 3D point cloud) before generating a histogram. The machine vision system may generate a 3D voxel grid for at least a portion of the 3D point cloud. The machine vision system may determine, for each voxel in the 3D voxel grid, whether the position of one or more of the plurality of 3D data points falls within the voxel to generate a set of 3D points associated with the voxel.

[0068] According to some embodiments, a voxel may store a set of 3D points, a representative point or a set of points, a normal or other vector, etc. associated with it. According to some embodiments, a machine vision system may determine a representative normal or vector of a voxel. For example, for each voxel in a 3D voxel grid, the machine vision system may determine the surface normal and / or orientation of a representative 3D data point (e.g., a median point) of the voxel to generate the set of orientations. As another example, the machine vision system may determine the surface normal vector using an associated set of 3D point positions, adjacent 3D data point positions, and / or information from a 3D sensor. In some embodiments, if a voxel is not associated with any 3D point, the voxel may be set to zero. In some embodiments, the technique may include determining a representative normal or vector for each voxel based on an associated set of 3D data points. For example, a representative vector may be determined by calculating a component average and / or by extracting a feature vector from a cumulative matrix (e.g., formed by accumulating the outer product of each vector with itself, vvT), etc. As a further example, the technique may include determining a vector for each of the associated 3D points and storing a set of vectors.

[0069] Referring to computerized method 500 , according to some embodiments, when a 3D point cloud is represented using a voxel grid at step 502 , an orientation at step 504 is determined using a surface normal and / or orientation of a representative 3D data point for each voxel to generate a set of orientations for generating a histogram at step 506 .

[0070] A 2D histogram may be generated to represent various aspects of a 3D point cloud, including global aspects and / or local aspects. According to some embodiments, the histogram may be a global feature descriptor. For example, the group orientation may be determined based on a fixed coordinate system associated with the 3D point cloud. For example, consider point clouds of multiple scenes, all of which are represented in some fixed coordinate system (e.g., a client coordinate system). Because the orientation histogram generated from each point cloud conveys information about a desired subgroup and / or the entire 3D point cloud as a whole, it may be viewed as a global descriptor of the scene (e.g., in the form of a distribution of orientations of surface point normals).

[0071] According to some embodiments, the histogram can be a local feature descriptor. For example, the set of orientations can be determined based on a local coordinate system associated with the 3D points of the 3D point cloud. For example, the histogram can be recalculated at one or more points of the 3D point cloud. Given the 3D points of the point cloud, the machine vision system can establish a local coordinate system based on the 3D points, wherein the origin of the local coordinate system is at the point. The inclination and azimuth of the orientation of the 3D point cloud can be calculated in the local coordinate system, and the inclination and azimuth are used as local descriptors after being binned as described herein.

[0072] A local 3D coordinate system can be obtained at a given point in a point cloud using various techniques. According to some examples, a machine vision system can select an initial local 3D coordinate frame whose origin is at the point and whose Z axis is aligned with the normal direction of the point (e.g., where the X and Y axes are arbitrary). The machine vision system can establish a final local coordinate frame by first finding its K nearest neighbors to determine its X axis, and then using their normals to calculate a direction histogram in the initial local 3D space. The machine vision system can identify the 2D bin location of the first distinguishable highest frequency peak of the histogram (e.g., along the increasing direction of azimuth), and use the direction of its azimuth as the X axis of the identification coordinate frame. The final local coordinate system for each point can depend on the geometry of the neighborhood of the point (e.g., so that it does not change with changes in point cloud sampling and object pose).

[0073] Figure 6 Two exemplary normal direction histograms of point cloud data for a frustum are shown in accordance with some embodiments. Figure 6 An exemplary histogram 600 of normals for a point cloud 602 of a frustum object in a first pose is shown. Figure 6 Also shown is an exemplary histogram 650 of normals of a point cloud 652 of a frustum object in a second pose. Both histograms 600, 650 show five significant peaks (600A-600E and 650A-650E, respectively), each corresponding to a planar patch of the frustum object. The horizontal direction of images 600 and 650 represents an azimuth interval covering an azimuth range of 0 to 360 degrees, and their vertical direction represents an inclination interval covering an inclination range of 0 to 90 degrees. In image 600, the five peaks are 600A: (91, 42), 600B: (183, 44), 600C: (275, 2), 600D: (271, 47), 600E: (358, 45); in image 650, the five peaks are 650A: (17, 60), 650B: (99, 25), 650C: (240, 38), 650D: (309, 67), 650E: (331, 24), where the first coordinate and the second coordinate of the pixel position respectively indicate the azimuth and the inclination angle in degrees.

[0074] Figure 7 Two exemplary normal direction histograms of point cloud data for a cylindrical object are shown in accordance with some embodiments. Figure 7 An exemplary histogram 700 of normals for a point cloud 702 of a cylindrical object in a first pose is shown. Figure 7Also shown is an exemplary histogram 750 of normals of a point cloud 752 of a cylindrical object in a second (different) pose. The horizontal directions of images 700 and 750 represent azimuth intervals covering an azimuth range of 0 to 360 degrees, and their vertical directions represent tilt intervals covering an inclination range of 0 to 90 degrees.

[0075] Both histograms 700 and 750 show significantly strong ridges. As can be seen in images 702 and 752, the part is made of concentric cylindrical surfaces that share a common axis. For each surface point of the part, its normal is (conceptually) perpendicular to the common axis and points away from the common axis. Therefore, the normals of the surface points are more or less in the same three-dimensional plane perpendicular to the common axis. For example, consider a 3D circle of unit radius perpendicular to the common axis: all unit normals start at the center of the unit circle and end on the circle, occupying some arc segment of the circle.

[0076] In image 702, the part is positioned so that its axis is roughly aligned with the X direction of the 3D coordinate space of the point cloud; when the normals of the surface points are projected into the XY domain, their projections will lie on a line in the XY domain (e.g., corresponding to two opposite directions with an azimuth difference of 180 degrees). This is reflected by the two (almost) vertical ridges 700A and 700B shown in image 700, whose azimuth distance is approximately half of the full X dimension (360 degrees). As another example, when the normals of the surface points are projected into the Z axis of the coordinate space, the projection will take on tilt values ​​from 0 to the maximum tilt angle (<90 degrees), depending on the field of view of the 3D sensor used to capture the point cloud.

[0077] In image 752, the part is rotated from that in image 700, and the normal to each surface point is no longer perpendicular to the X direction because the common axis of the part is not parallel to the X axis. When the normals to the surface points are projected to the XY domain and Z axis of a 3D coordinate space, respectively, which is conceptually similar to projecting the points of the 3D circle they represent (for the covered arc segments) to the XY domain and Z axis, this shows connected ridges 750A and 750B in image 750. Ridge 750A appears disconnected because it is a flipped U-shape, and since the azimuth direction is periodic with a period of 360 degrees, it is separated at an azimuth angle of 360 degrees.

[0078] Figure 8 Two exemplary normal direction histograms for a hemispherical object are shown in accordance with some embodiments. Figure 8 An exemplary histogram 800 of normals for a point cloud 802 of a hemisphere in a first pose is shown. Figure 8Also shown is an exemplary histogram 850 of normals for a point cloud 852 of a hemisphere in a second pose. Both histograms 800 and 850 show a uniform distribution of normal directions. The horizontal directions of histograms 800 and 850 represent azimuth intervals covering an azimuth range of 0 to 360 degrees, and their vertical directions represent tilt intervals covering a tilt angle range of 0 to 90 degrees.

[0079] In both images 800 and 850, there are some "blank" spaces 800A and 850A that correspond to the range of tilt angles to which no normal of a sphere surface point can be projected. The "blank" portions appear near the high end of the tilt direction. As an example, consider a setup where a sphere (e.g., a ball) is imaged by a 3D sensor that looks down at the ball from the top. A 3D sensor can typically only capture a certain portion of the top of the sphere's surface close to the sensor, and how much it can observe typically depends on the sensor's field of view (FOV) and its distance / pose to the sphere. Since in this example, the 2D histogram image is fixed in its tilt dimension (including 90 degrees), the sensor cannot capture portions of the surface that are outside the sensor's FOV but still on the upper hemisphere. Therefore, in this example, there are no points on the sphere's point cloud that have a tilt angle greater than the sensor's maximum viewing angle (which is less than 90 degrees due to the perspective sensing model).

[0080] Fig. 9 Two exemplary normal direction histograms of a cityscape object are shown in accordance with some embodiments. Fig. 9 An exemplary histogram 900 of normals for a point cloud 902 of a cityscape object in a first pose is shown. Fig. 9 Also shown is an exemplary histogram 950 of normals of a point cloud 952 of a cityscape object in a second (different) pose. The horizontal directions of the histograms 900, 950 represent azimuth bins covering an azimuth range from 0 to 360 degrees, and their vertical directions represent tilt bins covering an tilt range from 0 to 90 degrees.

[0081] Both histograms 900 and 950 show similar patterns in the normal direction, but there is an offset and rotation between them. In particular, the U-shaped pattern 900A in the histogram 900 appears as the U-shaped pattern 950A at the bottom left in the histogram 950. Fig. 9 In , the same part is used to capture the point cloud, but with different poses relative to the 3D sensor. The part consists of multiple sub-parts: top surface 902A (featured by several pairs of planar surfaces forming the top of the triangle), concentric cylindrical surface 902B (with Figure 7 ), two rows of top 902C and base surface 902D. The histograms shown in images 900 and 950 reflect the combination of normal information of the surfaces of these sub-parts. Figure 7As shown, the histogram of the concentric cylindrical surface is a "U" shaped ridge, and its inclination angle and azimuth are more dispersed than other sub-parts. For the two rows of box top and base surfaces, their normal directions are similar, and they contribute a single peak in the 2D histogram image. Since the top sub-part is composed of several pairs of planar patches, each patch has a small area; therefore, its histogram is characterized by a limited number of peaks. By combining all the histograms of these sub-parts, the total histograms of two different postures are shown in images 900 and 950, where the U-shape 900A, 950A corresponds to the concentric cylindrical surface, and the highest peak 900B, 950B (with the largest frequency, indicated by the darkest pixel) corresponds to the normal of the two rows of box top 902C and base surface 902D.

[0082] According to some embodiments, 1D histograms and 2D histograms can be analyzed to interpret 3D point clouds. The technique may include comparing two (or more) histograms to calculate a score indicating the similarity between / among the histograms. According to some embodiments, a 1D histogram can be used to measure the similarity between two groups of points. As described herein, each distance-based 1D histogram can provide an effective fingerprint for characterizing a 3D point cloud data set. Different scoring methods (e.g., histogram intersection, Bhattacharyya metric, and normalized cross-correlation, etc.) can be used to calculate the similarity between two 1D histograms to evaluate the extent to which their representative points look similar.

[0083] In some embodiments, the 1D histograms can be compared using one or more scoring metrics. For example, to compare two 1D histograms, the machine vision system can use one or more of four scoring metrics, which are based on intersection, dot product, Bhattacharyya metric, and normalized cross correlation, respectively.

[0084] In some embodiments, the machine vision system determines, for each bin, the smaller of the two relative frequency values ​​for that bin (each histogram). Figure 1 ) to determine an intersection-based score. The machine vision system can calculate the score as the sum of the smaller of these relative frequencies in all bins.

[0085] In some embodiments, both the dot product-based score and the Bhattacharyya metric score involve calculating, for each bin, the product of two relative frequency values ​​for that bin. The machine vision system can calculate the dot product score by calculating the sum of these bin products over all bins. The machine vision system can calculate the Bhattacharyya score by calculating the sum of the square roots of these bin products over all bins.

[0086] In some embodiments, the machine vision system can calculate a Normalized Cross Correlation (NCC) score by calculating (NCC + 1) / 2. The machine vision system can calculate the normalized cross correlation score by dividing the above dot product score by two factors (each histogram Figure 1 For each histogram, these factors can be the root mean square values ​​of the relative frequency distribution.

[0087] In some embodiments, each 1D histogram can first be normalized to produce a relative frequency distribution that sums to 1. A similarity score can then be calculated based on the two relative frequency distributions. Regardless of which metric the machine vision system uses, the resulting score can range from 0 to 1, where the larger the score, the more similar the two histograms are.

[0088] In some embodiments, in order to compare two 2D histograms, a set of high-frequency interval positions can be identified for each histogram. These intervals can use frequency peaks to represent the main 3D directions of the histogram. At each frequency peak interval, its X component and Y component respectively indicate the azimuth interval position and tilt interval position where the peak occurs; its Z component represents the frequency of the peak. In some embodiments, noise removal techniques can be first applied to the histogram to filter out intervals whose frequencies are less than a predetermined significance threshold.

[0089] In some embodiments, a machine vision system can identify frequency peaks by locating 2D spots (e.g., connected branches) in a filtered histogram that is characterized by compact clusters of adjacent non-zero frequency bins. The location (X, Y) of a peak can be the centroid of its constituent bins (e.g., the average of the azimuth bin location and the tilt bin location weighted by their frequencies); the frequency value (Z) of a peak is the sum of the frequency values ​​of its constituent bins. In some embodiments, if there are no significant high-frequency bins in the histogram, the average of all directions can be designated as the dominant direction.

[0090] In some embodiments, the machine vision system can find a rotation that best aligns the principal directions so as to minimize the difference in 3D directions conveyed by the two sets of peak positions. The machine vision system can obtain a rigid transformation corresponding to the best rotation.

[0091] In some embodiments, a measure of goodness of rotation (referred to as a rotation score for purposes of illustration) is calculated based on the distance between frequency peaks considered as corresponding peaks. In some embodiments, the machine vision system first calculates the average of the Euclidean distances between the vertices of the unit vectors in the directions of the corresponding frequency peaks. The rotation score can be calculated as 1 minus the average distance.

[0092] Another example of a score that can be calculated is a frequency matching score. For each interval of the first histogram with non-zero frequency values, the machine vision system can rotate the representative direction (e.g., taken as the center of the interval) through the above transformation to obtain the mapping direction. The mapped direction falls in a set of directions corresponding to one of the intervals of the second histogram. If multiple intervals of the first histogram are mapped to an interval of the second histogram, their frequencies are added to obtain the modified frequency. Therefore, the first histogram is mapped to the space of the second histogram. For each interval, the overlap measure of the two frequencies associated with the interval (i.e., the frequency of the second histogram and the mapped frequency of the first histogram) is calculated. The overlap measure is the ratio of the smaller frequency to the larger frequency of the two frequencies. Next, the machine vision system calculates the average of these interval overlap measures for all intervals with non-zero frequency content. Finally, the machine vision system will calculate the value of the frequency matching score by applying the piecewise quadratic sigmoid function to the average overlap measure. For example, it may be beneficial to apply the sigmoid function to push the intermediate value to 0 or 1.

[0093] In some embodiments, the overall similarity score between two 2D histograms can be calculated as the product of the rotation score and the frequency matching score. Like the score of the 1D histogram, the score can be in the range of 0 to 1, where the larger the score, the more similar the two histograms are.

[0094] According to some embodiments, a 2D normal direction histogram can be used to measure the similarity between two object surfaces. In a 2D histogram of directions, the column distance can linearly reflect the difference in azimuth (depending on the period of 360 degrees), while the row distance can linearly reflect the difference in tilt angle. According to some embodiments, by matching a portion of the histogram, the histograms calculated at different parts of the 3D point cloud and / or different viewpoints of the scene can be associated. For example, one or more peaks of one histogram can be matched with one or more peaks of another histogram to compare different histograms, thereby determining the correspondence between at least a portion of the first group of peaks and at least a portion of the second group of peaks. For example, consider two point clouds collected by imaging the same object in two different postures. The machine vision system can identify the peak positions in the two histograms for the two point cloud calculations and establish a correspondence between these frequency peaks. Such a correspondence can allow, for example, the machine vision system to estimate the rotation of an object from one view to the next. In a similar manner, the transition between views can also be estimated based on the histogram of the centroid image.

[0095] According to some embodiments, a 1D histogram may be generated based on a 2D histogram. For example, by row / column summing of the pixel values ​​of the 2D image, two 1D directional histograms (e.g., related to tilt angle and azimuth) may be derived from the 2D histogram. Each 1D directional histogram may provide effective features for characterizing the surface of the object. Whenever the surface is fixed (as may be done using 3D-based registration), such a 1D histogram may be a useful marker. Like distance-based 1D histograms, Figure 1 Similarly, different scoring methods can be used to calculate the similarity between two 1D orientation histograms representing two surfaces.

[0096] As described herein, techniques can be used to create global descriptors, such as a normal direction histogram that provides a global feature descriptor. Global histograms can be used by existing image processing techniques for various applications, such as for object recognition, classification, and / or registration, among others. According to some embodiments, the highest frequency peaks can be extracted from the histogram. For example, such peaks can provide useful statistical information about the surface properties of an object. Each peak can correspond to a dominant surface normal orientation in a coordinate space, and its frequency can indicate the number of points whose surface normals are at that orientation. Thus, each peak can be characterized by an orientation and a frequency. Given a frequency threshold, a machine vision system can identify all peaks whose frequencies exceed that threshold. The resulting peaks can represent an object by the relationship between the number of peaks and the orientations they represent.

[0097] As described herein, techniques can be used to create local descriptors, such as local orientation histograms. According to some embodiments, a local orientation histogram can represent a point cloud using key points. For example, a key point of a point cloud can be characterized by its local orientation histogram, which has a peak frequency on some non-zero rows (e.g., the peak is not on row 0). If any qualified peak exists on any non-zero row in its local orientation histogram, the point can be considered a key point. If the corresponding frequency of the peak exceeds a threshold (e.g., a predetermined threshold), the peak can be qualified. Each key point can be associated with a qualified peak list. Because the number of key points is usually much less than the number of original points, a point cloud of an object with a changing curvature surface can be simplified to a set of its key points. Key points can appear around the boundaries between surface patches with different orientations. Non-key points can be located on surfaces with a uniform direction (e.g., indicated by a large frequency sum on row 0 in the histogram), a slowly changing normal direction, etc. Compared with non-key points, key points may play a more important role in machine vision applications based on point clouds.

[0098] According to some embodiments, a local directional histogram can be used to search for an object using the correspondence between two sets of key points. For example, given an object, the key points of the object can be extracted from a training time point cloud collected under a typical collection posture to form a reference model. At runtime, key points can be extracted for each point cloud collection to form a runtime model based on a set of key points. By comparing the corresponding peaks of the key point pairs, a mathematical score between them can be calculated. For each correspondence between the key points of the reference model and the runtime model, the matching scores of the corresponding pairs of key points can be summed to obtain a total score. Different methods such as RANSAC can be used to correspond to the two sets of key points. The correspondence with the highest score can be used to determine the result.

[0099] According to some embodiments, the local direction histogram can be used to search for an object by comparing the model of the object. For example, a model can be established for a point, which may include the point, its identification coordinate frame, a region of interest (ROI) around the point, and / or a local direction histogram calculated using the point inside the ROI in the local coordinate frame. A comparison can be made directly between the histograms of the two involved models (e.g., for a given point, one at training time and the other at runtime). To improve efficiency, models can be established and searched at key points in training time and runtime.

[0100] Fig.10 is a diagram showing the use of a combination according to some embodiments Figures 6 to 9 Table 1000 of exemplary similarity scores for the 2D orientation histogram calculations described. Figures 6 to 9 The images 652, 752, 852 and 952 are used to represent the fingerprints used to determine the values ​​in the columns and rows, i.e., the combination of Figures 6 to 9Histograms 650, 750, 850, and 950 discussed separately. As shown along the diagonal, when each histogram is compared with itself, a full score of 1.0 is obtained. None of the scores is higher than 0.28, which indicates that no histogram has similar objects. For this example, in order to calculate the similarity of two 2D histograms, for each histogram, the interval with the highest frequency (e.g., a frequency peak exceeding a user-specified threshold) is identified, and each interval represents the main 3D orientation of the surface it represents. As described herein, the highest peak can be presented as the centroid of the interval identified as belonging to the same spot in the histogram image. If the two histograms contain enough frequency peaks, fitting is used to associate their main directions, thereby finding the best rotation between them as described herein, (e.g., because the histograms may have been generated using images capturing objects in different postures). Then, as described herein, the frequency similarity between the two sets of peak intervals aligned by rotation and considering the error from the rotation estimation is used to calculate the similarity score. If no significant high frequency bins are identified in any of the 2D histograms, the 1D orientation histograms derived from their respective tilt and azimuth angles are used by correlation to compute a similarity score.

[0101] The embodiments discussed herein may be used in a variety of different applications, some of which may include, but are not limited to, part picking in vision-guided robotics, three-dimensional inspection, automotive assembly, molded plastic and cast metal volume inspection, and assembly inspection. Such applications may include searching for and identifying the location and orientation of patterns of interest within an image (e.g., to guide a robotic gripper, or to inspect an object).

[0102] The technology operated according to the principles described herein can be implemented in any suitable manner. The processing and decision blocks of the above flowcharts represent the steps and actions that can be included in the algorithms for performing these different processes. The algorithms derived from these processes can be implemented as software that is integrated with one or more single-purpose or multi-purpose processors and guides their operation, can be implemented as functional equivalent circuits such as digital signal processing (DSP) circuits or application-specific integrated circuits (ASICs), or can be implemented in any other suitable manner. It should be understood that the flowcharts included herein do not describe the syntax or operation of any specific circuit or any specific programming language or programming language type. On the contrary, the flowcharts illustrate functional information that can be used by those skilled in the art to manufacture circuits or implement computer software algorithms to perform the processing of specific devices of the types of techniques described herein. It should also be understood that, unless otherwise specified herein, the specific order of steps and / or actions described in each flowchart is merely an illustration of an algorithm that can be implemented and can be changed in the implementation and embodiments of the principles described herein.

[0103] Accordingly, in some embodiments, the techniques described herein may be embodied in computer executable instructions implemented as software, including application software, system software, firmware, middleware, embedded code, or any other suitable type of computer code. Such computer executable instructions may be written using any of a variety of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled into executable machine language code or intermediate code executed on a framework or virtual machine.

[0104] When the techniques described herein are implemented as computer executable instructions, these computer executable instructions can be implemented in any suitable manner, including as multiple functional facilities, each of which provides one or more operations to complete the execution of algorithms that operate according to these techniques. No matter how it is instantiated, a "functional facility" is a structural component of a computer system that, when integrated with one or more computers and executed by one or more computers, enables one or more computers to perform a specific operational role. A functional facility can be a part of a software element or an entire software element. For example, a functional facility can be implemented as a function of a process, or as a discrete process, or as any other suitable processing unit. If the techniques described herein are implemented as multiple functional facilities, each functional facility can be implemented in its own way; it is not necessary for all functional facilities to be implemented in the same way. In addition, these functional facilities can be appropriately executed in parallel and / or serially, and information can be transmitted between each other using shared memory on one or more computers on which they are executed, using a message passing protocol, or in any other suitable way.

[0105] Typically, functional facilities include routines, programs, objects, components, data structures, etc. that perform specific tasks or realize specific abstract data types. Typically, the functions of functional facilities can be combined or distributed in the system in which they are operated as needed. In some embodiments, one or more functional facilities that perform the technology of this article can form a complete software package together. In alternative embodiments, these functional facilities can be suitable for interacting with other unrelated functional facilities and / or processes to realize software program applications.

[0106] Some exemplary functional facilities for performing one or more tasks have been described herein. However, it should be understood that the functional facilities and task divisions described are merely illustrations of functional facility types that can implement the exemplary techniques described herein, and embodiments are not limited to being implemented in any specific number, division, or type of functional facilities. In some embodiments, all functions can be implemented in a single functional facility. It should also be understood that in some embodiments, some functional facilities described herein can be implemented together with or separately from other functional facilities (i.e., as a single unit or a separate unit), or some of these functional facilities may not be implemented.

[0107] In some embodiments, computer executable instructions that implement the techniques described herein (when implemented as one or more functional facilities or implemented in any other manner) may be encoded on one or more computer-readable media to provide functionality to the media. Computer-readable media include, for example, magnetic media such as hard drives, optical media such as compact disks (CDs) or digital versatile disks (DVDs), permanent or non-permanent solid-state memory (e.g., flash memory, magnetic RAM, etc.), or any other suitable storage medium. Such computer-readable media may be implemented in any suitable manner. As used herein, "computer-readable media" (also referred to as "computer-readable storage media") refers to tangible storage media. Tangible storage media are non-temporary and have at least one physical structural component. In "computer-readable media" as used herein, at least one physical structural component has at least one physical property that can be changed in some way during the process of creating a medium with embedded information, during the process of recording information thereon, or during any other process of encoding the medium with information. For example, the magnetization state of a portion of the physical structure of a computer-readable medium may change during the recording process.

[0108] Further, some of the above-described techniques include acts to store information (e.g., data and / or instructions) in a particular manner for use by these techniques. In some embodiments of these techniques (such as embodiments in which the techniques are implemented as computer-executable instructions), the information may be encoded on a computer-readable storage medium. Where particular structures are described herein as advantageous formats for storing such information, these structures may be used to impart physical organization to the information when encoded on a storage medium. These advantageous structures may then provide functionality to the storage medium by affecting the operation of one or more processors that interact with the information; for example, by improving the efficiency of computer operations performed by the one or more processors.

[0109] In some but not all embodiments where the techniques may be implemented as computer-executable instructions, these instructions may be executed on one or more suitable computing devices operating in any suitable computer system, or one or more computing devices (or one or more processors of one or more computing devices) may be programmed to execute computer-executable instructions. When the instructions are stored in a manner accessible to the computing device or processor, such as a data storage device (e.g., an on-chip cache or instruction register, a computer-readable storage medium accessible via a bus, a computer-readable storage medium accessible via one or more networks and accessible by the device / processor, etc.), the computing device or processor may be programmed to execute these instructions. The functional facilities including these computer-executable instructions may be integrated with a single multi-purpose programmable digital computing device, a coordinated system of two or more multi-purpose computing devices that share processing power and jointly execute the techniques described herein, a single computing device or a coordinated system of computing devices (co-located or geographically distributed) dedicated to executing the techniques described herein, one or more field programmable gate arrays (FPGAs) for executing the techniques described herein, or any other suitable system, and direct their operation.

[0110] The computing device may include at least one processor, a network adapter, and a computer-readable storage medium. For example, the computing device may be a desktop or laptop personal computer, a personal digital assistant (PDA), a smart mobile phone, a server, or any other suitable computing device. The network adapter may be any suitable hardware and / or software that enables the computing device to communicate wired and / or wirelessly with any other suitable computing device through any suitable computing network. The computing network may include wireless access points, switches, routers, gateways, and / or other network devices, as well as any suitable wired and / or wireless communication media for exchanging data between two or more computers, including the Internet. The computer-readable medium may be suitable for storing data to be processed by the processor and / or instructions to be executed by the processor. The processor is capable of processing data and executing instructions. Data and instructions may be stored on a computer-readable storage medium.

[0111] The computing device may additionally have one or more components and peripherals, including input devices and output devices. These devices may be used, in particular, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of the output, and a speaker or other sound generating device for auditory presentation of the output. Examples of input devices that may be used for a user interface include a keyboard and a pointing device (such as a mouse, a touch pad, and a digitized graphics tablet). As another example, a computing device may receive input information by voice recognition or in other auditory formats.

[0112] Embodiments of implementing techniques in circuits and / or computer-executable instructions have been described. It should be appreciated that some embodiments may be in the form of methods, of which at least one example has been provided. The actions performed as part of the method may be ordered in any suitable manner. Accordingly, embodiments may be constructed in which the actions are performed in an order different from that illustrated, which may include performing some actions simultaneously, even though shown as sequential actions in the illustrative embodiments.

[0113] The various aspects of the above-described embodiments may be used alone, in combination, or in various arrangements not specifically discussed in the previously described embodiments, and therefore their application is not limited to the details and arrangements of the components set forth in the previous description or illustrated in the accompanying drawings. For example, an aspect described in one embodiment may be combined in any manner with an aspect described in other embodiments.

[0114] The use of ordinal numbers such as "first", "second", "third", etc. to modify claim elements in the claims does not in itself imply any priority, precedence or sequence of one claim element relative to another or a temporal order of the acts of performing the method, but merely serves as a label to distinguish one claim element with a certain name from another element with the same name (but using an ordinal number) to distinguish the claim elements.

[0115] In addition, the words and terms used herein are for descriptive purposes and should not be considered limiting. The use of "includes," "comprising," "having," "containing," "involving," and variations thereof herein is meant to encompass the items listed thereafter and their equivalents as well as additional items.

[0116] The term "exemplary" is used herein to mean serving as an example, instance, or illustration. Any embodiment, implementation, process, feature, etc. described herein as exemplary should therefore be understood as an illustrative example and should not be understood as a preferred or advantageous example unless otherwise specified.

[0117] Thus, having described several aspects of at least one embodiment, it will be appreciated that various changes, modifications and improvements will readily occur to those skilled in the art. Such changes, modifications and improvements are intended to be a part of this disclosure and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and accompanying drawings are merely exemplary.

[0118] Various aspects are described in this disclosure, including but not limited to the following: 1. A computerized method for generating a histogram of a three-dimensional (3D) point cloud, the method comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; determining a reference in spatial relationship to the 3D point cloud; for each 3D point in the plurality of 3D points, determining a distance to the reference to generate a distance set for the plurality of 3D points; and Generating a histogram including a group of entries based on the distance group includes, for each entry in the group of entries, interpolating a distance from the distance group that is within a distance range associated with the entry.

[0119] 2. The method according to item 1, further comprising: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

[0120] 3. The method according to 2, wherein determining the distance group comprises: For each voxel in the 3D voxel grid, a distance from the single 3D data point to the reference is determined to generate the distance set.

[0121] 4. The method according to any one of 1 to 3, wherein: The reference is a two-dimensional (2D) reference plane; and Determining the distance of each 3D point to generate the distance group includes determining the shortest distance of each 3D point to the reference plane.

[0122] 5. The method according to any one of 1 to 4, wherein: The reference is a reference line; and Determining the distance of each 3D point to generate the distance group includes: determining the shortest distance from each 3D point to the reference line.

[0123] 6. The method according to any one of 1 to 5, further comprising: An estimated centroid of the 3D point cloud is determined, wherein the reference is the estimated centroid.

[0124] 7. The method of 6, wherein determining the distance of each 3D point to generate the distance group comprises: determining the distance of each 3D point to the estimated centroid.

[0125] 8. The method according to any one of 1 to 7 further includes: comparing the histogram with a second histogram generated for a second 3D point cloud to determine a measure of similarity between the 3D point cloud and the second 3D point cloud.

[0126] 9. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to Generates a histogram of a three-dimensional (3D) point cloud, including: receiving data indicating a 3D point cloud comprising a plurality of 3D points; determining a reference in spatial relationship to the 3D point cloud; for each 3D point in the plurality of 3D points, determining a distance to the reference to generate a distance group for the plurality of 3D points; and Generating a histogram including a group of entries based on the distance group includes, for each entry in the group of entries, interpolating a distance from the distance group that is within a distance range associated with the entry.

[0127] 10. The non-transitory computer readable medium of claim 9, wherein the instructions are further operable to cause the one or more processors to perform: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

[0128] 11. The non-transitory computer readable medium of 10, wherein determining the distance group comprises: For each voxel in the 3D voxel grid, a distance from the single 3D data point to the reference is determined to generate the distance set.

[0129] 12. The non-transitory computer-readable medium of any one of 9 to 11, wherein: The reference is a two-dimensional (2D) reference plane; and Determining the distance of each 3D point to generate the distance group includes determining the shortest distance of each 3D point to the reference plane.

[0130] 13. The non-transitory computer-readable medium of any one of 9 to 12, wherein: The reference is a reference line; and Determining the distance of each 3D point to generate the distance group includes: determining the shortest distance from each 3D point to the reference line.

[0131] 14. The non-transitory computer readable medium of any one of 9 to 13, wherein the instructions are further operable to cause the one or more processors to perform: Determining an estimated centroid of the 3D point cloud comprises determining a distance of each 3D point to the estimated centroid, wherein the reference is the estimated centroid.

[0132] 15. A system comprising a memory storing instructions and at least one processor, the at least one processor being configured to execute the instructions to generate a histogram of a three-dimensional (3D) point cloud, comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; determining a reference in spatial relationship to the 3D point cloud; for each 3D point in the plurality of 3D points, determining a distance to the reference to generate a distance set for the plurality of 3D points; and Generating a histogram including a group of entries based on the distance group includes, for each entry in the group of entries, interpolating a distance from the distance group that is within a distance range associated with the entry.

[0133] 16. The system of claim 15, wherein the instructions are further operable to cause the at least one processor to perform: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

[0134] 17. The system of 16, wherein determining the distance group comprises: For each voxel in the 3D voxel grid, a distance from the single 3D data point to the reference is determined to generate the distance set.

[0135] 18. A system according to any one of 15 to 17, wherein: The reference is a two-dimensional (2D) reference plane; and Determining the distance of each 3D point to generate the distance group includes determining the shortest distance of each 3D point to the reference plane.

[0136] 19. A system according to any one of 15 to 18, wherein: The reference is a reference line; and Determining the distance of each 3D point to generate the distance group includes: determining the shortest distance from each 3D point to the reference line.

[0137] 20. The system according to any one of 15 to 20, wherein the instructions are further operable to cause the at least one processor to perform: Determining an estimated centroid of the 3D point cloud comprises determining a distance of each 3D point to the estimated centroid, wherein the reference is the estimated centroid.

[0138] 21. A computerized method for generating a histogram of a three-dimensional (3D) point cloud, the method comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; Generating an orientation group includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point, wherein the orientation includes at least a first value of a first component and a second value of a second component; Based on the orientation groups, a histogram comprising interval groups is generated, wherein: Each interval in the set of intervals is associated with a first range of values ​​for the first component and a second range of values ​​for the second component; and Generating the histogram includes, for each bin in the group of bins, adding an orientation from the group of orientations having a first value and a second value within the first value range and the second value range, respectively, associated with the bin.

[0139] 22. The method of 21, wherein the group intervals are arranged in two dimensions, wherein a first dimension is associated with the first component and a second dimension is associated with the second component.

[0140] 23. A method according to any one of 21 to 22, wherein the first component comprises an inclination angle and the second component comprises an azimuth angle.

[0141] 24. The method according to any one of 21 to 23, further comprising: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

[0142] 25. The method according to 24, wherein generating the orientation group comprises: For each voxel in the 3D voxel grid, an orientation of the single 3D data point is determined to generate the orientation set.

[0143] 26. The method according to any one of 21 to 25, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

[0144] 27. A method according to any one of 21 to 26, wherein generating the orientation group comprises: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

[0145] 28. The method of any one of 21 to 27 further comprises comparing the histogram with a second histogram associated with a second 3D point cloud to determine data indicating a similarity measure between the 3D point cloud and the second 3D point cloud.

[0146] 29. The method of 28, wherein comparing the histogram with the second histogram comprises: determining a first set of peaks of the histogram and a second set of peaks of the second histogram; A correspondence between at least a portion of the first group of peaks and at least a portion of the second group of peaks is determined.

[0147] 30. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to generate a histogram of a three-dimensional (3D) point cloud comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; Generating an orientation group includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point, wherein the orientation includes at least a first value of a first component and a second value of a second component; Based on the orientation groups, a histogram comprising interval groups is generated, wherein: Each interval in the set of intervals is associated with a first range of values ​​for the first component and a second range of values ​​for the second component; and Generating the histogram includes, for each bin in the group of bins, adding an orientation from the group of orientations having a first value and a second value within the first value range and the second value range, respectively, associated with the bin.

[0148] 31. The non-transitory computer readable medium of claim 30, wherein the instructions are further operable to cause the one or more processors to perform: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

[0149] 32. The non-transitory computer readable medium of 31, wherein generating the orientation group comprises: For each voxel in the 3D voxel grid, an orientation of the single 3D data point is determined to generate the orientation set.

[0150] 33. A non-transitory computer-readable medium according to any one of 30 to 32, wherein generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

[0151] 34. A non-transitory computer-readable medium according to any one of 30 to 33, wherein generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

[0152] 35. The non-transitory computer readable medium of any one of 30 to 34, wherein the instructions are further operable to cause the one or more processors to perform: Comparing the histogram with a second histogram associated with a second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud comprises: determining a first set of peaks of the histogram and a second set of peaks of the second histogram; and A correspondence between at least a portion of the first group of peaks and at least a portion of the second group of peaks is determined.

[0153] 36. A system comprising a memory storing instructions and at least one processor, the at least one processor configured to execute the instructions to generate a histogram of a three-dimensional (3D) point cloud, comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; Generating an orientation group includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point, wherein the orientation includes at least a first value of a first component and a second value of a second component; Based on the orientation groups, a histogram comprising interval groups is generated, wherein: Each interval in the set of intervals is associated with a first range of values ​​for the first component and a second range of values ​​for the second component; and Generating the histogram includes, for each bin in the group of bins, adding an orientation from the group of orientations having a first value and a second value within the first value range and the second value range, respectively, associated with the bin.

[0154] 37. The system of claim 36, wherein the instructions are further operable to cause the at least one processor to perform: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

[0155] 38. A system according to any one of 36 to 37, wherein generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

[0156] 39. A system according to any one of 36 to 38, wherein generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

[0157] 40. The system of any one of 36 to 39, wherein the instructions are further operable to cause the at least one processor to perform: Comparing the histogram with a second histogram associated with a second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud comprises: determining a first set of peaks of the histogram and a second set of peaks of the second histogram; and A correspondence between at least a portion of the first group of peaks and at least a portion of the second group of peaks is determined.

Claims

1. A computerized method for generating a histogram of a three-dimensional 3D point cloud to represent the 3D point cloud for comparison with other point clouds, the method comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; Generating an orientation group includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point, wherein the orientation includes at least a first value of a first component and a second value of a second component; Based on the orientation groups of the 3D points, a histogram including a group of intervals representing the orientation groups of the 3D points is generated, wherein: Each interval in the set of intervals is associated with a first range of values ​​for the first component and a second range of values ​​for the second component; and Generating the histogram comprises, for each bin in the group of bins, adding orientations from the group of orientations having first and second values ​​within the first and second value ranges, respectively, associated with the bin to determine a representation of a plurality of orientations of the 3D point common to each bin; and The histogram is stored for comparison with a second histogram associated with a second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud.

2. The method according to claim 1, wherein: The groups of intervals are arranged in two dimensions, wherein a first dimension is associated with the first component and a second dimension is associated with the second component.

3. The method according to claim 1, wherein: The first component comprises a tilt angle, and the second component comprises an azimuth angle.

4. The method according to claim 1, further comprising: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

5. The method according to claim 4, wherein: Generating the orientation group comprises: For each voxel in the 3D voxel grid, an orientation of the single 3D data point is determined to generate the orientation set.

6. The method according to claim 1, wherein: Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

7. The method according to claim 1, wherein: Generating the orientation group includes: for each 3D point in the 3D point cloud, determining an orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

8. The method of claim 1, further comprising comparing the histogram to a second histogram associated with a second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud.

9. The method according to claim 8, wherein: Comparing the histogram with the second histogram comprises: determining a first set of peaks of the histogram and a second set of peaks of the second histogram; and A correspondence between at least a portion of the first group of peaks and at least a portion of the second group of peaks is determined.

10. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to generate a histogram of a three-dimensional 3D point cloud to represent the 3D point cloud for comparison with other point clouds, comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; Generating an orientation group includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point, wherein the orientation includes at least a first value of a first component and a second value of a second component; Based on the orientation groups of the 3D points, a histogram including a group of intervals representing the orientation groups of the 3D points is generated, wherein: Each interval in the set of intervals is associated with a first range of values ​​for the first component and a second range of values ​​for a second component; and Generating the histogram comprises, for each bin in the group of bins, adding orientations from the group of orientations having first and second values ​​within the first and second value ranges, respectively, associated with the bin to determine a representation of a plurality of orientations of the 3D point common to each bin; and The histogram is stored for comparison with a second histogram associated with a second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud.

11. The non-transitory computer readable medium of claim 10, wherein: The instructions are further operable to cause the one or more processors to perform: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

12. The non-transitory computer readable medium of claim 11, wherein: Generating the orientation group comprises: For each voxel in the 3D voxel grid, an orientation of the single 3D data point is determined to generate the orientation set.

13. The non-transitory computer readable medium of claim 10, wherein: Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

14. The non-transitory computer readable medium of claim 10, wherein: Generating the orientation group includes: determining, for each 3D point in the 3D point cloud, an orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

15. The non-transitory computer readable medium of claim 10, wherein: The instructions are further operable to cause the one or more processors to perform: Comparing the histogram with the second histogram associated with the second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud, comprises: determining a first set of peaks of the histogram and a second set of peaks of the second histogram; and A correspondence between at least a portion of the first group of peaks and at least a portion of the second group of peaks is determined.

16. A system comprising a memory storing instructions and at least one processor, the at least one processor being configured to execute the instructions to generate a histogram of a three-dimensional 3D point cloud to represent the 3D point cloud for comparison with other point clouds, comprising: receiving data indicating a 3D point cloud comprising a plurality of 3D points; Generating an orientation group includes determining, for each 3D point in the 3D point cloud, an orientation of the 3D point, wherein the orientation includes at least a first value of a first component and a second value of a second component; Based on the orientation groups of the 3D points, a histogram including a group of intervals representing the orientation groups of the 3D points is generated, wherein: Each interval in the set of intervals is associated with a first range of values ​​for the first component and a second range of values ​​for a second component; and Generating the histogram comprises, for each bin in the group of bins, adding orientations from the group of orientations having first and second values ​​within the first and second value ranges, respectively, associated with the bin to determine a representation of a plurality of orientations of the 3D point common to each bin; and The histogram is stored for comparison with a second histogram associated with a second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud.

17. The system of claim 16, wherein: The instructions are further operable to cause the at least one processor to perform: generating a 3D voxel grid for at least a portion of the 3D point cloud, wherein each voxel in the 3D voxel grid comprises a same set of dimensions; For each voxel in the 3D voxel grid, determining whether one or more of the plurality of 3D data points are within the voxel to generate a 3D point group associated with the voxel; for each voxel in the 3D voxel grid having an associated set of 3D points, determining a single 3D data point for the voxel based on the associated set of 3D data points; and The single 3D data point is stored in the voxel.

18. The system of claim 16, wherein: Generating the orientation group includes: for each 3D point in the 3D point cloud, determining the orientation of the 3D point based on a fixed coordinate system associated with the 3D point cloud.

19. The system of claim 16, wherein: Generating the orientation group includes: for each 3D point in the 3D point cloud, determining an orientation of the 3D point based on a local coordinate system associated with the 3D point in the 3D point cloud.

20. The system of claim 16, wherein: The instructions are further operable to cause the at least one processor to perform: Comparing the histogram with the second histogram associated with the second 3D point cloud to determine data indicative of a similarity measure between the 3D point cloud and the second 3D point cloud, comprises: determining a first set of peaks of the histogram and a second set of peaks of the second histogram; and A correspondence between at least a portion of the first group of peaks and at least a portion of the second group of peaks is determined.

Images