Method and apparatus for identifying surface features in three-dimensional images

Abstract

The technology described herein relates to methods, apparatuses, and computer readable media configured to identify surface features of a portion of a three-dimensional (3D) point cloud. Data indicative of a path along the 3D point cloud is received, where the 3D point cloud includes a plurality of 3D data points. A plurality of 3D data point lists is generated, where: each 3D data point list extends through the 3D point cloud at a location that intersects the received path; and each 3D data point list intersects the received path at a different location. Characteristics associated with surface features are identified in at least some of the plurality of 3D data point lists. The identified characteristics are grouped based on one or more attributes of the identified characteristics. Surface features are identified based on the grouped characteristics.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application Serial No. 63 / 023,186, filed May 11, 2020, entitled “Methods and Apparatus for Identifying Surface Features in Three-Dimensional Images”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The techniques described herein generally relate to methods and apparatuses for machine vision, including techniques for identifying surface features of objects captured in three-dimensional images. Background Technology

[0004] Machine vision systems can include robust imaging capabilities, including three-dimensional (3D) imaging devices. For example, a 3D sensor can image a scene to generate a set of 3D points, each including its (x, y, z) position in a 3D coordinate system (e.g., where the z-axis of the coordinate system represents the distance from the 3D imaging device). Such a 3D imaging device can generate a 3D point cloud, which includes a set of 3D points captured during the 3D imaging process. However, the absolute number of 3D points in a 3D point cloud can be enormous (e.g., compared to the 2D data of the scene). Furthermore, a 3D point cloud may only include pure 3D data points and therefore may not include data indicating relationships between / among the 3D points, or other information such as surface normals. Processing 3D points without data indicating relationships between other points can be complex. Therefore, although 3D point clouds can provide a large amount of 3D data, performing machine vision tasks on 3D point cloud data can be complex, time-consuming, and / or require significant processing resources, etc. Summary of the Invention

[0005] According to the disclosed subject matter, apparatus, systems, and methods are provided for improved machine vision techniques, particularly for improved machine vision techniques capable of locating surface features of objects captured in 3D images. For example, the techniques can locate wrinkles, ridges, and / or other crease edges in the surface of a 3D image.

[0006] Some aspects relate to a computerized method for identifying surface features of a portion of a three-dimensional (3D) point cloud. The method includes receiving data indicating a path along the 3D point cloud, wherein the 3D point cloud comprises a plurality of 3D data points. The method includes generating a plurality of lists of 3D data points, wherein each list of 3D data points extends through the 3D point cloud at a location where it intersects the received path, and each list of 3D data points intersects the received path at a different location. The method includes identifying properties associated with surface features in at least some of the plurality of lists of 3D data points. The method includes grouping the identified properties based on one or more attributes of the identified properties. The method includes identifying surface features of a portion of the 3D point cloud based on the grouped properties.

[0007] Based on some examples, generating multiple lists of 3D data points involves: for each list of 3D data points, determining a set of 3D data points based on a plane set at the location to generate the list of 3D data points. This plane can be orthogonal to the received path, such that the planes used to generate each of the 3D data point lists are parallel.

[0008] According to some examples, the surface features identified in a portion of a 3D point cloud are crease edges, and the characteristic associated with crease edges is corners.

[0009] According to some examples, grouping the identified features includes fitting lines to the identified features. Fitting lines to the identified features may include: fitting a first line to a first portion of the identified features and fitting a second line to a second portion of the identified features; determining a first orientation of the first line that is included within a threshold difference of a second orientation of the second line; and combining the first line and the second line into a single representative line.

[0010] According to some examples, receiving data indicating a path along a 3D point cloud includes receiving data indicating a region of interest. The region of interest can include width, length, and height, and the path is specified based on these dimensions.

[0011] According to some examples, the method includes determining a count of the identified features; and based on the count, determining the coverage of surface features along a portion of a 3D point cloud. The method may include determining a score for each of the identified features; and based on the count of the identified features and the score of each of the identified features, determining a total score for that feature.

[0012] Based on some examples, a path can be a linear path, a curved path, a circular path, a spiral path, or some combination thereof, specified by direction.

[0013] Some embodiments relate to a non-transitory computer-readable medium including instructions that, when executed by one or more processors on a computing device, are operable to cause one or more processors to perform any of the methods described in the embodiments herein.

[0014] Some embodiments relate to a system including a memory storing instructions and a processor configured to execute the instructions to perform any of the methods described in the embodiments herein.

[0015] Therefore, the features of the disclosed subject matter have been outlined rather broadly to facilitate a better understanding of its subsequent detailed description and its contribution to the prior art. Additional features of the disclosed subject matter will, of course, be described below and will form the subject matter of the appended claims. It should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. Attached Figure Description

[0016] In the accompanying drawings, each identical or nearly identical component illustrated in the various figures is indicated by the same reference numerals. For clarity, not every component is labeled in every figure. The drawings are not necessarily drawn to scale, but rather focus on illustrating various aspects of the technology and equipment described herein.

[0017] Figure 1 This is a diagram illustrating an exemplary machine vision system according to some embodiments.

[0018] Figure 2 This is a flowchart illustrating an exemplary computerized method for identifying surface features of a portion of a 3D point cloud, according to some embodiments.

[0019] Figure 3 This is a diagram illustrating an example of a 3D point cloud according to some embodiments.

[0020] Figure 4 This illustrates crossing according to some embodiments. Figure 3 An example diagram of the path of a point cloud.

[0021] Figure 5A This is a diagram illustrating an example of a plane for determining a list of 3D data points according to some embodiments.

[0022] Figure 5B It is shown along some embodiments Figure 5A A diagram of a 3D data point list on a plane.

[0023] Figure 5C This is a diagram illustrating two additional planes and an associated list of 3D data points according to some embodiments.

[0024] Figures 6A to 6C This illustrates a method based on some embodiments. Figures 5B to 5C An example diagram of a corner identified from a list of 3D data points.

[0025] Figure 7 This is an illustration from some embodiments. Figures 6A to 6C An example diagram of the identification of corners.

[0026] Figure 8 This is a diagram illustrating an example of an identified crease edge according to some embodiments. Detailed Implementation

[0027] The techniques described in this paper provide data simplification techniques that can be used to analyze 3D point cloud images. The inventors have recognized that conventional machine vision techniques can be significantly inefficient when using 3D images (such as 3D point cloud data, voxel meshes, and / or 3D volumetric representations). For example, 3D point clouds typically consist of hundreds of thousands or millions of points. Directly interpreting such a large number of 3D points in space can be quite challenging. Furthermore, 3D point clouds typically do not include information about the spatial relationships between the 3D points. Therefore, identifying object features in 3D images can be time-consuming and require significant processing resources. Some techniques attempt to directly identify local regions in a point cloud by searching for properties of features (e.g., crease edges) and then linking the identified regions to identify overall surface features. However, such techniques can be slow and rudimentary because locally identifying the characteristics of similar objects with arbitrary orientations can be challenging.

[0028] The inventors have made technological improvements to machine vision technology to address these and other inefficiencies. According to some embodiments, the technology may include accessing and / or receiving a search volume and a search path, and searching along that path to locate (e.g., identify and / or localize) features of an object surface in 3D image data. As an illustrative example, the technology can be used to locate crease edges (e.g., folds in folded paper, and / or corners of boxes, etc.) in various orientations (e.g., in concave or convex views). Crease edges and / or other object features (e.g., arcs, circles, etc.) can be used for various machine vision tasks, such as measuring and / or inspecting manufactured parts.

[0029] In the following description, numerous specific details are set forth regarding the systems and methods of the disclosed subject matter, as well as the environments in which such systems and methods may operate, in order to provide a thorough understanding of the disclosed subject matter. Furthermore, it will be understood that the examples provided below are exemplary, and other systems and methods within the scope of the disclosed subject matter can be envisioned.

[0030] Figure 1An exemplary machine vision system 100 according to some embodiments is shown. The exemplary machine vision system 100 includes a camera 102 (or other imaging acquisition device) and a computer 104. Although in Figure 1 Only one camera 102 is shown, but it should be understood that multiple cameras can be used in a machine vision system (e.g., where a point cloud is derived from point clouds from multiple cameras). Computer 104 includes one or more processors and a human-machine interface in the form of a computer monitor, and optionally one or more input devices (e.g., keyboard, mouse, trackball, etc.). Camera 102 includes a lens 106 and a camera sensor element (not shown), as well as other components. Lens 106 includes a field of view 108 and focuses light from the field of view 108 onto the sensor element. The sensor element generates a digital image of the camera field of view 108 and provides this image to the processor, which is part of computer 104. Figure 1 As shown in the example, object 112 travels along conveyor 110 into the field of view 108 of camera 102. While object 112 is in the field of view 108, camera 102 can generate one or more digital images of the object for processing, as discussed further herein. In operation, the conveyor may contain multiple objects. For example, during inspection, these objects may pass sequentially within the field of view 108 of camera 102. In this way, camera 102 can acquire at least one image of each observed object 112.

[0031] In some embodiments, camera 102 is a three-dimensional (3D) imaging device. As an example, camera 102 may be a 3D sensor that scans a scene line by line, such as a DS-line of a laser profilometer 3D displacement sensor available from the assignee of this application, Cognex Corp. According to some embodiments, the 3D imaging device may generate a set of (x, y, z) points (e.g., where the z-axis adds a third dimension, such as distance from the 3D imaging device). The 3D imaging device may use various 3D image generation techniques, such as shape-from-shading, stereo imaging, time-of-flight techniques, projector-based techniques, and / or other 3D generation techniques. In some embodiments, machine vision system 100 includes a two-dimensional imaging device, such as a two-dimensional (2D) CCD or CMOS imaging array. In some embodiments, the two-dimensional imaging device generates a 2D array of brightness values.

[0032] In some embodiments, the machine vision system processes 3D data from camera 102. The 3D data received from camera 102 may include, for example, point clouds and / or distance images. A point cloud may include a set of 3D points on or near the surface of a solid object. For example, these points may be represented by their coordinates in a straight line or other coordinate system. In some embodiments, other information may optionally be present, such as a grid or raster structure indicating which points are neighbors on the object surface. In some embodiments, information about surface features (including curvature, surface normals, edges, and / or color and albedo information), whether derived from sensor measurements or previously calculated, may be included in the input point cloud. In some embodiments, 2D and / or 3D data may be obtained from 2D and / or 3D sensors, from CAD or other solid models, and / or through preprocessing distance images, 2D images, and / or other images.

[0033] According to some embodiments, the set of 3D points may be part of a 3D point cloud within a user-specified region of interest and / or include data from the specified region of interest within the 3D point cloud. For example, since a 3D point cloud may include so many points, it may be desirable to specify and / or define one or more regions of interest (e.g., to limit the space on which the techniques described herein are applied).

[0034] Examples of computer 104 may include, but are not limited to, a single server computer, a series of server computers, a single personal computer, a series of personal computers, a minicomputer, a mainframe computer, and / or a computing cloud. Various components of computer 104 may execute one or more operating systems, examples of which may include, but are not limited to, Microsoft Windows Server. TM Novell Netware TM Red Hat Linux TM Unix and / or custom operating systems. One or more processors of computer 104 may be configured to process operations stored in memory connected to one or more processors. Memory may include, but is not limited to: hard disk drives; flash drives; tape drives; optical drives; RAID arrays; random access memory (RAM); and read-only memory (ROM).

[0035] As described herein, techniques are provided for identifying and locating object features, such as crease edges, on the surface of a 3D image. According to some embodiments, the techniques may include selecting a set of planes based on a specified path; and determining a corresponding list of points from the 3D image based on the intersection of the surfaces of the planes and points in the 3D image. As described herein, the list of points may be a linear array of points. It should be understood that points can provide a 1D manifold of 3D (e.g., a 1D path along the object surface). The region including the list of points may be a 2D plane. Thus, the list of points can provide a 2D path along a 2D plane. The techniques may analyze each list of points to extract object characteristics, such as corners and / or other points with high curvature (e.g., a path along a contour at a given scale). The extracted characteristics may be grouped based on their 3D proximity and / or other properties. According to some examples, the techniques may fit lines to the extracted features to identify overall surface features of an object in a 3D image.

[0036] Figure 2 This is a flowchart illustrating an exemplary computerized method 200 for identifying surface features (e.g., crease edges) of a portion of a 3D point cloud according to some embodiments. As described herein, a 3D point cloud comprises a large number of 3D data points and may include hundreds of thousands or millions of 3D data points. Figure 3 This is a diagram illustrating an example of a 3D point cloud 300 according to some embodiments. For illustrative purposes, only a small number of 3D points 302A, 302B to 302N are shown in the 3D point cloud 300, which are collectively referred to as 3D points 302. Also for illustrative purposes, Figure 3 The outline of box 304 is shown (e.g., a box existing in a scene captured by a 3D imaging device). It should be understood that the 3D point cloud 300 does not include information indicating box 304. Instead, the 3D point cloud 300 includes only data on 3D points 302, and may not even include data indicating relationships between the 3D points 302 themselves. For example, the 3D point cloud 300 may simply include the (x, y, z) position of each 3D point 302 in the 3D point cloud 300. Figure 3 The point cloud shown is intended to provide examples of the techniques described in this article. Therefore, it should be understood that the techniques are not intended to be limited to these examples.

[0037] At step 202, the machine vision system (e.g., Figure 1 The machine vision system 100 accesses and / or receives data indicating a path along a 3D point cloud. According to some embodiments, Figure 4 It shows the way Figure 3An example of path 402 of point cloud 300. Path 402 is a linear path. Path 402 can be specified as, for example, a direction (e.g., a direction specified using the coordinate system associated with 3D point cloud 300). However, it should be understood that while path 402 is a linear path, this is for illustrative purposes only and is not intended to be limiting. Paths of various shapes and types can be used as needed. For example, paths can include curved paths, circular paths, spiral paths, and / or rectangular paths, etc.

[0038] According to some embodiments, a region of interest (ROI) in a 3D point cloud can be used to specify a path. For example, a machine vision system can receive and / or access a specified ROI (e.g., a box, a sphere, etc.) that includes data indicating the path. The path can be specified relative to the ROI. For example, if the ROI is a box, it can include data indicating its width, length, and height, and can include data specifying the path based on the width, length, and height. As another example, the ROI can be a cylinder, and it can include data specifying the path based on a cylindrical coordinate system associated with the cylinder.

[0039] At step 204, the machine vision system generates multiple lists of 3D data points. Each list of 3D data points extends through the 3D point cloud at a location where it intersects the path. Therefore, each list of 3D data points can capture a 1D manifold of points that may be captured along the object surface by the 3D point cloud. According to some embodiments, a set of two-dimensional planes extending along the path can be used to generate the list of points, such that each list of points lies within that plane and / or within a threshold distance of that plane. The planes can have various arrangements along the path. For example, the planes can be parallel to each other and extend along the direction of a straight path. As another example, the planes can be orthogonal to a curved path along the path direction. Each plane, and therefore each list of 3D data points, intersects the received path at a different location and / or in a different direction.

[0040] Figure 5A An example of a plane 502 for determining a list of 3D data points is shown according to some embodiments. In this example, plane 502 is set orthogonal to path 402, but as described herein, this is for illustrative purposes and is not intended to be limiting. Figure 5B As shown, the machine vision system determines the 3D data point list 504 by identifying points along plane 502. Figure 5C Two additional lists of 3D data points are shown according to some embodiments. For example... Figure 5C As shown, according to some embodiments, plane 506 is used to identify an associated list of 3D data points 508, and plane 510 is used to identify an associated list of 3D data points 512. In this example, planes 502, 506, and 510 are parallel to each other.

[0041] At step 206, the machine vision system identifies features associated with surface features from at least some of the multiple lists of 3D data points. Features may be, for example, defining characteristics of an object, such as ridges, creases, corners, and / or edges. For example, the technique may identify corners or other points of high curvature at a given scale along a path of contour. Figures 6A to 6C The following are illustrations based on some embodiments. Figures 5B to 5C Examples of corners 602, 604, and 606 identified in the 3D data point lists 504, 508, and 512. For illustrative purposes, corners 602, 604, and 606 are shown with lines along these points; however, this is done for illustrative purposes because machine vision systems can internally store data indicating corners in various formats. For example, data indicating corners may include points and / or one or more of a set of rays, the angle between the rays, the corner score, data indicating whether the corner is concave or convex, etc.

[0042] At step 208, the machine vision system groups the identified features based on one or more attributes of the identified features. For example, the machine vision system may group the identified features based on their proximity in 3D and / or based on other attributes (e.g., whether grouping the features forms a path within the expected orientation of the features). Figure 7 The following are examples of some embodiments. Figures 6A to 6C The exemplary grouping of the identified corners 602, 604, and 606. For example, a machine vision system can determine that each corner has the same orientation (or similar orientations within a threshold) in the coordinate system of a 3D point cloud, and group these three corners as potentially related to each other.

[0043] At step 210, the machine vision system identifies surface features of a portion of the 3D point cloud based on the grouped characteristics. According to some embodiments, the machine vision system can fit a line to each group of identified characteristics. For example, the machine vision system can determine the best-fit line for each group, such as by using least-squares fitting techniques and / or RANSAC. Figure 8 An example of an identified crease edge 800 according to some embodiments is shown. Figure 8 As shown, the crease edge 800 represents a portion of the edge 802 of the 3D object 304 captured by the 3D point cloud 300. Therefore, the technique can be used to identify features in the 3D point cloud, which can then be used to ultimately identify the object captured by the 3D point cloud.

[0044] As described herein, 3D point clouds may exhibit some noise, causing determined features to have some threshold noise differences (e.g., compared to features of a real object captured by a machine vision system). According to some embodiments, lines can be fitted to recognized features by fitting multiple lines to different sets of features and combining lines within a threshold orientation. For example, a machine vision system can fit a first line to a first portion of the recognized features and a second line to a second portion of the recognized features. The machine vision system can determine that the first line has a first orientation within a threshold difference of a second orientation of the second line (e.g., these lines are approximately collinear), and can combine the first and second lines into a single representative line.

[0045] According to some embodiments, machine vision systems can be configured to use thresholds (e.g., spacing or orientation thresholds) to determine whether to recognize multiple lines. For example, if the machine vision system is imaging an object with linear features that are very close together (e.g., about micrometers), the machine vision system can be configured with thresholds such that the machine vision system recognizes two lines instead of combining the points / lines into a single line. As an illustrative example, if the object has lines known to be 10 micrometers apart, the threshold can be set based on the known distance (e.g., distance 10 micrometers, 12 micrometers, etc.) such that if the machine vision system determines that the distance between two lines is less than the threshold distance, the machine vision system will separate the two lines into two lines instead of combining them into a single line. As another example, the machine vision system can use thresholds when determining which 3D points belong to an associated line or feature.

[0046] According to some embodiments, a machine vision system can determine the coverage of surface features in a point cloud. For example, the machine vision system can determine a count of identified features and, based on that count, determine the coverage of surface features along a portion of a 3D point cloud.

[0047] According to some embodiments, a machine vision system can score identified features. For example, a machine vision system can determine a score for each of the identified features. The machine vision system can determine the total score for a feature based on the count of the identified features and the score of each of the identified features.

[0048] Various scoring techniques can be used to score each identified feature. For example, a machine vision system can score each corner based on how many samples are close to the straight line segment forming the corner. Then, the machine vision system can score the found crease edges based on the number and quality (e.g., collinearity) of matching corners. The scoring technique can be designed to determine the reliability of the edges determined based on the identified corners (e.g., rather than determining whether the edges are perfectly formed).

[0049] As described in this article, Figures 3 to 8 This is a simplified diagram used to illustrate the techniques described herein, and modifications can be made to various aspects without departing from the spirit of the techniques described herein. For example, the technique can extract nonlinear features (e.g., creases) by linking edge lines or corners and reporting the chain as a result. In some embodiments, the machine vision system can analyze 3D points radially along the circumference of a circular path. Such techniques can be used, for example, to accurately fit a circle at an approximate location (e.g., to obtain the radius and / or other dimensions of a circular feature). As another example, the technique can extract linear creases in all possible directions by applying the process described herein to a set of extraction plane orientations. As a further example, the techniques described herein can be repeated with various corner detection scales (e.g., in exponential order) to find crease edges at various scales, and optionally select the highest-scoring result from a group that is spatially close. For example, such a method can eliminate the need to specify scales on all surface features.

[0050] In some embodiments, a machine vision system can be configured to operate at a desired resolution. At high resolution, linear features may appear non-linear (e.g., wavy) due to surface defects and / or imaging artifacts, while at coarser / lower resolution, linear features may appear more linear. For example, a high-resolution imaging device can extract micrometer-level details, allowing ridges on machined aluminum parts invisible to the human eye to be detected by the imaging device. In some embodiments, the resolution can be controlled by downsampling the point cloud before identifying surface features. The machine vision system can, for example, merge multiple 3D points into a voxel by reducing multiple 3D points in a voxel to a single 3D point. For example, for one or more voxels of a plane, the machine vision system can determine multiple 3D data points in the voxel, determine a single 3D data point of the voxel based on the 3D data points, and store the single 3D data point in the voxel. According to some embodiments, the machine vision system can compute the centroid of the 3D points in the voxel. According to some embodiments, the technique can determine the centroid of a voxel based on points in the current voxel and / or points in neighboring voxels. For example, the centroid can be calculated based on points in the current voxel and points in the twenty-six (26) voxels adjacent to the current voxel (e.g., nine voxels below the edge of the voxel, nine voxels above the edge of the voxel, and eight voxels around the edge of the voxel).

[0051] The techniques described in this article can be used in a variety of machine vision applications. For example, they can be used for object detection, inspection, and / or analysis. As an example, the techniques can be used for object measurement, such as measuring the length / size of a specific crease by using its edges, and / or comparing its distance to other creases, etc. As another example, for inspection, the techniques can be used to inspect objects to ensure that creases are correctly formed within the object, and / or to ensure proper positioning / setting on the object, etc.

[0052] The techniques operating according to the principles described herein can be implemented in any suitable manner. The processing and decision boxes in the flowcharts above represent steps and actions that may be included in algorithms that perform these different processes. The algorithms derived from these processes can be implemented as software integrated with and instructing the operation of one or more single-purpose or multi-purpose processors, as functionally equivalent circuits such as digital signal processing (DSP) circuits or application-specific integrated circuits (ASICs), or in any other suitable manner. It should be understood that the flowcharts included herein do not describe the syntax or operation of any particular circuit or any particular programming language or type of programming language. Rather, the flowcharts illustrate functional information that those skilled in the art can use to manufacture circuits or implement computer software algorithms to perform processing of a particular type of technique described herein. It should also be understood that, unless otherwise indicated herein, the specific order of steps and / or actions described in each flowchart is merely an illustration of the algorithms that can be implemented and may vary in implementations and embodiments of the principles described herein.

[0053] Accordingly, in some embodiments, the techniques described herein can be embodied in computer-executable instructions implemented as software, including application software, system software, firmware, middleware, embedded code, or any other suitable type of computer code. Such computer-executable instructions can be written using a variety of suitable programming languages ​​and / or any of the programming or scripting tools, and can also be compiled into executable machine language code or intermediate code that executes on a framework or virtual machine.

[0054] When the techniques described herein are implemented as computer-executable instructions, these instructions can be implemented in any suitable manner, including as multiple functional facilities, each providing one or more operations to perform the execution of an algorithm based on these techniques. Regardless of how they are instantiated, a "functional facility" is a structural component of a computer system that, when integrated with and executed by one or more computers, causes the one or more computers to perform a specific operational role. A functional facility can be part of or an entire software element. For example, a functional facility can be implemented as a process, either as a discrete process or as any other suitable processing unit. If the techniques described herein are implemented as multiple functional facilities, each functional facility can be implemented in its own way; not all functional facilities need to be implemented in the same way. Furthermore, these functional facilities can be executed in parallel and / or serially as appropriate, and can exchange information with each other using shared memory on the computer on which they are executed, using message passing protocols, or in any other suitable manner.

[0055] Typically, functional facilities include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. The functionality of functional facilities can generally be combined or distributed as needed within the system in which they operate. In some implementations, one or more functional facilities that implement the techniques described herein can together form a complete software package. In alternative embodiments, these functional facilities may be adapted to interact with other unrelated functional facilities and / or processes to enable software program applications.

[0056] This document has described some exemplary functional facilities for performing one or more tasks. However, it should be understood that the described functional facilities and task partitioning are merely illustrative of the types of functional facilities that can implement the exemplary techniques described herein, and embodiments are not limited to implementation in any particular number, partition, or type of functional facilities. In some implementations, all functions may be implemented in a single functional facility. It should also be understood that in some implementations, some of the functional facilities described herein may be implemented together with or separately from other functional facilities (i.e., as a single unit or separate units), or some of these functional facilities may not be implemented.

[0057] In some embodiments, computer-executable instructions implementing the techniques described herein (when implemented as one or more functional facilities or in any other way) may be encoded on one or more computer-readable media to provide functionality to the medium. Computer-readable media include magnetic media such as hard disk drives, optical media such as optical discs (CDs) or digital versatile discs (DVDs), permanent or non-permanent solid-state storage (e.g., flash memory, magnetic RAM, etc.), or any other suitable storage medium. Such computer-readable media may be implemented in any suitable manner. As used herein, a “computer-readable medium” (also referred to as a “computer-readable storage medium”) means a tangible storage medium. Tangible storage media are non-transitory and have at least one physical structural component. In a “computer-readable medium” as used herein, at least one physical structural component has at least one physical characteristic that can be altered in some way during the creation of a medium with embedded information, during the recording of information thereon, or during any other process of encoding the medium with information. For example, the magnetization state of a portion of the physical structure of a computer-readable medium may be altered during the recording process.

[0058] Furthermore, some of the aforementioned technologies include actions of storing information (e.g., data and / or instructions) in a specific manner for use by these technologies. In some implementations of these technologies, such as those where the technology is implemented as computer-executable instructions, information can be encoded on a computer-readable storage medium. Where specific structures are described herein as advantageous formats for storing this information, these structures can be used to give the information a physical organization when encoded on the storage medium. These advantageous structures can then provide functionality to the storage medium by influencing the operation of one or more processors interacting with the information; for example, by improving the efficiency of computer operations performed by the processors.

[0059] In some, but not all, implementations in which the technology can be implemented as computer-executable instructions, these instructions can be executed on one or more suitable computing devices operating in any suitable computer system, or one or more computing devices (or one or more processors of one or more computing devices) can be programmed to execute the computer-executable instructions. A computing device or processor can be programmed to execute these instructions when they are stored in a manner accessible to the computing device or processor, such as a data storage device (e.g., an on-chip cache or instruction register, a computer-readable storage medium accessible via a bus, a computer-readable storage medium accessible via one or more networks and accessible by the device / processor, etc.). Functional facilities including these computer-executable instructions can be integrated with and direct the operation of a single multipurpose programmable digital computing device, a coordinated system of two or more multipurpose computing devices sharing processing power and jointly executing the technologies described herein, a single computing device dedicated to executing the technologies described herein, or a coordinated system of computing devices (located in one location or geographically distributed), one or more field-programmable gate arrays (FPGAs) for executing the technologies described herein, or any other suitable system.

[0060] A computing device may include at least one processor, a network adapter, and a computer-readable storage medium. The computing device may be, for example, a desktop or laptop computer, a personal digital assistant (PDA), a smartphone, a server, or any other suitable computing device. The network adapter may be any suitable hardware and / or software that enables the computing device to communicate wired and / or wirelessly with any other suitable computing device over any suitable computing network. The computing network may include wireless access points, switches, routers, gateways, and / or other network devices, and any suitable wired and / or wireless communication media, including the Internet, for exchanging data between two or more computers. The computer-readable medium may be adapted to store data to be processed by the processor and / or instructions to be executed by the processor. The processor is capable of processing data and executing instructions. Data and instructions may be stored on the computer-readable storage medium.

[0061] Computing devices may additionally include one or more components and peripherals, including input devices and output devices. These devices are particularly useful for presenting user interfaces. Examples of output devices that can be used to provide a user interface include printers or displays for visual presentation of output, and speakers or other sound-generating devices for auditory presentation of output. Examples of input devices that can be used for a user interface include keyboards and pointing devices such as mice, touchpads, and digitizers. As another example, computing devices may receive input information via speech recognition or in other auditory formats.

[0062] Embodiments of implementing the technology in circuits and / or computer-executable instructions have been described. It should be understood that some embodiments may be in the form of methods, at least one example of which has been provided. Actions performed as part of a method may be ordered in any suitable manner. Accordingly, embodiments may be constructed in which actions are performed in a different order than illustrated, which may include performing several actions simultaneously, even if they are shown as sequential actions in the illustrative embodiments.

[0063] The various aspects of the above embodiments can be used individually, in combination, or in various arrangements not specifically discussed in the foregoing embodiments, and therefore their application is not limited to the details and arrangements of the components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment can be combined with aspects described in other embodiments in any way.

[0064] The use of ordinal numbers such as “first,” “second,” and “third” to modify claim elements in claims does not imply any priority, order of precedence, or sequence of actions of one claim element relative to another, or the temporal order of the actions of the method. Rather, it serves merely as a label to distinguish one claim element with a certain name from another element with the same name (but using an ordinal number).

[0065] Furthermore, the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The terms “including,” “contains,” “has,” “includes,” “involves,” and variations thereof are used herein to refer to items listed thereafter, their equivalents, and additional items.

[0066] The term “exemplary” is used herein to mean that it is used as an example, instance, or illustration. Any embodiments, implementations, processes, features, etc., described herein as exemplary should therefore be understood as illustrative examples and not as preferred or advantageous examples, unless otherwise stated.

[0067] Therefore, having described several aspects of at least one embodiment, it should be understood that various changes, modifications, and improvements will readily occur to those skilled in the art. Such changes, modifications, and improvements are intended to be part of this disclosure and are intended to remain within the spirit and scope of the principles described herein. Accordingly, the foregoing description and figures are merely exemplary.

[0068] Various aspects are described in this disclosure, including, but not limited to, the following:

[0069] 1. A computerized method for identifying surface features of a portion of a three-dimensional (3D) point cloud, the method comprising:

[0070] Receive data indicating a path along a 3D point cloud, wherein the 3D point cloud comprises a plurality of 3D data points;

[0071] Generate multiple lists of 3D data points, including:

[0072] Each list of 3D data points extends through the 3D point cloud at the location where it intersects with the received path; and

[0073] Each list of 3D data points intersects the received path at a different location;

[0074] Identify properties associated with surface features from at least some of the plurality of 3D data point lists;

[0075] The characteristics of the identification are grouped based on one or more attributes of the identified characteristics; and

[0076] Based on the characteristics of the grouping, the surface features of the portion of the 3D point cloud are identified.

[0077] 2. The method according to Clause 1, wherein generating the plurality of 3D data point lists comprises: for each 3D data point list, determining a set of 3D data points based on a plane set at the location to generate the 3D data point list.

[0078] 3. The method according to Clause 2, wherein the plane is orthogonal to the received path such that the plane used to generate each of the 3D data point list is parallel.

[0079] 4. The method according to clauses 1 to 3, wherein the surface feature identified in the portion of the 3D point cloud is a crease edge, and the feature associated with the crease edge is a corner.

[0080] 5. The method according to clauses 1 to 4, wherein grouping the identified features comprises: fitting a line to the identified features.

[0081] 6. The method according to Clause 5, wherein fitting the line to the identified feature comprises:

[0082] A first line is fitted to a first portion of the identified feature, and a second line is fitted to a second portion of the identified feature;

[0083] Determine that the first line includes a first orientation within a threshold difference of the second orientation of the second line; and

[0084] Combine the first line and the second line into a single representative line.

[0085] 7. The method according to Clauses 1 to 6, wherein receiving the data indicating a path along the 3D point cloud comprises: receiving data indicating a region of interest.

[0086] 8. The method according to Clause 7, wherein the designated region of interest includes width, length and height, and the path is designated based on the width, the length and the height.

[0087] 9. The method described under Articles 1 to 8 further includes:

[0088] Determine the count of the identified features;

[0089] Based on the count, the coverage area of ​​the surface feature along a portion of the 3D point cloud is determined.

[0090] 10. The method described in Clause 9 further includes:

[0091] Determine the score for each of the identified features;

[0092] The total score of a feature is determined based on the count of the identified features and the score of each of the identified features.

[0093] 11. The method according to clauses 1 to 10, wherein the path is:

[0094] A linear path specified by direction;

[0095] curved path;

[0096] Circular path;

[0097] Spiral paths; or some combination thereof.

[0098] 12. A non-transitory computer-readable medium comprising instructions, when executed by one or more processors on a computing device, operable to cause the one or more processors to recognize surface features of a portion of a three-dimensional (3D) point cloud, comprising:

[0099] Receive data indicating a path along a 3D point cloud, wherein the 3D point cloud comprises a plurality of 3D data points;

[0100] Generate multiple lists of 3D data points, including:

[0101] Each list of 3D data points extends through the 3D point cloud at the location where it intersects with the received path; and

[0102] Each list of 3D data points intersects the received path at a different location;

[0103] Identify properties associated with surface features from at least some of the plurality of 3D data point lists;

[0104] The characteristics of the identification are grouped based on one or more attributes of the identified characteristics; and

[0105] Based on the characteristics of the grouping, the surface features of the portion of the 3D point cloud are identified.

[0106] 13. The non-transitory computer-readable medium according to Clause 12, wherein generating the plurality of 3D data point lists comprises: for each 3D data point list, determining a set of 3D data points based on a plane disposed at the location to generate the 3D data point list.

[0107] 14. The non-transitory computer-readable medium according to clauses 12 to 13, wherein grouping the identified features includes fitting a line to the identified features, comprising:

[0108] A first line is fitted to a first portion of the identified feature, and a second line is fitted to a second portion of the identified feature;

[0109] Determine that the first line includes a first orientation within a threshold difference of the second orientation of the second line; and

[0110] Combine the first line and the second line into a single representative line.

[0111] 15. The non-transitory computer-readable medium according to clauses 12 to 14, wherein receiving the data indicating a path along the 3D point cloud comprises: receiving data indicating a specified region of interest including width, length, and height, and specifying the path based on the width, the length, and the height.

[0112] 16. The non-transitory computer-readable medium according to clauses 12 to 15, wherein the instructions are operable to cause the one or more processors to perform:

[0113] Determine the count of the identified features;

[0114] Based on the count, the coverage area of ​​the surface feature along a portion of the 3D point cloud is determined.

[0115] 17. A system comprising a memory storing instructions and at least one processor configured to execute the instructions to identify surface features of a portion of a three-dimensional (3D) point cloud, comprising:

[0116] Receive data indicating a path along a 3D point cloud, wherein the 3D point cloud comprises a plurality of 3D data points;

[0117] Generate multiple lists of 3D data points, including:

[0118] Each list of 3D data points extends through the 3D point cloud at the location where it intersects with the received path; and

[0119] Each list of 3D data points intersects the received path at a different location;

[0120] Identify properties associated with surface features from at least some of the plurality of 3D data point lists;

[0121] The characteristics of the identification are grouped based on one or more attributes of the identified characteristics; and

[0122] Based on the characteristics of the grouping, the surface features of the portion of the 3D point cloud are identified.

[0123] 18. The system according to Clause 17, wherein generating the plurality of 3D data point lists comprises: for each 3D data point list, determining a set of 3D data points based on a plane disposed at the location to generate the 3D data point list.

[0124] 19. The system according to claims 17 to 18, wherein grouping the identified features comprises: fitting a line to the identified features, including:

[0125] A first line is fitted to a first portion of the identified feature, and a second line is fitted to a second portion of the identified feature;

[0126] Determine that the first line includes a first orientation within a threshold difference of the second orientation of the second line; and

[0127] Combine the first line and the second line into a single representative line.

[0128] 20. The system according to clauses 17 to 19, wherein receiving the data indicating a path along the 3D point cloud comprises: receiving data indicating a specified region of interest including width, length, and height, and specifying the path based on the width, the length, and the height.

Claims

1. A computerized method for identifying surface features of a portion of a three-dimensional (3D) point cloud, the method comprising: Receive data indicating a path along a 3D point cloud, wherein the 3D point cloud comprises a plurality of 3D data points; Generate multiple lists of 3D data points, including: Each list of 3D data points extends through the 3D point cloud at the location where it intersects with the received path; and Each 3D data point list intersects the received path at a different location, thus separating the various lists of 3D data points from each other. Identify a set of surface properties from at least some of the plurality of 3D data point lists, wherein each property of the set of surface properties is associated with a different surface feature; The characteristics of the identification are grouped based on one or more attributes of the identified characteristics; and Based on the characteristics of the grouping, surface features of the portion of the 3D point cloud are identified, wherein the surface features are associated with the identified characteristics and the surface features extend through the 3D point cloud along the identified characteristics of the grouping.

2. The method according to claim 1, wherein, Generating the plurality of 3D data point lists includes: for each 3D data point list, determining the 3D data point list based on a plane set at the location to generate the 3D data point list.

3. The method according to claim 2, wherein, The plane is orthogonal to the received path, such that the plane used to generate each of the 3D data point list is parallel.

4. The method according to claim 1, wherein, The surface feature identified as a portion of the 3D point cloud is a crease edge, and the characteristic associated with the crease edge is a corner.

5. The method according to claim 1, wherein, Grouping the identified features includes fitting lines to the identified features.

6. The method according to claim 5, wherein, Fitting the line to the identified features includes: A first line is fitted to a first portion of the identified feature, and a second line is fitted to a second portion of the identified feature; Determine that the first line includes a first orientation within a threshold difference of the second orientation of the second line; and Combine the first line and the second line into a single representative line.

7. The method according to claim 1, wherein, Receiving the data indicating a path along the 3D point cloud includes receiving data indicating a region of interest.

8. The method according to claim 7, wherein, The specified region of interest includes width, length, and height, and the path is specified based on the width, length, and height.

9. The method according to claim 1, further comprising: Determine the count of the identified features; Based on the count, the coverage area of ​​the surface feature along a portion of the 3D point cloud is determined.

10. The method of claim 9, further comprising: Determine the score for each of the identified features; The total score of the surface feature is determined based on the count of the identified features and the score of each of the identified features.

11. The method according to claim 1, wherein, The path is: A linear path specified by direction; curved path; Circular path; Spiral paths; or some combination thereof.

12. A non-transitory computer-readable medium comprising instructions, when executed by one or more processors on a computing device, operable to cause the one or more processors to recognize surface features of a portion of a three-dimensional (3D) point cloud, comprising: Receive data indicating a path along a 3D point cloud, wherein the 3D point cloud comprises a plurality of 3D data points; Generate multiple lists of 3D data points, including: Each list of 3D data points extends through the 3D point cloud at the location where it intersects with the received path; and Each 3D data point list intersects the received path at a different location, thus separating the various lists of 3D data points from each other. Identify a set of surface properties from at least some of the plurality of 3D data point lists, wherein each property of the set of surface properties is associated with a different surface feature; The characteristics of the identification are grouped based on one or more attributes of the identified characteristics; and Based on the characteristics of the grouping, surface features of the portion of the 3D point cloud are identified, wherein the surface features are associated with the identified characteristics and the surface features extend through the 3D point cloud along the identified characteristics of the grouping.

13. The non-transitory computer-readable medium according to claim 12, wherein, Generating the plurality of 3D data point lists includes: for each 3D data point list, determining the 3D data point list based on a plane set at the location to generate the 3D data point list.

14. The non-transitory computer-readable medium according to claim 12, wherein, Grouping the identified features includes: fitting lines to the identified features, including: A first line is fitted to a first portion of the identified feature, and a second line is fitted to a second portion of the identified feature; Determine that the first line includes a first orientation within a threshold difference of the second orientation of the second line; and Combine the first line and the second line into a single representative line.

15. The non-transitory computer-readable medium according to claim 12, wherein, Receiving the data indicating a path along the 3D point cloud includes: receiving data indicating a specified region of interest including width, length, and height, and specifying the path based on the width, length, and height.

16. The non-transitory computer-readable medium according to claim 12, wherein, The instructions are operable to cause the one or more processors to execute: Determine the count of the identified features; Based on the count, the coverage area of ​​the surface feature along a portion of the 3D point cloud is determined.

17. A system for identifying surface features of a portion of a three-dimensional (3D) point cloud, the system comprising a memory storing instructions and at least one processor configured to execute the instructions to identify the surface features of the portion of the 3D point cloud, comprising: Receive data indicating a path along a 3D point cloud, wherein the 3D point cloud comprises a plurality of 3D data points; Generate multiple lists of 3D data points, including: Each list of 3D data points extends through the 3D point cloud at the location where it intersects with the received path; and Each 3D data point list intersects the received path at a different location, thus separating the various lists of 3D data points from each other. Identify a set of surface properties from at least some of the plurality of 3D data point lists, wherein each property of the set of surface properties is associated with a different surface feature; The characteristics of the identification are grouped based on one or more attributes of the identified characteristics; and Based on the characteristics of the grouping, surface features of a portion of the 3D point cloud are identified, wherein the surface features are associated with the identified characteristics and the surface features extend through the 3D point cloud along the identified characteristics of the grouping.

18. The system according to claim 17, wherein, Generating the plurality of 3D data point lists includes: for each 3D data point list, determining the 3D data point list based on a plane set at the location to generate the 3D data point list.

19. The system according to claim 17, wherein, Grouping the identified features includes: fitting lines to the identified features, including: A first line is fitted to a first portion of the identified feature, and a second line is fitted to a second portion of the identified feature; Determine that the first line includes a first orientation within a threshold difference of the second orientation of the second line; and Combine the first line and the second line into a single representative line.

20. The system according to claim 17, wherein, Receiving the data indicating a path along the 3D point cloud includes: receiving data indicating a specified region of interest including width, length, and height, and specifying the path based on the width, length, and height.

Images