Method and apparatus for determining the volume of a 3D image

Abstract

The techniques described herein relate to methods, devices, and computer-readable media configured to determine an estimated volume of an object captured by a three-dimensional (3D) point cloud. A 3D point cloud including a plurality of 3D points and a reference plane spatially associated with the 3D point cloud are received. A 2D grid of bins is constructed along the reference plane, with each bin of the 2D grid including a length and a width extending along the reference plane. For each bin of the 2D grid, the number of 3D points in the bin and the height of the bin above the reference plane are determined. The estimated volume of the object captured by the 3D point cloud is based on the calculated number of 3D points in each bin and the height of each bin.

Description

[Technical Field]

[0001] Related Applications This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 63 / 023,195, filed May 11, 2020, entitled "METHODS AND APPARATUS FOR DETERMINING VOLUMES OF 3D IMAGES," and U.S. Provisional Application No. 63 / 049,839, filed July 9, 2020, entitled "METHODS AND APPARATUS FOR DETERMINING VOLUMES OF 3D IMAGES," both of which are incorporated herein by reference in their entireties.

[0002] The techniques described herein generally relate to methods and apparatus for machine vision, including techniques for determining the volume of a 3D image, and more particularly to methods and apparatus for determining an approximate volume directly from a 3D point cloud image. [Background technology]

[0003] A machine vision system may include robust imaging capabilities, including a three-dimensional (3D) imaging device. For example, a 3D sensor may image a scene and generate a set of 3D points, each of which includes an (x, y, z) location within a 3D coordinate system (e.g., the z-axis of the coordinate system represents the distance from the 3D imaging device). Such a 3D imaging device may generate a 3D point cloud that includes the set of 3D points captured during the 3D imaging process. However, the absolute number of 3D points in the 3D point cloud may be large (e.g., compared to 2D data of the scene). Furthermore, because a 3D point cloud may include only pure 3D data points, it may not include other information, such as data indicating relationships between / among the 3D points or surface normal information, which may complicate processing of the 3D points without data indicating relationships between other points. Therefore, while a 3D point cloud may provide a large amount of 3D data, performing machine vision tasks on the 3D point cloud data may be complex, time-consuming, require significant processing resources, etc. Summary of the Invention [Problem to be solved by the invention]

[0004] In accordance with the disclosed subject matter, apparatus, systems, and methods are provided for improved machine vision techniques, and in particular for improved machine vision techniques that can quickly and efficiently determine the approximate volume of shapes captured within 3D point cloud images. [Means for solving the problem]

[0005] Some aspects relate to a computerized method for determining an estimated volume of an object captured by a three-dimensional (3D) point cloud. The method includes receiving a 3D point cloud including a plurality of 3D points and data indicative of a reference plane spatially related to the 3D point cloud; constructing a 2D grid of bins along the reference plane, each bin of the 2D grid having a length and a width extending along the reference plane; determining, for each bin of the 2D grid, a number of 3D points in the bin and a height of the bin above the reference plane; and calculating an estimated volume of the object captured by the 3D point cloud based on the calculated number of 3D points in each bin and the height of each bin.

[0006] According to some examples, determining a number of 3D points and a height of each bin of the 2D grid includes, for each bin of the 2D grid, determining a first number of 3D points in the bin on a first side of the reference plane and a first height of the bin from the first side, and calculating an estimated volume of the object includes calculating a first estimated volume of the object. The method may include, for each bin of the 2D grid, determining a second number of 3D points in the bin on a second side of the reference plane opposite the first side and a second height of the bin from the second side, and calculating a second estimated volume of the object. Calculating the estimated volume may include summing the first estimated volume and the second estimated volume.

[0007] According to some examples, determining the number of 3D points for each bin may include ignoring 3D points of the plurality of 3D points within a threshold distance of the reference plane.

[0008] According to some examples, calculating the estimated volume based on the determined number of 3D points in each bin and the height of each bin includes calculating a median number of 3D points in the bin in the 2D grid based on the 3D points in each bin. The method may include, for each bin of the 2D grid, determining a fill weight of the bin based on the number of 3D points in the bin and the median number of 3D points in the bin, determining a median height of the 3D points in the bin from a reference plane, and calculating an estimated volume based on the fill weight of each bin and the median height of each bin. Calculating the estimated volume may include calculating the estimated volume of each bin by calculating the product of (a) the fill weight of the bin, (b) the median height of the 3D points of the bin, and (c) the common area of ​​the bins of the 2D grid, and adding the estimated volume for each bin, to calculate the estimated volume of the object.

[0009] According to some examples, the method includes orienting the point cloud based on a reference plane. Orienting the point cloud based on the reference plane may include determining a reference direction of the reference plane and aligning axes of a coordinate system of the 3D point cloud with the reference direction. Aligning axes of the coordinate system of the 3D point cloud to the reference direction of the reference plane may include determining a transformation that maps the axes to the reference direction and mapping 3D points of the point cloud to the reference plane using the transformation. The axis may be a Z-axis, and the method may further include, for each bin, determining a bin height based on the Z-axis values ​​for each of the 3D points in the bin.

[0010] According to some examples, the method may include receiving data indicating a desired resolution of the estimation volume and determining a length and width of bins of the 2D grid based on the desired resolution.

[0011] Some aspects relate to a non-transitory computer-readable medium that includes instructions that, when executed by one or more processors on a computing device, are operable to cause the one or more processors to perform any of the methods of the techniques described herein.

[0012] Some aspects relate to a system comprising: a memory that stores instructions; and a processor configured to execute the instructions to perform any of the methods of the techniques described herein.

[0013] There has therefore been outlined, rather broadly, the features of the disclosed subject matter in order that the detailed description that follows may be better understood, and in order that the present contribution to the art may be better appreciated. There are, of course, additional features of the disclosed subject matter that will be described below and that will form the subject matter of the claims appended hereto. It is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.

[0014] In the drawings, each identical or nearly identical component shown in various figures is represented by the same reference character. For clarity, not all components may be shown in every figure. The drawings are not necessarily drawn to scale, with emphasis instead being placed on illustrating various aspects of the techniques and devices described herein. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 illustrates an exemplary machine vision system according to some embodiments. [Figure 2] 1 is a flow chart of an exemplary computerized method for determining an estimated volume of an object captured by a 3D point cloud, according to some embodiments. [Figure 3] FIG. 1 illustrates an example of a 3D point cloud according to some embodiments. [Figure 4A] FIG. 4 is a diagram of the 3D point cloud and reference plane of FIG. 3 with a 2D grid of bins, according to some embodiments. [Figure 4B] FIG. 4 is a diagram of the 3D point cloud and reference plane of FIG. 3 with a 2D grid of bins, according to some embodiments. [Figure 4C]FIG. 4 is a diagram of the 3D point cloud and reference plane of FIG. 3 with a 2D grid of bins, according to some embodiments. [Figure 5] FIG. 4C illustrates the bin from FIGS. 4A-4B with points on a first side of a reference plane, according to some embodiments. [Figure 6] FIG. 4C illustrates a second bin from FIGS. 4A-4B having points on a second side of the reference plane opposite the first side, according to some embodiments. [Figure 7] 1A-1C illustrate exemplary curved reference patterns with objects thereon, according to some embodiments. [Figure 8] 1A-1C illustrate example surfaces of a portion of a reference pattern for a golden part, according to some embodiments. [Figure 9] 9 illustrates the surface of the reference pattern from FIG. 8 being compared with the first detection surface of the first runtime portion, according to some embodiments. [Figure 10] 9 illustrates the surface of the reference pattern from FIG. 8 being compared to a second detected surface of a second runtime object, according to some embodiments. [Figure 11] FIG. 10 illustrates a reference plane having a normal vector and bins arranged parallel to the normal vector, according to some embodiments. [Figure 12] FIG. 10 illustrates pattern directions and bins for a reference pattern, according to some embodiments, where the bins start at the surface of the reference pattern and are generated parallel to the pattern reference direction. [Figure 13] FIG. 10 illustrates an example of using a reference cylinder, according to some embodiments. [Figure 14] FIG. 10 illustrates an exemplary rectangular region for the surface of a reference cylinder, according to some embodiments. [Figure 15] FIG. 1 illustrates exemplary bins in cylindrical coordinates and corresponding bins in Cartesian coordinates, according to some embodiments. [Figure 16] FIG. 1 illustrates an example concave bin in cylindrical coordinates and a corresponding bin in Cartesian coordinates, according to some embodiments. [Figure 17] FIG. 1 illustrates an exemplary hemisphere in Cartesian coordinates and a corresponding hemisphere in spherical coordinates, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0016] The techniques described herein provide a rapid approximation of the volume of objects captured within a 3D point cloud image. The inventors recognize that conventional machine vision techniques can suffer from significant inefficiencies when using conventional 3D point cloud data. 3D point clouds often contain hundreds of thousands or even millions of points. Accordingly, the inventors recognize that attempting to directly calculate aspects of a 3D point cloud image, such as calculating volume, using such a large number of 3D points in space can be incredibly complex and time-consuming. For example, because 3D point clouds contain such a large number of 3D points and typically do not contain information about the spatial relationships between the 3D points (e.g., therefore, 3D point clouds themselves do not have volume), attempting to calculate volume from a pure 3D point cloud may be infeasible in many machine vision applications where performing such an interpretation may have time constraints, hardware resource limitations, etc. Additionally or alternatively, 3D point clouds often contain only a partial view of a scene with some visible surfaces (e.g., a top view), while other surfaces are partially and / or entirely missing (e.g., hidden side walls, missing bottom walls, etc.) Because some surfaces are missing, conventional techniques may not be able to determine the volume of objects in such 3D point clouds.

[0017] The inventors have developed technical improvements to machine vision techniques to address these and other inefficiencies. The techniques described herein can rapidly calculate volume directly from a point cloud (e.g., a point cloud generated directly by a 3D sensor). Some embodiments can use a reference shape, such as a ground plane, to calculate volume relative to the reference shape. The techniques can utilize a grid of 2D bins positioned relative to the reference shape used to approximate volume within the 3D point cloud. The techniques described herein can significantly increase the speed at which volume can be calculated for a 3D point cloud compared to conventional techniques. For example, by making several assumptions about the context in which the reference object is used, the techniques can calculate approximate volumes that are accurate enough to meet the needs of machine vision applications. Approximate volumes calculated according to the techniques described herein can be used in place of other complex volume calculations in various machine vision applications, including object detection, analysis, inspection, and the like. Furthermore, by using a reference shape, the reference shape can be used to calculate volume when one or more surfaces of an object otherwise captured in the 3D point cloud are missing.

[0018] In the following description, numerous specific details are set forth regarding the systems and methods of the disclosed subject matter, as well as the environments in which such systems and methods may operate, in order to provide a thorough understanding of the disclosed subject matter. Furthermore, it will be understood that the examples provided below are exemplary, and that other systems and methods are contemplated that are within the scope of the disclosed subject matter.

[0019] FIG. 1 illustrates an exemplary machine vision system 100 in accordance with some embodiments. The exemplary machine vision system 100 includes a camera 102 (or other image acquisition device) and a computer 104. The computer 104 includes one or more processors, a human-machine interface in the form of a computer display, and optionally one or more input devices (e.g., a keyboard, a mouse, a trackball, etc.). The camera 102 includes, among other components, a lens 106 and a camera sensor element (not shown). The lens 106 includes a field of view 108, which focuses light from the field of view 108 onto the sensor element. The sensor element generates a digital image of the camera field of view 108 and provides this image to a processor that forms part of the computer 104. As shown in the example of FIG. 1, an object 112 travels along a conveyor 110 into the field of view 108 of the camera 102. The camera 102 may generate one or more digital images of the object 112 while the object 112 is within the field of view 108 for processing, as described further herein. During operation, the conveyor may include multiple objects. These objects may pass within the field of view 108 of the camera 102, such as during an inspection process. Thus, the camera 102 may capture at least one image of each observed object 112.

[0020] In some embodiments, the camera 102 is a three-dimensional (3D) imaging device. As an example, the camera 102 may be a 3D sensor that scans a scene line-by-line, such as the DS line of a Laser Profiler 3D Displacement Sensor available from Cognex Corporation, the assignee of the present application. According to some embodiments, the 3D imaging device may generate an (x, y, z) point set (e.g., where the z axis adds a third dimension, such as distance from the 3D imaging device). The 3D imaging device may use various 3D image generation techniques, such as shape-from-shading, stereoscopic imaging, time-of-flight techniques, projector-based techniques, and / or other 3D generation techniques. In some embodiments, the machine vision system 100 includes a two-dimensional (2D) imaging device, such as a 2D CCD or CMOS imaging array. In some embodiments, the 2D imaging device generates a 2D array of intensity values. In some embodiments, the machine vision system processes the 2D data, such as by generating a 2D gradient field image. A gradient field image may include, for example, a set of cells related to magnitude and direction. For example, a gradient field may include Cartesian components of a vector (x,y) that may imply magnitude and direction, a gradient field may store actual values ​​(r,theta), etc.

[0021] In some embodiments, the machine vision system processes 3D data from the camera 102. The 3D data received from the camera 102 may include, for example, a point cloud and / or a range image. A point cloud may include a group of 3D points on or near the surface of a solid object. For example, the points may be presented in terms of their coordinates in a rectilinear or other coordinate system. In some embodiments, other information may optionally be present, such as a mesh or grid structure indicating which points are adjacent on the object's surface. In some embodiments, information about surface features, including curvature, surface normal, edge, and / or color, derived from sensor measurements or pre-calculated, as well as albedo information, may be included in the input point cloud. In some embodiments, the 2D and / or 3D data may be obtained from 2D and / or 3D sensors, CAD or other solid models, and / or by pre-processing range images, 2D images, and / or other images.

[0022] Examples of computer 104 may include, but are not limited to, a single server computer, a series of server computers, a single personal computer, a series of personal computers, a minicomputer, a mainframe computer, and / or a computing cloud. Various components of computer 104 may run one or more operating systems, and examples of one or more operating systems may include, but are not limited to, Microsoft Windows Server™, Novell Netware™, Redhat Linux™, Unix, and / or custom operating systems. One or more processors of computer 104 may be configured to process operations stored in memory coupled to the one or more processors. Memory may include, but is not limited to, hard disk drives, flash drives, tape drives, optical drives, RAID arrays, random access memory (RAM), and read-only memory (ROM).

[0023] Described herein are computerized techniques for quickly calculating an estimated or approximate volume of a shape captured by a 3D point cloud. According to some embodiments, a machine vision system selects a reference orientation and rotates a 3D image based on the reference orientation. For example, a reference orientation may be specified relative to a reference object (e.g., a direction perpendicular to a reference plane such as a conveyor belt), and the machine vision system may rotate the 3D image to align it with the reference orientation. The machine vision system constructs a 2D grid of bins based on the reference orientation. The machine vision system calculates the number of points in each bin and the average height of each bin based on these points. The machine vision system calculates the median number of points across all bins, normalizes the number of points in each bin using the determined median number of points, and determines a weight for each bin. The machine vision system estimates the volume as the union of the volumes of each bin, where the volume of each bin is calculated based on the weight and average height of the bin.

[0024] FIG. 2 is a flow chart of an exemplary computerized method for determining an estimated volume of an object captured by a three-dimensional (3D) point cloud, according to some embodiments. In step 202, a machine vision system (e.g., machine vision system 100 of FIG. 1) accesses (e.g., receives) a 3D point cloud having a plurality of 3D points. According to some embodiments, the 3D point cloud may be a voxel grid, a 3D lattice, or the like. FIG. 3 is a diagram illustrating an example 3D point cloud 300, according to some embodiments. For purposes of illustration, the 3D point cloud 300 shows only a small number of 3D points 302A, 302B, through 302N, collectively referred to as 3D points 302.

[0025] In step 204, the machine vision system receives a reference object, such as a reference plane, specified spatially relative to the 3D point cloud. Further referring to FIG. 3, FIG. 3 shows a schematic of a plane 304 representing the reference plane. It should be understood that the 3D point cloud 300 does not include information indicating the reference plane 304. Instead, the 3D point cloud only includes data for the 3D points 302 and may not even include (and typically does not include) data indicating the relationships between the 3D points 302 themselves. For example, the 3D point cloud 300 may simply include the (x, y, z) positions of each 3D point 302 within the 3D point cloud 300. The point cloud shown in FIG. 3 is used to provide examples of the techniques described herein. It should be understood, therefore, that the techniques are not intended to be limited by these examples.

[0026] According to some embodiments, the 3D point cloud may be presented in terms of coordinates, in terms of a line or other coordinate system relative to the 3D point cloud. In some embodiments, the technique may orient the point cloud based on a reference object (because, for example, the axes of the coordinate system may not be aligned with the direction(s) of the plane). Orienting the point cloud may include, for example, determining a reference direction of the reference plane (e.g., a vertical direction orthogonal to the reference plane, such as a conveyor belt) and aligning an axis (e.g., an X, Y, or Z axis) of the 3D point cloud's coordinate system with the reference direction. For example, the machine vision system may map the Z axis of the coordinate system to the reference direction. According to some embodiments, the system may access and / or generate a transformation that may specify the mapping of the 3D point cloud coordinate system to the orientation of the reference plane. The machine vision system may determine and / or use this transformation to map the 3D points of the point cloud to the reference plane.

[0027] According to some embodiments, a machine vision system determines aspects of the bins, such as the size and / or number of bins, to meet the requirements of a particular vision application. The number of bins and the bin size may be selected (e.g., by the machine vision system) based on various factors, such as the accuracy and / or speed requirements of the vision application. According to some embodiments, a larger bin size may enable the technique to run more quickly while providing a coarser volume estimate. With a larger bin size, more 3D points are assigned to each bin, and fewer total bins are required to cover the entire point cloud. Conversely, for example, if the bin size is set to a (significantly) small value, more bins are required to cover the same point cloud, and fewer points are assigned to each individual bin. Increasing the total number of bins may increase the total computation time, but the computed volume is typically finer and more accurate when using larger bins. The technique may control the speed and / or accuracy of volume estimation by varying the size of the bins so that more (or fewer) 3D points are assigned to each bin and a smaller (or larger) total number of bins is required to cover the entire point cloud.

[0028] For example, assume that the basis of each bin is a square of size 1 (e.g., the size is configured based on the coordinate system of the reference plane, as described further below). The technique can control the speed and / or accuracy of volume estimation by scaling the bin size so that more (or fewer) points are in each bin. For example, if the bin size is increased (e.g., using squares of size 2 or 3) so that a large number of points are in each bin, fewer bins are used, allowing the technique to run more quickly but resulting in a coarser volume. Conversely, if the bin size is reduced (e.g., using squares of size 0.5 or 0.25) so that fewer points are in each bin, more bins are used, which may slow down the calculation speed but result in a more accurate volume.

[0029] In step 206, the machine vision system constructs a 2D grid of bins along the reference plane. Each bin has a length and a width extending along the reference plane. The machine vision system may construct each bin of the 2D grid with the same length and width (which may be equal, as in the case of a square). According to some embodiments, the scale and / or size of the bins may be selected or specified, such as through predetermined configuration data. For example, the machine vision systems described herein may access data indicating a desired resolution of the estimated volume and determine the bin size (e.g., the bin's length and width) based on the desired resolution. FIGS. 4A-4B illustrate the 3D point cloud 300 and reference plane 304 of FIG. 3 and a 2D grid of bins, according to some embodiments. In this example, the 2D grid of bins includes bins 402A, 402B, through 402H, collectively referred to herein as bins 402. Each bin has the same length and width (which may be equal), shown in one example as length 404 and width 406 of bin 402E. 4B includes dotted lines extending from the bins 402 of the 2D grid to illustrate the three-dimensionality of the bins relative to 3D points above and below the reference plane 304. It should be understood that these dotted lines are included for illustrative purposes only and are not intended to indicate bin heights, which may be determined as described further herein. For example, FIG. 4C illustrates the approximate heights of the bins, including the heights of bins 402A, 402B, 402C, 402F, 402G, and 402H (the heights / bins of bins 402D and 402E are not shown because neither of these bins includes 3D points).

[0030] In step 208, the machine vision system determines, for each bin in the 2D grid, the number of 3D points in the bin. The machine vision system also determines a height for each bin from the reference object based on the 3D points in the bin. As described herein, the 3D point cloud may be mapped to a reference plane. The height of each bin may be determined based on the mapping of the point cloud to the reference plane. For example, if the Z axis of the 3D point cloud coordinate system is mapped or oriented to a reference direction orthogonal to the reference plane, the machine vision system may determine a height for each bin based on the Z axis values ​​of the 3D points in the bin. For example, the machine vision system may calculate an average Z value for each 3D point to determine and average the height values ​​of the bins. Figure 5 is a diagram illustrating bin 402F from Figure 4, in accordance with some embodiments. Using Figure 5 as an example to illustrate how the machine vision system processes bins, the machine vision system determines that bin 402F contains 21 3D points. The machine vision system also determines the height of the bins based on the 21 3D points within these bins, indicated using line 502 (e.g., being the average height of the 3D points within the bin), whereby the height of bin 402F is indicated as height 504. Some embodiments calculate the height by calculating the median of the heights of the 3D points, but it should be appreciated that other techniques may be used to calculate the height. For example, the height may be calculated based on the average of the determined weights, the height of one or more 3D points with the greatest height, the height of one or more 3D points with the least height, etc.

[0031] According to some embodiments, when determining the number of 3D points in each bin, the machine vision system may ignore some 3D points in the bin. For example, the machine vision system may ignore 3D points within a threshold distance from a reference plane (e.g., to avoid incorporating 3D points captured on the surface of the reference plane, if present in the scene). For example, the system may ignore points near the surface of the reference surface because such points are more likely to belong to the (e.g., physical) reference plane (e.g., a conveyor belt) rather than the object itself. In some instances, due to error / measurement noise, these points may not lie perfectly on the reference plane. Instead, the points may be slightly offset from the reference plane. Because the height of the object may be expected to be (significantly) larger than the measurement noise, a threshold may be used to separate points belonging to the reference shape from points belonging to the object. According to some embodiments, the threshold may be selected based on the expected measurement noise of the 3D camera, the range of heights of the objects being measured, etc.

[0032] According to some embodiments, the 3D points of the 3D point cloud may be located on both sides of a reference plane. For example, as shown in FIG. 3, 3D points 302A and 302B are located on a first side of the reference plane 304 (e.g., above the reference plane 304), and point 302N is located on a second side of the reference plane 304 (e.g., below the reference plane 304). For each bin, the technique may include determining a first number of 3D points on the first side of the reference plane (and, e.g., associated heights from the first side of the reference plane) and / or determining a second number of 3D points on the second side of the reference plane (and, e.g., associated heights from the second side of the reference plane). FIG. 6 is a diagram illustrating bin 402H from FIG. 4 in accordance with some embodiments, where the 3D points are on the opposite side from the 3D points of bin 402F in FIG. 5. Using FIG. 6 as an example to illustrate how a machine vision system processes 3D points in a bin on the other side of the reference plane 304, the machine vision system determines that bin 402H contains seven 3D points. The machine vision system also determines the height of the bin from the other side of the reference plane based on the 3D points within the bin, shown using line 602 (e.g., determined as the mid-height of the seven 3D points within the bin), whereby the height of bin 402H is shown as height 604. The machine vision system may designate points above the reference plane as convex points and points below the reference plane as concave points. For example, when a 3D sensor observes an object from above, points below the reference plane may occur when the reference feature has holes, voids, and / or other surface depressions. In such an example, the technique may measure points below the reference plane to measure the volume of the surface depression. The reference feature in such an example may be the top surface of the object being measured, and various objects may or may not have surface depressions (e.g., due to defects) and / or may have surface depressions of different sizes. As another example, a machine vision system may use the technique to simply inspect that a hole(s) or void has some nominal volume.

[0033] In step 210, the machine vision system calculates an estimated volume of the object captured by the 3D point cloud based on the determined number of 3D points in each bin and the height of each bin. According to some embodiments, the machine vision system may calculate the mean or average number of 3D points in the bins of the 2D grid based on the 3D points in each bin. The machine vision system may use the mean number of 3D points to determine the weight of each bin. For example, the machine vision system may determine a fill weight for each bin based on the number of 3D points in the bin and the mean number of 3D points. The weight of each bin may represent how fully occupied the bin is with 3D points. For example, if the number of 3D points in a bin is greater than the mean number of points, the machine vision system may assign the bin a weight indicating that the bin is full (e.g., a weight of 1.0). As another example, if the number of 3D points in a bin is half the mean number of points, the machine vision system may assign the bin a weight indicating that the bin is only half full (e.g., a weight of 0.50). As a further example, if the number of 3D points in a bin is three-quarters of the median number of points, the machine vision system may assign the bin a weight that indicates the bin is three-quarters full (e.g., a weight of 0.75).

[0034] The machine vision system may calculate the estimated volume of each bin by calculating the estimated volume of each bin and summing the volumes of the bins. For example, the machine vision system may calculate the estimated volume of each bin by calculating the product of (a) the fill weight of the bin, (b) the mid-height of the bin's 3D points, and (c) the common / shared area of ​​the bins in a 2D grid (e.g., the product of the length and width of the bins in the 2D grid). The machine vision system may sum the estimated volumes of each bin to calculate the estimated volume of the object.

[0035] As described herein, there may be 3D points on the opposite side of the reference plane. The technique may include determining, for each bin, a first volume on one side of the reference plane (e.g., a convex volume of points above the reference plane) and a second volume on the other side of the reference plane (e.g., a concave volume of points below the reference plane). For example, the machine vision system may calculate the convex volume based on the number of 3D points in each bin above the reference plane and the determined height of each bin based on these 3D points and the first side of the reference plane. The machine vision system may use this information to determine an estimated convex volume for each bin, as described herein, and add the estimated convex volumes for each bin to determine an estimated total convex volume. The machine vision system may also calculate the concave volume based on the number of 3D points in each bin below the reference plane and the determined height of each bin from the opposite side of the reference plane relative to these 3D points. The machine vision system may similarly use this information to determine an estimated concave volume for each bin, add the estimated concave volumes for each bin, and determine an estimated total concave volume. According to some embodiments, the machine vision system may output both the first volume and the second volume, combine the first volume and the second volume, etc. to determine the total volume (e.g., sum, subtract, etc. to determine a single representation volume).

[0036] As described herein, an estimated volume of an object may be determined based on a reference, such as a reference plane. Various techniques may be used to calculate the volume of an object above and / or below a plane. As an illustrative example, according to some embodiments, the nominal volume of a particular bin i may be calculated using Equation 1: NominalBinVolume i =BinBaseArea i ×BinHeight i (Formula 1) can be calculated according to the formula: BinBaseArea i is the fundamental area of ​​bin i relative to the reference plane, BinHeight i is the height of bin i.

[0037] The weight-determined volume of bin i is calculated using Equation 2: V i =Weight i ×NominalBinVolume i (Formula 2) can be calculated according to the formula: Weight i is the determined weight of bin i, NominalBinVolume i is the nominal volume of bin i.

[0038] The total volume of the object is given by Equation 3:

[0039]

number

[0040] The volume can be determined based on the individual weights determined according to the formula:

[0041] While a reference plane is used as an example to illustrate some aspects of the techniques described herein, it should be understood that other references may be used to calculate an estimated volume of a shape. For example, according to some embodiments, a reference pattern may be used to measure the volume of an object located on a non-flat surface. The reference pattern may be used, for example, to measure the volume above a curved surface (e.g., a curved tray). Figure 7 illustrates an exemplary curved reference pattern 700 with objects 702, 704, and 706 thereon, according to some embodiments. The curved reference pattern 700 may be used with the techniques described herein to estimate the volume of objects 702, 704, and / or 706.

[0042] Another exemplary use of a reference pattern is in inspection applications where a newly manufactured part is compared against a “golden part” or template part (e.g., to detect defects). In such an embodiment, the surface of the golden part is the reference pattern and may be used to inspect the runtime part. The volume difference may be either “convex” or “concave,” depending on whether the runtime part has more or less material compared to the golden part, respectively. If the runtime part exactly matches the golden part, the calculated volume may be zero and / or negligible (e.g., indicating no difference between the runtime and golden parts). In inspection applications, the volume measurements of each runtime part may be compared to preset tolerances to determine whether the part passes inspection. For example, any detected convex or concave volume that exceeds the application's set tolerance may be flagged as a possible defect.

[0043] FIG. 8 is a diagram illustrating an example surface 800 of a portion of a reference pattern for a golden part, according to some embodiments. FIG. 9 is a diagram illustrating surface 800 of the reference pattern from FIG. 8 being compared to a first detected surface 900 of a first run-time part, according to some embodiments. Techniques described herein may be used to calculate the difference between surface 800 and surface 900. In this example, the difference includes concave volume 902 and convex volumes 904 and 906. In this example, surface 900 of the first run-time part is similar to, but not exactly equal to, surface 800 of the golden part. Because such convex and concave volumes are minor deviations, the system may not classify such deviations as possibly including defects. It should be appreciated that while surface 800 of the reference pattern in FIGS. 8-9 is a two-dimensional pattern, a three-dimensional pattern may be used as a reference object, reflecting the three-dimensional aspects of the reference object.

[0044] FIG. 10 is a diagram illustrating surface 800 of the reference pattern from FIG. 8 being compared to a second detected surface 1000 of a second run-time object, according to some embodiments. Techniques described herein may be used to calculate the difference between surface 800 and surface 1000. In this example, the difference includes a convex volume 1002 and a concave volume 1004. As shown in FIG. 10 , both convex volume 1002 and concave volume 1004 include non-negligible deviations. Therefore, in inspection applications, the volumes of these regions may be compared to a tolerance threshold to flag portions of the part and / or the entire part as potentially defective. As described herein, inspection applications may provide local convex and / or concave estimated volumes, global convex and / or concave estimated volumes, combined representations of the overall volume (e.g., by summing and / or subtracting convex and concave estimated volumes), and the like. In some embodiments, a graphical display may be provided (e.g., as shown in FIG. 10 ) to visualize the convex and / or concave volumes relative to the reference object.

[0045] According to some embodiments, volume calculations for a reference pattern may be performed using similar techniques described herein for calculating volume for a reference plane. When working with a plane, the technique may use the plane's normal vector, rotate the points, and generate bins parallel to the vector direction. For example, as shown in FIG. 11 , reference plane 1100 has normal vector 1102, and bins (including bin 1106) are positioned parallel to the normal vector. When working with a reference pattern, data may be provided to indicate a pattern direction, and the technique may generate bins parallel to the pattern direction. For example, as shown in FIG. 12 , pattern direction 1202 may be provided for reference pattern 1200, and bins (including bin 1206) are created starting at the surface of reference pattern 1200 and parallel to pattern reference direction 1202.

[0046] According to some embodiments, in the case of a planar reference, the bottoms of the bins may coincide with this plane. This may not be the case when working with a reference pattern. For example, in the case of a reference pattern, the bottoms of some and / or all of the bins may be created perpendicular to the reference pattern direction, and the bin positions may be determined by the mid-height of the pattern at the bin positions.

[0047] According to some embodiments, the bin size may be selected based on the shape of the reference pattern. For example, if the pattern has high curvature and / or many sharp features, small bins may be used to capture the details. In some embodiments, different sized bins may be used for different portion(s) of the reference pattern. For example, for portions with high curvature and / or sharp features, small bins may be used to capture the details of these features, while larger bins may be used for portions with low curvature and / or gentle features. As another example, if the pattern is better approximated by a piecewise flat surface, larger bin sizes may be used rather than smaller bin sizes. According to some embodiments, calculations for calculating the height of each bin, the weight of each bin, the volume of each bin, and / or the total volume described in conjunction with the reference plane may be used for the reference pattern.

[0048] According to some embodiments, if a bin starts at a pattern and extends toward a point above the pattern, the bin is considered "convex" and contributes to the total convex volume (e.g., determined based on the up direction of the reference pattern). If a bin starts at a pattern and extends toward a point below the pattern, the bin may be considered "concave" (e.g., also determined based on the reference direction of the reference pattern). The "up" direction may be the pattern reference direction.

[0049] According to some embodiments, a cylinder may be used as a reference for volume calculations. The cylinder may be used in vision applications seeking to identify ridges or depressions on cylindrical objects, such as cylindrical batteries, tubes, etc. Thus, the techniques described herein may be used to calculate the volume of an object under inspection relative to a reference cylinder. FIG. 13 shows an example of using a reference cylinder 1300, according to some embodiments. As shown, a ridge defect may result in a convex volume 1302, while a depression may result in a concave volume 1304.

[0050] According to some embodiments, various techniques may be used to perform volume calculations. For example, volume may be calculated relative to a reference cylinder by converting a point from Cartesian coordinates (x, y, z) to cylindrical coordinates (ρ, φ, z), where ρ is the radial coordinate, φ is the azimuthal angle, and z is the height of the cylinder. The relationship between Cartesian and cylindrical coordinates is given by Equation 4:

[0051]

number

[0052] may be provided by

[0053] The relationship shown in Equation 4 assumes that the reference cylinder has an axis aligned with the Cartesian z-axis. If this is not the case, one can first apply a transformation to align the reference cylinder axis with the z-axis. In the space of cylindrical coordinates, the surface of the reference cylinder is a rectangular region. FIG. 14 is a diagram showing an example rectangular region 1400 for the surface of a reference cylinder 1450, according to some embodiments. In this example, the surface of the reference cylinder in cylindrical coordinates lies on the plane ρ=r. According to some embodiments, this technique can be used to calculate volume from the reference plane, generate square elementary bins, assign points to bins, and calculate bin weights and heights.

[0054] The volume of an individual bin may be calculated according to various techniques. Figure 15 is a diagram illustrating an example bin 1500 in cylindrical coordinates and a corresponding bin 1550 in Cartesian coordinates, according to some embodiments. For example, the nominal volume of a convex bin i is calculated according to Equation 5:

number

[0055] Since bin i extends beyond the outside of the reference cylinder, the nominal volume achieved by Equation 5 can be called the nominal volume of a "convex bin."

[0056] 16 is a diagram illustrating an example concave bin 1600 in cylindrical coordinates and a corresponding bin 1650 in Cartesian coordinates, according to some embodiments. For example, the nominal volume of concave bin i is given by Equation 6:

number

[0057] Since bin i extends toward the inside of the reference cylinder, the nominal volume in Equation 6 can be called the nominal volume of a "concave bin."

[0058] According to some examples, in some vision applications, it may be desirable to measure volume changes relative to a spherical object. One illustrative example is the inspection of dimples on a golf ball. According to some embodiments, the technique may use a reference sphere to calculate the volume of an object. According to some embodiments, a technique similar to that described herein may be used for a reference cylinder, but rather than using cylindrical coordinates, the technique may use spherical coordinates. For example, the technique may use the volume calculation algorithm described herein adapted for use with a reference sphere (e.g., rather than a plane or cylinder). According to some embodiments, the technique may transform points to spherical coordinates and apply a volume calculation method in this transformed space. According to some embodiments, the technique may convert the results back to the original Cartesian coordinate space. In some embodiments, bins of equal area in the space of spherical coordinates may not map to equal-area sections on the surface of the reference sphere in Cartesian coordinates. To address this, the technique may be configured to consider the location of each bin relative to the original sphere (e.g., whether it is near a pole or an equator) when converting the volume calculated for each bin back to the original Cartesian coordinate space.

[0059] According to some embodiments, the relationship between Cartesian coordinates (x, y, z) and spherical coordinates (ρ, θ, φ) is expressed as follows in Equation 7:

[0060]

number

[0061] where ρ is the radial coordinate, θ is the tilt, and φ is the azimuthal angle of the sphere.

[0062] 17 is a diagram illustrating an example hemisphere 1700 in Cartesian coordinates (x, y, z) and a corresponding hemisphere 1750 in spherical coordinates (ρ, θ, φ), according to some embodiments. In this example, the spherical coordinates have θ varying from 0 to π / 2, as an example. Other examples may vary differently. For example, for a full sphere, θ varies from 0 to π, and φ varies from 0 to 2π.

[0063] Techniques operating according to the principles described herein may be implemented in any suitable manner. The processing and decision blocks in the flow charts above represent steps and operations that may be included within algorithms that perform these various operations. The algorithms derived from these operations may be implemented as software integrated with one or more special-purpose or general-purpose processors to direct these operations, as functionally equivalent circuitry such as digital signal processing (DSP) circuitry or application-specific integrated circuits (ASICs), or in any other suitable manner. It should be understood that the flow charts contained herein do not represent any particular circuitry, the syntax or operations of any particular programming language, or type of programming language. Instead, the flow charts represent functional information that one skilled in the art may use to fabricate circuits or implement computer software algorithms to perform the operations of a particular device that performs the techniques of the types described herein. Unless otherwise specified herein, it should also be understood that the specific sequence of steps and / or operations depicted in each flow chart is merely indicative of an algorithm, and that these algorithms may be implemented and modified within implementations and embodiments of the principles described herein.

[0064] Thus, in some embodiments, the techniques described herein may be embodied in computer-executable instructions implemented in software, including application software, system software, firmware, middleware, embedded code, or any other suitable type of computer code. Such computer-executable instructions may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0065] When the techniques described herein are embodied as computer-executable instructions, these computer-executable instructions can be implemented in any suitable manner, including several functional means, each of which performs one or more operations to complete the execution of an algorithm operating in accordance with these techniques. However, the illustrated "functional means" are structural components of a computer system that, when integrated with and executed by one or more computers, cause the one or more computers to perform a particular operational role. A functional means may be part or all of a software element. For example, a functional means may be implemented as a function of a process, as a separate process, or as any other suitable processing unit. When the techniques described herein are implemented as multiple functional means, each functional means may be implemented in a unique manner. Not all functional means need be implemented in the same way. Furthermore, these functional means may execute in parallel and / or serially, as appropriate, and communicate information between each other using shared memory on the computer(s) executing the functional means, using a message-passing protocol, or in any other suitable manner.

[0066] Generally, functional means include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the functional means may be combined or distributed as desired within the systems in which the functional means operate. In some implementations, one or more functional means that perform the techniques herein may together form a complete software package. These functional means may, in alternative embodiments, be adapted to cooperate with other unrelated functional means and / or processes to implement a software program application.

[0067] Several exemplary functional means for performing one or more tasks have been described herein. It should be understood that the functional means and portions of the described tasks are merely indicative of types of functional means that may implement the exemplary techniques described herein, and that embodiments are not limited to being implemented with any particular number, portion, or type of functional means. In some implementations, all functions may be implemented within a single functional means. It should also be understood that in some implementations, some of the functional means described herein may be implemented together with others (i.e., as a single unit) or separately from others (or as separate units), and that some of these functional means may not be implemented.

[0068] Computer-executable instructions implementing the techniques described herein (whether implemented as one or more functional means or in any other manner) may, in some embodiments, be encoded on one or more computer-readable media, providing functionality to the media. Computer-readable media include magnetic media such as hard disk drives, optical media such as compact discs (CDs) or digital versatile discs (DVDs), permanent or non-permanent solid-state memory (e.g., flash memory, magnetic RAM, etc.), or any other suitable storage media. Such computer-readable media may be implemented in any suitable manner. As used herein, a "computer-readable medium" (also referred to as a "computer-readable storage medium") refers to a tangible storage medium. A tangible storage medium is non-transitory and has at least one physical structural component. As used herein, a "computer-readable medium" refers to at least one physical structural component that has at least one physical characteristic that can be altered in some way during the process of generating a medium having embedded information, recording information onto the medium, or any other process of encoding the medium with information. For example, the magnetization state of portions of the physical structure of the computer-readable medium may be altered during the recording process.

[0069] Furthermore, some of the techniques described above involve categorizing information (e.g., data and / or instructions) in a particular manner for use by those techniques. In some implementations of these techniques, such as those in which the techniques are implemented as computer-executable instructions, the information may be encoded on a computer-readable storage medium. Where particular structures are described herein as advantageous formats for storing this information, such structures may be used to impart a physical information organization when encoded on the storage medium. In this case, these advantageous structures may provide functionality to the storage medium by affecting the operation of one or more processors associated with the information, e.g., by increasing the efficiency of computer operations performed by the processor(s).

[0070] In some, but not all, implementations in which the present techniques may be embodied as computer-executable instructions, these instructions may be executed on one or more suitable computing device(s) operating within any suitable computer system, or one or more computing devices (or one or more processors of one or more computing devices) may be programmed to execute the computer-executable instructions. A computing device or processor may be programmed to execute the instructions if the instructions are stored in a manner accessible to the computing device or processor, such as in a data store (e.g., an on-chip cache or instruction register, a computer-readable storage medium accessible via a bus, a computer-readable storage medium accessible via one or more networks and accessible to the device / processor, etc.). The functional means containing these computer-executable instructions may be integrated with or manage the operation of a single general-purpose programmable digital computing device, the coordinate system of two or more general-purpose computing devices that share processing power and together perform the techniques described herein, the coordinate system of a single computing device or computing devices (co-located or geographically distributed) dedicated to performing the techniques described herein, one or more field programmable gate arrays (FPGAs) that perform the techniques described herein, or any other suitable system.

[0071] A computing device may include at least one processor, a network adapter, and a computer-readable storage medium. The computing device may be, for example, a desktop or laptop personal computer, a personal digital assistant (PDA), a smart mobile phone, a server, or any other suitable computing device. The network adapter may be any suitable hardware and / or software that enables the computing device to communicate, wired and / or wirelessly, with any other suitable computing device over any suitable computing network. A computing network may include wireless access points, switches, routers, gateways, and / or other networking equipment for exchanging data between two or more computers, including the Internet, as well as any suitable wired and / or wireless communication medium(s). The computer-readable medium may be adapted to store data to be processed and / or instructions to be executed by the processor. The processor enables the processing of data and the execution of instructions. The data and instructions may be stored on the computer-readable storage medium.

[0072] A computing device may further have one or more components and peripherals, including input / output devices. These devices may be used, among other things, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen, which presents output visually, and a speaker or other sound-generating device, which presents output audibly. Examples of input devices that may be used for a user interface include keyboards and pointing devices, such as mice, touch pads, and digitizing tablets. As another example, a computing device may receive input information through voice recognition or in other audible forms.

[0073] Embodiments have been described in which techniques are implemented within circuits and / or computer-executable instructions. It should be understood that some embodiments may be in the form of a method, of which at least one example is provided. Operations performed as part of a method may be reordered in any suitable manner. Thus, embodiments may be constructed in which operations are performed in an order other than the illustrated order, and may include performing some operations simultaneously despite being shown as sequential operations in the exemplary embodiments.

[0074] The various aspects of the above-described embodiments may be used alone, in combination, or in various configurations not specifically described in the above embodiments, and therefore the application is not limited to the details and arrangements of components shown or illustrated in the above description. For example, aspects described in one embodiment may be combined in any way with aspects described in other embodiments.

[0075] The use of ordinal numbers such as "first," "second," "third," etc. in the claims to modify claimed elements does not, in itself, imply any priority, precedence, or order of a claimed element relative to other claimed elements, or the chronological order in which method operations are performed, but is merely used as a label to distinguish a claimed element having a particular name from another element having the same name (even though an ordinal number is used), thereby distinguishing the claimed elements.

[0076] Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," "having," "containing," "involving," and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof, as well as additional items.

[0077] The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Accordingly, any embodiments, implementations, steps, features, etc. described herein as exemplary are to be understood as illustrative examples and not as preferred or advantageous examples, unless expressly specified otherwise.

[0078] Having thus described several aspects of at least one embodiment, it should be understood that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.

[0079] Various aspects are described in this application, including but not limited to the following.

[0080] 1. A computerized method for determining an estimated volume of an object captured by a three-dimensional (3D) point cloud, the method comprising: receiving data, the data comprising: a 3D point cloud containing multiple 3D points, and A reference plane spatially related to the 3D point cloud and configuring a 2D grid of bins along the reference plane, each bin of the 2D grid including a length and a width extending along the reference plane; for each bin of the 2D grid, determining the number of 3D points in the bin and the height of the bin above the reference plane; calculating an estimated volume of an object captured by the 3D point cloud based on the calculated number of 3D points in each bin and the height of each bin; A method comprising: 2. Determining the number of 3D points and height of each bin of the 2D grid comprises, for each bin of the 2D grid: a first number of 3D points within the bin on a first side of the reference plane; and a first height of the bin from the first side determining 1. The method of claim 1, wherein calculating the estimated volume of the object includes calculating a first estimated volume of the object. 3. For each bin of the 2D grid, determining a second number of 3D points within the bin on a second side of the reference plane opposite the first side, and a second height of the bin from the second side; calculating a second estimated volume of the object; The method according to 2., further comprising: 4. The method of claim 3, wherein calculating the estimated volume includes summing the first estimated volume and the second estimated volume. 5. The method of any one of 1. to 4., wherein determining the number of 3D points for each bin includes ignoring 3D points of the plurality of 3D points within a threshold distance of the reference plane. 6. Calculating an estimated volume of the object based on the determined number of 3D points in each bin and the height of each bin, calculating a median number of 3D points in a bin of said 2D grid based on the 3D points in each bin; The method according to any one of 1. to 5., 7. For each bin of the 2D grid: determining a fill weight of the bin based on the number of 3D points in the bin and the median number of 3D points in the bin; determining the mean height of 3D points within the bin from the reference plane; calculating the estimated volume based on the fill weight of each bin and the mid-height of each bin; 6. The method according to claim 6, further comprising: 8. Calculating the estimated volume comprises: Calculating an estimated volume for each bin by calculating the product of (a) the fill weight of the bin, (b) the mid-height of the bin's 3D points, and (c) the common area of ​​the bins in the 2D grid; adding the estimated volumes of each bin to calculate an estimated volume of the object; 7. The method according to claim 7, comprising: 9. The method according to any one of 1. to 8., further comprising aligning the orientation of the point cloud based on the reference plane. 10. Orienting the point cloud based on the reference plane includes: determining a reference direction of the reference plane; aligning axes of a coordinate system of the 3D point cloud with the reference direction; 9. The method according to claim 9, comprising: 11. Aligning the axes of the coordinate system of the 3D point cloud with the reference direction of the reference plane includes: determining a transformation that maps the axis to the reference direction; using said transformation to map 3D points of said point cloud onto said reference plane; 10. The method according to claim 10, comprising: 12. The axis is the Z axis; 12. The method of claim 11, wherein the method further comprises, for each bin, determining a height of the bin based on a Z-axis value for each of the 3D points within the bin. 13. receiving data indicating a desired resolution of the estimation volume; determining a length and width of a bin of the 2D grid based on the desired resolution; The method according to any one of 1. to 12., further comprising: 14. A non-transitory computer-readable medium comprising instructions operable, when executed by one or more processors on a computing device, to cause the one or more processors to determine an estimated volume of an object captured by a three-dimensional (3D) point cloud, the instructions comprising: receiving data, the data comprising: a 3D point cloud containing multiple 3D points, and A reference plane spatially related to the 3D point cloud indicating, receiving, constructing a 2D grid of bins along the reference plane, each bin of the 2D grid including a length and a width extending along the reference plane; for each bin of the 2D grid, determining the number of 3D points within the bin and the height of the bin above the reference plane; and calculating an estimated volume of an object captured by the 3D point cloud based on the calculated number of 3D points in each bin and the height of each bin; 1. A non-transitory computer-readable medium containing an implementation of 15. Determining the number of 3D points and height of each bin of the 2D grid includes, for each bin of the 2D grid: a first number of 3D points within the bin on a first side of the reference plane; and a first height of the bin from the first side determining 14. The non-transitory computer-readable medium of claim 14, wherein calculating an estimated volume of the object includes calculating a first estimated volume of the object. 16. A non-transitory computer-readable medium as described in 14. to 15., wherein determining the number of 3D points for each bin includes ignoring 3D points of the plurality of 3D points within a threshold distance of the reference plane. 17. Calculating an estimated volume of the object based on the determined number of 3D points in each bin and the height of each bin includes: calculating a median number of 3D points in a bin of said 2D grid based on the 3D points in each bin; The non-transitory computer-readable medium according to any one of items 14 to 16, including: 18. A non-transitory computer-readable medium as described in any one of claims 14 to 17, wherein the instructions are further configured to cause the one or more processors to orient the point cloud based on the reference plane. 19. The instructions cause the one or more processors to: receiving data indicative of a desired resolution of the estimation volume; and determining the length and width of the bins of the 2D grid based on the desired resolution; The non-transitory computer-readable medium according to any one of claims 14 to 18, further configured to perform the following: 20. A system comprising: a memory storing instructions; and at least one processor configured to execute the instructions for determining an estimated volume of an object captured by a three-dimensional (3D) point cloud, the instructions comprising: receiving data, the data comprising: a 3D point cloud containing multiple 3D points, and A reference plane spatially related to the 3D point cloud indicating, receiving, constructing a 2D grid of bins along the reference plane, each bin of the 2D grid including a length and a width extending along the reference plane; for each bin of the 2D grid, determining the number of 3D points within the bin and the height of the bin above the reference plane; and calculating an estimated volume of an object captured by the 3D point cloud based on the calculated number of 3D points in each bin and the height of each bin; The system includes the implementation of

Claims

1. 1. A computerized method for determining an estimated volume of an object captured by a three-dimensional (3D) point cloud, comprising: The method includes receiving data, the data comprising: a 3D point cloud comprising a plurality of 3D points; and A reference plane spatially related to the 3D point cloud indicates; The method further includes constructing a 2D grid of bins along the reference plane, each bin of the 2D grid including a length and a width extending along the reference plane; The method further includes determining, for each bin of the 2D grid, a number of 3D points in the bin and heights of the 3D points in a reference direction from the reference plane, the reference direction being perpendicular to the reference plane; The method further includes determining a height of the bin in the reference direction from the reference plane based on a height of the 3D point; The method further includes calculating an estimated volume of an object captured by the 3D point cloud based on the determined number of 3D points in each bin and a height of each bin; The method.

2. Determining the number of 3D points and height of each bin of the 2D grid comprises, for each bin of the 2D grid: a first number of 3D points within the bin on a first side of the reference plane; and a first height of the bin from the first side determining The method of claim 1 , wherein calculating an estimated volume of the object comprises calculating a first estimated volume of the object.

3. For each bin of the 2D grid, determining a second number of 3D points within the bin on a second side of the reference plane opposite the first side, and a second height of the bin from the second side; calculating a second estimated volume of the object; The method of claim 2 further comprising:

4. The method of claim 3 , wherein calculating the estimated volume comprises summing the first estimated volume and the second estimated volume.

5. The method of claim 1 , wherein determining the number of 3D points for each bin comprises ignoring 3D points of the plurality of 3D points within a threshold distance of the reference plane.

6. calculating an estimated volume of the object based on the determined number of 3D points in each bin and the height of each bin, calculating an average number of 3D points per bin of the 2D grid based on the 3D points in each bin; The method of claim 1 , comprising:

7. For each bin of the 2D grid, determining a fill weight of the bin based on a ratio of the number of 3D points in the bin to the average number of 3D points per bin; determining the mean height of 3D points within the bin from the reference plane; calculating the estimated volume based on the fill weight of each bin and the mid-height of each bin; The method of claim 6 further comprising:

8. Calculating the estimated volume comprises: Calculating an estimated volume for each bin by calculating the product of (a) the fill weight of the bin, (b) the mid-height of the bin's 3D points, and (c) the common area of ​​the bins in the 2D grid; adding the estimated volumes of each bin to calculate an estimated volume of the object; The method of claim 7, comprising:

9. The method of claim 1 , further comprising orienting the point cloud based on the reference plane.

10. Orienting the point cloud based on the reference plane includes: determining a reference direction of the reference plane; aligning axes of a coordinate system of the 3D point cloud with the reference direction; 10. The method of claim 9, comprising:

11. aligning axes of the coordinate system of the 3D point cloud with reference directions of a reference plane; determining a transformation that maps the axis to the reference direction; using the transformation to map 3D points of the point cloud onto the reference plane; The method of claim 10, comprising:

12. the axis is the Z axis, The method of claim 11 , wherein the method further comprises, for each bin, determining a height of the bin based on a Z-axis value for each of the 3D points within the bin.

13. receiving data indicating a desired resolution of the estimation volume; determining a bin length and width of the 2D grid based on the desired resolution; The method of claim 1 further comprising:

14. 1. A non-transitory computer-readable medium comprising instructions operable, when executed by one or more processors on a computing device, to cause the one or more processors to determine an estimated volume of an object captured by a three-dimensional (3D) point cloud: The instructions include receiving data, the data comprising: a 3D point cloud comprising a plurality of 3D points; and A reference plane spatially related to the 3D point cloud indicates; The instructions further include configuring a 2D grid of bins along the reference plane, each bin of the 2D grid including a length and a width extending along the reference plane; The instructions further include determining, for each bin of the 2D grid, a number of 3D points in the bin and heights of the 3D points in a reference direction from the reference plane, the reference direction being perpendicular to the reference plane; The instructions further include determining a height of the bin in the reference direction from the reference plane based on a height of the 3D point; The instructions further include calculating an estimated volume of an object captured by the 3D point cloud based on the determined number of 3D points in each bin and a height of each bin; The non-transitory computer-readable medium.

15. Determining the number of 3D points and height of each bin of the 2D grid comprises, for each bin of the 2D grid: a first number of 3D points within the bin on a first side of the reference plane; and a first height of the bin from the first side determining The non-transitory computer-readable medium of claim 14 , wherein calculating an estimated volume of the object comprises calculating a first estimated volume of the object.

16. 15. The non-transitory computer-readable medium of claim 14, wherein determining the number of 3D points for each bin includes ignoring 3D points of the plurality of 3D points within a threshold distance of the reference plane.

17. calculating an estimated volume of the object based on the determined number of 3D points in each bin and the height of each bin, calculating an average number of 3D points per bin of the 2D grid based on the 3D points in each bin; 15. The non-transitory computer-readable medium of claim 14, comprising:

18. 15. The non-transitory computer-readable medium of claim 14, wherein the instructions are further configured to cause the one or more processors to orient the point cloud based on the reference plane.

19. The instructions may cause the one or more processors to: receiving data indicative of a desired resolution of the estimation volume; and determining the length and width of the bins of the 2D grid based on the desired resolution; 15. The non-transitory computer-readable medium of claim 14, further configured to perform:

20. 1. A system comprising: a memory storing instructions; and at least one processor configured to execute the instructions for determining an estimated volume of an object captured by a three-dimensional (3D) point cloud: The instructions include receiving data, the data comprising: a 3D point cloud comprising a plurality of 3D points; and A reference plane spatially related to the 3D point cloud indicates; The instructions further include configuring a 2D grid of bins along the reference plane, each bin of the 2D grid including a length and a width extending along the reference plane; The instructions further include determining, for each bin of the 2D grid, a number of 3D points in the bin and heights of the 3D points in a reference direction from the reference plane, the reference direction being perpendicular to the reference plane; The instructions further include determining a height of the bin in the reference direction from the reference plane based on a height of the 3D point; The instructions further include calculating an estimated volume of an object captured by the 3D point cloud based on the determined number of 3D points in each bin and a height of each bin; The system.

Images