Method and apparatus for processing image data for machine vision

Abstract

To provide a method, a system, and a storage medium for testing a posture of a model with respect to image data, such as testing the posture of the model in three-dimensional data.SOLUTION: A method of testing a pose of a model by a field to determine a score for the pose, the method comprising receiving three-dimensional data of an object comprising a set of data entries, converting the three-dimensional data into a field comprising a set of cells each having an associated value, and determining representative data based on one or more data entries from the set of data entries of the three-dimensional data for each cell value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology described herein generally relates to methods and apparatus for machine vision, including techniques for processing image data and searching for patterns within images. [Background technology]

[0002] One task often performed by machine vision systems is to attempt to locate and identify the location and orientation of a pattern of interest within an image. Some techniques use a model to represent the pattern of interest, which can include multiple probes. Each probe is a point of interest and associated data (such as a position or vector). Each probe can be used, for example, to determine a measure of similarity between a feature or region of the runtime image and a pattern feature or region at a particular location. Multiple probes can be applied to the runtime image at multiple poses, and information from the probe at each pose can be used to determine the most likely pose of the pattern in the runtime image.

[0003] To speed up the pattern recognition process, some techniques use a multi-step approach to the pattern search process. For example, a first step may involve a coarse search that attempts to find one or more coarse regions in the image that may contain the pattern. A second step (and / or multiple additional steps) may be used to refine the search by searching each of the one or more coarse regions for the pattern. For example, an algorithm may use multiple different models, with each model being used by the system at a different associated resolution of the image. Thus, during the pattern recognition process, a model associated with a coarse resolution may be used to initially identify a rough, approximate pose of an instance of the pattern in the image. Then, a model associated with a relatively finer resolution may be used to more accurately identify the pose of the pattern instance in the image. This iterative process continues until the finest resolution model is used and the exact pose of the pattern instance is determined. Summary of the Invention

[0004] In accordance with the disclosed subject matter, apparatus, systems and methods are provided for improved machine vision techniques, particularly improved machine vision techniques that increase the speed and accuracy of pattern searches within images.

[0005] Some aspects relate to a computerized method for testing a pose of a model in three-dimensional data, the method including receiving three-dimensional data of an object, the three-dimensional data including a set of data entries, converting the three-dimensional data into a field including a set of cells each having an associated value, including determining, for each cell value, representative data based on one or more data entries from the set of data entries of the three-dimensional data, and testing the pose of the model in the field to determine a score for the pose.

[0006] In some examples, converting the three-dimensional data into a field includes generating a three-dimensional array of a set of values.

[0007] In some examples, converting the three-dimensional data into a field includes generating a dense grid, the dense grid containing data for each cell of the grid.

[0008] In some examples, the set of data entries for the three-dimensional data includes a list of points, and determining the representative data based on the one or more data entries for each cell value includes determining a vector based on one or more associated points in the list of points. Determining the vector for the one or more associated points can include determining based on the list of points, where the cells are associated with interior portions of the object, and determining the vector includes setting the vector to zero. Testing the pose of the model against the field to determine a score can include testing a set of probes of the model against the field to determine a score, which includes summing a dot product of each probe and an associated vector in the field.

[0009] In some examples, converting the three-dimensional data into a field containing a set of cells each having an associated value, including determining a representative vector for each cell value, includes generating an accumulation matrix including calculating the cross product of each vector of the set of vectors with itself, where the set of vectors is data determined based on one or more data entries from the set of data entries of the three-dimensional data, and further including extracting eigenvectors, eigenvalues, or both from the accumulation matrix to determine the representative vectors.

[0010] Some aspects relate to a system for testing a pose of a model in three-dimensional data, the system including receiving three-dimensional data of an object, the three-dimensional data including a set of data entries, converting the three-dimensional data into a field including a set of cells each having an associated value, including determining, for each cell value, representative data based on one or more data entries from the set of data entries in the three-dimensional data, and testing the pose of the model against the field to determine a score for the pose.

[0011] In some examples, converting the three-dimensional data into a field includes generating a three-dimensional array of a set of values.

[0012] In some examples, converting the three-dimensional data into a field includes generating a dense grid, the dense grid containing data for each cell of the grid.

[0013] In some examples, the set of data entries for the three-dimensional data includes a list of points, and determining the representative data based on the one or more data entries for each cell includes determining a vector based on one or more associated points in the list of points. Determining the vector for the one or more associated points may include determining based on the list of points, where the cells are associated with interior portions of the object, and determining the vector includes setting the vector to zero. Testing the pose of the model against the field to determine a score may include testing a set of probes of the model against the field to determine a score, where the score includes a sum of each probe and its associated vector in the field.

[0014] In some examples, converting the three-dimensional data into a field containing a set of cells each having an associated value, including determining a representative vector for each cell value, includes generating an accumulation matrix including calculating the cross product of each vector of the set of vectors with itself, where the set of vectors are data determined based on one or more data entries from the set of data entries of the three-dimensional data, and extracting eigenvectors, eigenvalues, or both from the accumulation matrix to determine the representative vectors.

[0015] Some aspects relate to at least one non-transitory computer-readable storage medium having processor-executable instructions stored thereon that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform operations including receiving three-dimensional data of an object, the three-dimensional data including a set of data entries, converting the three-dimensional data into a field including a set of cells each having an associated value, including, for each cell value, determining representative data based on one or more data entries from the set of data entries of the three-dimensional data, and testing a pose of a model with the field to determine a score for the pose.

[0016] In some examples, converting the three-dimensional data into a field includes generating a three-dimensional array of a set of values.

[0017] In some examples, converting the three-dimensional data into a field includes generating a dense grid, the dense grid containing data for each cell of the grid.

[0018] In some examples, the set of data entries for the three-dimensional data includes a list of points, and determining the representative data for each cell based on the one or more data entries includes determining a vector based on one or more associated points in the list of points.

[0019] In some examples, determining a vector for one or more associated points includes determining based on a list of points, the cells being associated with an interior portion of the object, and determining the vector includes setting the vector to zero.

[0020] In some examples, converting the three-dimensional data into a field containing a set of cells each having an associated value, including determining a representative vector for each cell value, includes generating an accumulation matrix including calculating the cross product of each vector of the set of vectors with itself, where the set of vectors are data determined based on one or more data entries from the set of data entries of the three-dimensional data, and extracting eigenvectors, eigenvalues, or both from the accumulation matrix to determine the representative vectors.

[0021] Some aspects relate to a computerized method for testing the pose of a model against image data. The method includes receiving image data of an object, the image data including a set of data entries. The method includes determining a set of regions of the image data, each region in the set of regions including an associated set of neighboring data entries in the set of data entries. The method includes generating processed image data, the processed image data including a set of cells each having an associated value, and generating the processed image data includes, for each region in the set of regions, determining a maximum possible score for each data entry in the associated set of neighboring data entries from the image data. The method includes setting one or more values ​​of a value set based on the determined maximum possible score. The method includes testing the pose of the model using the processed image data.

[0022] In some examples, receiving the image data includes receiving 2D image data, where each data entry contains a 2D vector, and determining the maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on the 2D vector within a region associated with the value.

[0023] In some examples, testing the poses of the model using the processed data includes determining that the pose scores do not exceed a predetermined threshold, testing multiple probes of the model against associated scalar values ​​of the processed data, and excluding from further testing the set of poses associated with each region used to determine the associated scalar values.

[0024] In some examples, receiving the image data includes receiving 3D image data, where each data entry includes a 3D vector, and determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on the 3D vector within a region associated with the value. Testing the pose of the model using the processed data includes determining that the score of the pose does not exceed a predetermined threshold, testing multiple probes of the model against associated scalar values ​​in the processed data, and excluding from further testing the set of poses associated with each region used to determine the associated scalar value.

[0025] In some examples, converting the three-dimensional data into a second field including a second set of cells each associated with a second value includes, for each second cell value, determining representative data based on one or more data entries from the set of data entries of the three-dimensional data, and testing the pose of the model with the second field based on testing the pose of the model with the field.

[0026] Some aspects relate to a system for testing the pose of a model against image data, the system comprising one or more processors configured to receive image data of an object, the image data containing a set of data entries. The one or more processors are configured to determine a set of regions of the image data, where each region in the set of regions contains an associated set of neighboring data entries in the set of data entries. The one or more processors are configured to generate processed image data, where the processed image data contains a set of cells each having an associated value, and generating the processed image data is configured to, for each region in the set of regions, determine from the image data a maximum possible score for each data entry in the associated set of neighboring data entries. The one or more processors are configured to set one or more values ​​of the set of values ​​based on the determined maximum possible score. The one or more processors are configured to test the pose of the model using the processed image data.

[0027] In some examples, receiving the image data includes receiving 2D image data, where each data entry contains a 2D vector, and determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on the 2D vector within a region associated with the value. Testing the pose of the model using the processed data includes determining that the score of the pose does not exceed a predetermined threshold, testing multiple probes of the model against associated scalar values ​​in the processed data, and excluding from further testing the set of poses associated with each region used to determine the associated scalar value.

[0028] In some examples, receiving the image data includes receiving 3D image data, where each data entry includes a 3D vector, and determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on the 3D vector within a region associated with the value. Testing the pose of the model using the processed data includes determining that the score of the pose does not exceed a predetermined threshold, testing multiple probes of the model against associated scalar values ​​in the processed data, and excluding from further testing the set of poses associated with each region used to determine the associated scalar value.

[0029] In some examples, converting the three-dimensional data into a second field including a second set of cells each associated with a second value includes, for each second cell value, determining representative data based on one or more data entries from the set of data entries of the three-dimensional data, and testing the pose of the model with the second field based on testing the pose of the model with the field.

[0030] Some aspects relate to at least one non-transitory computer-readable storage medium having processor-executable instructions stored thereon, which, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform an operation of receiving image data of an object, the image data containing a set of data entries. The instructions cause the at least one computer hardware processor to determine a set of regions of the image data, where each region in the set of regions contains an associated set of adjacent data entries in the set of data entries. The instructions cause the at least one computer hardware processor to generate processed image data, where the processed image data contains a set of cells each having an associated value, and generating the processed image data includes, for each region in the set of regions, determining from the image data a maximum possible score for each data entry in the associated set of adjacent data entries. The instructions cause the at least one computer hardware processor to set one or more values ​​of a set of values ​​based on the determined maximum possible score. The instructions cause the at least one computer hardware processor to test the pose of a model using the processed image data.

[0031] In some examples, the instructions cause at least one computer hardware processor to perform an operation of receiving image data, where receiving the image data includes receiving 3D image data, where each data entry includes a 3D vector, and determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on the 3D vector within a region associated with the value. Testing the pose of the model using the processed data includes determining that the score of the pose does not exceed a predetermined threshold, testing multiple probes of the model against associated scalar values ​​in the processed data, and excluding from further testing the set of poses associated with each region used to determine the associated scalar value.

[0032] In some examples, receiving the image data includes receiving 3D image data, where each data entry includes a 3D vector, and determining the maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on the 3D vector within a region associated with the value.

[0033] In some examples, testing the poses of the model using the processed data includes determining that the pose scores do not exceed a predetermined threshold, testing multiple probes of the model against associated scalar values ​​of the processed data, and excluding from further testing the set of poses associated with each region used to determine the associated scalar values.

[0034] In some examples, the instructions cause at least one computer hardware processor to perform operations of transforming the three-dimensional data into a second field containing a second set of cells each associated with a second value, including determining, for each second cell value, representative data based on one or more data entries from the set of data entries in the three-dimensional data, and testing the pose of the model with the second field based on testing the pose of the model with the field.

[0035] Some aspects relate to a computerized method for testing the pose of a three-dimensional model, the method including: storing a three-dimensional model, the three-dimensional model comprising a set of probes; receiving three-dimensional data of an object, the three-dimensional data comprising a set of data entries; converting the three-dimensional data into a set of fields, which generates a first field comprising a first set of values, wherein each value in the first set of values ​​indicates a first characteristic of one or more associated data entries from the set of data entries, the first characteristic being different from a second characteristic; generating a second field comprising a second set of values, wherein each value in the second set of values ​​indicates a second characteristic of one or more associated data entries from the set of data entries, the second characteristic being different from the first characteristic; and testing the pose of the three-dimensional model with the set of fields to determine a score for the pose, which includes testing the set of probes against the set of fields.

[0036] In some examples, generating the first field and the second field includes generating a three-dimensional array for each field, where each three-dimensional array is comprised of a set of three indices containing an index for each dimension, and where each three-dimensional array includes an x-position, a y-position, and a z-position of each of the first value and the second value related by the set of three indices.

[0037] In some examples, the probe, the first set of values ​​of the first field, and the second set of values ​​of the second field include surface normal data, edge boundary data, intensity data, or some combination thereof.

[0038] In some examples, testing the posture to determine a score for the posture includes summing values ​​associated with the dot products of each probe.

[0039] In some examples, the method includes testing a plurality of poses to determine a plurality of associated scores, determining which poses of the plurality of poses have scores above a predetermined threshold to generate a set of poses, and storing the set of poses for subsequent processing, wherein each pose in the set of poses represents a local peak in the associated scores, and the method further includes refining the set of poses to determine a top pose of the model.

[0040] Some aspects relate to a system for determining parameters for image acquisition, the system comprising one or more processors configured to: store a three-dimensional model comprising a set of probes; receive three-dimensional data of an object comprising a set of data entries; convert the three-dimensional data into a set of fields, which generates a first field comprising a first set of values, wherein each value in the first set of values ​​indicates a first feature of one or more associated data entries from the set of data entries, the first feature being different from a second feature; generate a second field comprising a second set of values, wherein each value in the second set of values ​​indicates a second feature of one or more associated data entries from the set of data entries, the second feature being different from the first feature; and test a pose of the three-dimensional model against the set of fields to determine a score for the pose, which includes testing the set of probes against the set of fields.

[0041] In some examples, generating the first field and the second field includes generating a three-dimensional array for each field, where each three-dimensional array is comprised of a set of three indices containing an index for each dimension, and where each three-dimensional array includes an x-position, a y-position, and a z-position of each of the first value and the second value related by the set of three indices.

[0042] In some examples, the probe, the first set of values ​​in the first field, and the second set of values ​​in the second field include surface normal data, edge boundary data, intensity data, or some combination thereof.

[0043] In some examples, testing the posture to determine a score for the posture includes summing values ​​associated with the dot products of each probe.

[0044] In some examples, the one or more processors are further configured to test the plurality of poses to determine a plurality of associated scores, determine which poses of the plurality of poses have a score above a predetermined threshold to generate a set of poses, and store the set of poses for subsequent processing.

[0045] In some examples, each pose in the set of poses represents a local peak in the associated score, and the method includes further refining the set of poses to determine a top pose for the model.

[0046] Some embodiments relate to at least one non-transitory computer-readable storage medium having processor-executable instructions stored thereon that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform operations of: storing a three-dimensional model comprising a set of probes; receiving image data of the object comprising a set of data entries; converting the three-dimensional data into a set of fields, which generates a first field comprising a first set of values, where each value in the first set of values ​​indicates a first characteristic of one or more associated data entries from the set of data entries; generating a second field comprising a second set of values, where each value in the second set of values ​​indicates a second characteristic of one or more associated data entries from the set of data entries, where the second characteristic is different from the first characteristic; and testing a pose of the three-dimensional model with the set of fields to determine a score for the pose, which includes testing a set of probes against the set of fields.

[0047] In some examples, generating the first field and the second field includes generating a three-dimensional array for each field, where each three-dimensional array is comprised of a set of three indices containing an index for each dimension, and where each three-dimensional array includes an x-position, a y-position, and a z-position of each of the first value and the second value related by the set of three indices.

[0048] In some examples, the probe, the first set of values ​​of the first field, and the second set of values ​​of the second field include surface normal data, edge boundary data, intensity data, or some combination thereof.

[0049] In some examples, testing the posture to determine a score for the posture includes summing values ​​associated with the dot products of each probe.

[0050] In some examples, the instructions further cause the one or more processors to test the plurality of poses to determine a plurality of associated scores, determine which poses of the plurality of poses have a score above a predetermined threshold to generate a set of poses, and store the set of poses for subsequent processing.

[0051] In some examples, each pose in the set of poses represents a local peak in the associated score, and the method further includes refining the set of poses to determine a top pose for the model.

[0052] The foregoing has outlined, rather broadly, the features of the disclosed subject matter in order that the detailed description that follows may be better understood, and in order that the present contribution to the art may be better appreciated. There are, of course, additional features of the disclosed subject matter that will be described hereinafter and which form the subject of the claims appended hereto. The phraseology and terminology employed herein should be understood to be for the purpose of description and not of limitation.

[0053] In the drawings, each identical or nearly identical component shown in various figures is represented by the same reference numeral. For clarity, not every component is shown in every drawing. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating various aspects of the techniques and apparatus described herein. [Brief explanation of the drawings]

[0054] [Figure 1] 1 illustrates an exemplary machine vision system according to some embodiments.

[0055] [Figure 2A] 1 illustrates an exemplary method for generating fields and using the fields to test a model, according to some embodiments.

[0056] [Figure 2B] 1 illustrates an exemplary method for generating a field, according to some embodiments.

[0057] [Figure 3] 1 illustrates exemplary three-dimensional image data and associated fields, according to some embodiments.

[0058] [Figure 4] 1 illustrates an exemplary method for generating processed image data for coarse search, according to some embodiments.

[0059] [Figure 5] 1 illustrates exemplary two-dimensional image data and associated processed image data according to some embodiments.

[0060] [Figure 6] 10 illustrates the exclusion of a set of exemplary poses during a coarse search and data entry of associated 2D image data, according to some embodiments.

[0061] [Figure 7]1 illustrates exemplary three-dimensional image data and associated processed image data, according to some embodiments.

[0062] [Figure 8] 10 illustrates the exclusion of a set of exemplary poses during a coarse search and data entry of associated 3D image data, according to some embodiments.

[0063] [Figure 9] 1 illustrates an exemplary method for initial pose search, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0064] In the following description, numerous specific details are set forth regarding the systems and methods of the disclosed subject matter, as well as the environments in which such systems and methods operate, in order to provide a thorough understanding of the disclosed subject matter. In addition, it will be understood that the examples provided below are exemplary and that other systems and methods are contemplated that are within the scope of the disclosed subject matter.

[0065] FIG. 1 illustrates an exemplary machine vision system 100 according to some embodiments. The exemplary machine vision system 100 includes a camera 102 (or other image capture device) and a computer 104. The computer 104 includes one or more processors and a human-machine interface in the form of a computer display, and optionally one or more input devices (e.g., a keyboard, a mouse, a trackball, etc.). The camera 102 includes a lens 106 and a camera sensor element (not shown), among other components. The lens 106 has a field of view 108, and the lens 106 focuses light from the field of view 108 onto the sensor element. The sensor element generates a digital image of the camera field of view 108 and provides the image to a processor forming part of the computer 104. As shown in the example of FIG. 1, an object 112 moves along a conveyor 110 into the field of view 108 of the camera 102. While the object 112 is within the field of view 108, the camera 102 can generate one or more digital images of the object 112 for processing, as described below. The conveyor may carry multiple objects during operation. These objects may pass sequentially through the field of view 108 of the camera 102, such as during an inspection process. In this manner, the camera 102 may capture at least one image of each object 112 observed.

[0066] In some embodiments, the camera 102 is a two-dimensional imager, such as a two-dimensional (2D) CC and / or CMOS imaging array. In some embodiments, the two-dimensional imager generates a 2D array of intensity values. In some embodiments, the machine vision system processes the 2D data, for example, by generating a two-dimensional gradient field image. The gradient field image can include a set of cells associated with, for example, magnitude and direction. For example, the gradient field can include Cartesian components (x, y) of a vector, which includes magnitude and direction, and the gradient field can store actual (r, theta) values, etc. In some embodiments, the camera 103 is a three-dimensional (3D) imager. The 3D imager can generate a set of (x, y, z) points (e.g., where the z axis is the third dimension, such as distance from the 3D imager). The 3D imager can use various 3D image generation techniques, such as shape from shading, stereo imaging, time-of-flight techniques, projector-based techniques, and / or other 3D generation techniques.

[0067] In some embodiments, the machine vision system processes 3D data from the camera 103. The 3D data received from the camera 103 can include, for example, a point cloud and / or a range image 108. A point cloud can include a collection of three-dimensional points on or near the surface of a solid. For example, the points can be represented by coordinates in a rectilinear coordinate system or other coordinate system. In some embodiments, other information indicating which points are adjacent on the object, such as a mesh or grid structure, can optionally be present. In some embodiments, the input point cloud can include information about surface features, including curvature, surface normals, edges, and / or color and albedo information, derived from sensor measurements or previously calculated. In some embodiments, the 2D and / or 3D data can be obtained from 2D and / or 3D sensors, from CAD or other solid models, and / or by range images, 2D images, and / or other images and / or preprocessing.

[0068] Examples of computer 104 may include, but are not limited to, a single server computer, a series of server computers, a single personal computer, a series of personal computers, a minicomputer, a mainframe computer, and / or a computing cloud. Various components of computer 104 may run one or more operating systems, examples of which include Microsoft Windows Server 2008 R2, Microsoft Windows Server 2008 R2, and Microsoft Windows Server 2008 R2. TM , Novell Netware TM The computer 104 may be configured to run a variety of operating systems, including, but not limited to, Linux, Red Hat Linux, Unix, and / or custom operating systems. The one or more processors of the computer 104 may be configured to process operations stored in memory coupled to the one or more processors. The memory may include, but is not limited to, a hard disk drive, a flash drive, a tape drive, an optical drive, a RAID array, random access memory (RAM), and read-only memory (ROM).

[0069] As discussed herein, to search for a model within an image of an object, the technique can be configured to perform two (or more) stages, including a first stage for determining an approximate or coarse location of the model within the image and a second stage for refining the coarse location to determine the specific location of the model. Using multiple stages can be beneficial because the search can be performed using different technical techniques for each stage, e.g., to improve search speed, efficiency, etc. For example, in the case of a 2D pattern search technique, the technique can include training a model of the object encompassing a set of probes. Each of the 2D probes can include an (x, y) position and orientation. The machine vision system stores the trained model for use in searching runtime images acquired later of a scene (e.g., a scene of an object 112 moving along a conveyor belt 110 depicted in FIG. 1). The machine vision system can acquire an image of the scene, such as a 2D image with pixel values ​​representing accumulated light in each cell. The machine vision system can convert the 2D image into a dense gradient field, where each cell contains a single gradient value that reflects the variance of the data for a particular image (e.g., the gradient is near zero in certain ranges and large near edges). The gradient value may be, for example, a vector, a scalar value (e.g., magnitude or direction angle), and / or the like. For example, if each cell contains a vector, the magnitude of the vector may be zero. The machine vision system can test various transformations (e.g., translation, rotation, scale, skew, aspect, perspective, etc.) of the 2D model to calculate a score for a particular configuration. The score can reflect, for example, how well the probe matches the vector in the 2D image (e.g., if the vector is a gradient of the input 2D information, as discussed further herein). For example, some techniques can test the model bin by bin and take the highest-scoring position for subsequent refinement. Some techniques can be configured to focus on local peaks since close matches are similar, and the system can be configured to try to avoid marking all close matches as candidate poses. For example, in a 3D0 context, the image may be a range image, a point cloud, and / or the like.As a general matter, and as discussed further herein, a machine vision system can train a 3D model of an object that encompasses a set of probes. The machine vision system can use the 3D model to locate the pose of the 3D model in a runtime 3D image. As discussed herein, the machine vision system can process the 3D image to generate one or more 3D fields to facilitate locating the pose of the 3D model. The fields can encompass a set of vectors. The Cartesian components (x, y, z) of the vectors can be stored, which may include magnitude and direction, and the fields can store real (r, lat, long) values ​​and / or the like.

[0070] The embodiments discussed herein can be used in a variety of different applications, some of which may include, but are not limited to, part picking in vision-guided robotics, 3D inspection, automotive kitting, molded plastic and cast metal volumetric inspection, and assembly inspection. Such applications can include locating and identifying the location and orientation of a pattern of interest within an image (e.g., for guiding a robotic gripper or inspecting an object). In some embodiments, a training step is used to develop a model representing the pattern of interest, which can include multiple probes. Each probe is a point of interest and associated data (e.g., position and vector) that can be used, for example, to measure the similarity of a runtime image feature or region to a feature or region of the pattern at a particular location. Multiple probes can be applied to the runtime image in multiple poses, and information from the probe at each pose can be used to determine the most likely pose of the pattern in the runtime image.

[0071] The present inventors have discovered that existing machine vision technologies can suffer from significant inefficiencies when using conventional 3D data. In particular, the inventors have discovered that searching for neighbors of a point in 3D data (e.g., searching for nearby points in a point cloud) often consumes a significant amount of processing time. For example, while machine vision systems are efficient at processing data in close proximity, they are extremely inefficient at searching for or randomly accessing data. In particular, computing devices may include hardware optimized for massive parallelization of tasks that are often repeated in consecutive memory locations. Breaking such parallelization with a conditional branch can significantly reduce performance (e.g., because the branch typically requires a halt in parallel activity, consumes time to execute the branch / jump, and then resumes parallel activity). The present inventors have developed technical improvements to machine vision technology to address these and other inefficiencies. As discussed further herein, this technology involves developing a dense field from the 3D data, where the dense field contains data for each field value determined based on the 3D data. The inventors have discovered that because each value encapsulates data, machine vision techniques can use the field values ​​as part of a process to avoid searching for neighboring points as described above, which can significantly reduce the processing time of existing machine vision techniques. By creating the data for each entry in the field or grid and processing them sequentially, the techniques disclosed herein can avoid time-consuming branches that break parallelization, thereby significantly improving performance.

[0072] FIG. 2A illustrates an exemplary method 200 for generating a field from a 3D image for testing 3D modes, according to some embodiments. In step 202, a machine vision system receives three-dimensional data (e.g., a point cloud, a dense 3D image, etc.) of an object. The three-dimensional data includes a set of data entries, such as a range image (e.g., with the height of each cell), a point cloud, and / or other information related to the 3D data. In step 204, the machine vision system converts the three-dimensional data into a field. The field includes a set of values. The method includes determining, for each value of the field, representative data derived from the 3D data. The representative data may include, for example, vectors and / or other representative information for the data entries, such as simply the magnitude and / or direction of a vector instead of a complete vector, a matrix (e.g., vv, as discussed further herein), and the like. T ), and / or the like. The method may include determining representative data based on one or more data entries from the set of data entries of the three-dimensional data. In step 206, the machine vision system tests the model against the fields to determine a score for the pose. In step 208, the machine vision system determines whether the pose is acceptable or whether the pose should be skipped (e.g., ignored, discarded, etc.). For example, the machine vision system may score the pose of the 3D model against the fields and determine whether the score exceeds a threshold. As another example, as discussed further herein, the machine vision system may score the pose of the 3D model against multiple fields. If the score meets (e.g., exceeds) a threshold, the method proceeds to step 210, where the machine vision system saves the pose (e.g., for subsequent refinement, as discussed herein). If the score does not meet the threshold, the method proceeds to step 212, where it determines whether there are additional poses of the 3D model to test. If there are more poses, the method returns to step 206, where it tests another pose of the 3D model. If there are no more poses, the method proceeds to step 214, where it ends.

[0073] Referring to step 204, converting the three-dimensional data to a field can include generating a three-dimensional array of a set of values. For example, the three dimensions can represent the x, y, and z axes of the 3D data. Each value in the set of values ​​can be a vector. The vectors can be represented in various ways. For example, in some embodiments, each vector can be stored as x, y, and z components, each of which can be represented as a specific number of bits, e.g., a signed 8-bit integer, and the values ​​can range from -127 to 127. As another example, in some embodiments, each vector can be represented using a magnitude and two angles. In some embodiments, converting the three-dimensional data to a field can include generating a dense grid. The dense grid can, for example, include a value for each possible spot in the grid. The spots in the grid may or may not be connected to the original locations of the data entries in the 3D data. For example, a kernel or filter can be used so that the grid spots have a different grid than the 3D data. The process can include converting the point cloud to a dense field, where the dense field contains a vector at each possible location in the dense field.

[0074] In some embodiments, the techniques may include applying a transformation to the 3D data before generating the field. For example, the techniques may include applying a rigid transformation, a linear transformation, and / or a non-linear transformation before generating the field. In some embodiments, the techniques may include applying one or more transformations to account for distortion. For example, a point cloud may be acquired such that values ​​are skewed along one or more axes due to delay, and the point cloud may be transformed to account for this skew. In some embodiments, the techniques may include searching for distortion of the model when searching for the pose of the model. For example, the degree of skewness of degrees of freedom may be found by testing various transformations of the model.

[0075] With further reference to step 204 of FIG. 2, FIG. 2B illustrates an exemplary method 250 for generating a field, according to some embodiments. For example, in some embodiments, this technique can be used to convert a 3D point cloud into a dense field. The dense field contains data at each field value, as discussed further herein (e.g., a point cloud lists only the locations of points, and the absence of a point includes the point being missing or not in a nearby location). As noted above, the data can contain vectors. The (x, y, z) of a field cell indicates where the vector occurs in the field. In some embodiments, field values ​​that are inside a three-dimensional object in the 3D data can be set to zero. Thus, in some embodiments, fields facilitate access to information compared to 3D data, such as compared to a point cloud. In some embodiments, fields can make 3D data appear more uniform, while other 3D data, such as a point cloud, can highlight specific regions. As another example, in some embodiments, this technique can be used to convert a dense 3D image into a dense field. For example, as discussed further herein, a system could determine the gradient at each point for the dense 3D image and apply method 250.

[0076] In step 252, the machine vision system determines a set of vectors for each 3D data entry (e.g., a 3D point). For example, the machine vision system can use information from neighboring 3D data point positions and / or a 3D sensor to determine a surface normal vector and an edge vector for each 3D data entry. Any of the vectors can have a length of zero (0), for example, to indicate that there is no distinct normal or edge for a particular data entry.

[0077] In step 254, for each 3D data entry (e.g., point), the machine vision system determines the field cell (e.g., voxel) that contains it. In step 256, the machine vision system determines accumulated data associated with each field cell. In some embodiments, the technique includes determining an input vector associated with each field cell. In some embodiments, the system can accumulate aggregate information about the vectors associated with all 3D data points that fall into that field cell. For example, the aggregate information can include the vector components themselves (if polarity is meaningful) and / or other information, such as the cross product of each vector with itself, vv T (e.g., can be used when polarity is not meaningful). In some embodiments, the technique can include expanding the range of influence of each point, for example, by blurring or thickening a 3D feature. In some embodiments, the system can create replication in a predetermined pattern around each 3D data point. The predetermined pattern can be relative to the direction of the vector, for example. For example, the technique can thicken surfaces (e.g., by replicating normals above and below), thicken edges (e.g., by replicating edges in a cylinder around the crease vector), etc. The predetermined pattern can differ depending on what the vector represents, such as whether it represents a normal or an edge.

[0078] In step 258, the machine vision determines representative data for each field cell based on the accumulated data from step 256. In some embodiments, if a field cell is not associated with any accumulated data (e.g., not associated with any 3D data entry, e.g., no vector falls into the field cell), the field cell can be set to zero (e.g., zero is used to refer to a zero vector when the field contains vectors).

[0079] In some embodiments, the technique may include determining a representative vector for each field cell based on the accumulated data for that cell determined in step 254. The representative vector may be determined, for example, by calculating component-wise averages (e.g., the cross product of each vector with itself, vv T In some embodiments, a regularization constant can be added to the denominator to prevent division by zero, reduce the length of the representative vector when there is little data contributing to it, etc. For example, the matrix M can be determined by extracting eigenvectors from an accumulated matrix (formed by accumulating vectors v1 through v2). n For a set of n vectors v containing i where i=1-n). TIFF2026035868000002.tif17123

[0080] In some embodiments, the machine vision system can save matrix M as it generates the field (e.g., for posture testing). In some embodiments, it can save representative data of matrix M, such as only six unique values ​​in the matrix, information representative of the six unique values ​​(e.g., only five (or fewer) of the six values ​​because adding a constant multiple of the identity matrix can zero out one of the values), etc.

[0081] In some embodiments, the machine vision system can determine the representative vectors using matrix M. For example, the machine vision system can determine the representative vectors using eigendecomposition as described above. The representative vectors can be calculated using eigenvalues, eigenvectors, and / or both. In some embodiments, the eigenvalues ​​can be used to determine the magnitude of the representative vector. For example, the largest eigenvalue can be used as a representation of the largest magnitude. As another example, one or more additional eigenvalues ​​can be used in combination with the largest eigenvalue (e.g., to account for potential inconsistencies in the vectors within the representative vector). In some embodiments, the eigenvectors can be used to determine the direction of the representative vector. For example, the eigenvector associated with the largest eigenvalue can be used to represent the dominant direction. The representative vectors can be determined by multiplying the eigenvalues ​​by the eigenvectors. For example, the eigenvector associated with the largest eigenvalue can be multiplied by only the largest eigenvalue, or by the difference between the largest and second largest eigenvalues ​​(e.g., which can be zero if the largest and second largest eigenvalues ​​are the same value), etc. In some embodiments, for example, vv T To remove the square of the magnitude from , the eigenvectors can be multiplied by the square root of the eigenvalue. For example, the eigenvector associated with the largest eigenvalue can be multiplied by the square root of the largest eigenvalue, the square root of the difference between the largest and second largest eigenvalue, etc.

[0082] In some embodiments, the fields can be normalized. For example, the system can normalize the fields by mapping the length of each vector (via a sigmoid) without changing the orientation. Normalization can be used to adjust the response of the posture score to a threshold, for example. For example, some embodiments can simply bin the results as passing (e.g., above a threshold) or failing (e.g., below a threshold). In some embodiments, the fields can be normalized more stably (e.g., using a sigmoid to output a length between 0 and 1).

[0083] In some embodiments, the fields are not normalized. For example, in some embodiments, the raw data (e.g., vector magnitudes) may be meaningful without normalization. For example, shorter vector magnitudes may mean less confidence / agreement about a normal or edge, while longer vector magnitudes may mean more confidence. In some embodiments, scoring techniques (e.g., the dot product discussed herein) may incorporate such data (e.g., magnitudes) from the fields, and therefore it may be desirable to use unnormalized fields.

[0084] In some embodiments, each field cell value can be based on one associated data entry in the 3D data, multiple data entries in the 3D data, and / or no data entries. In some embodiments, the technique can include determining a vector for each of the field cell values. For example, the data entries in the three-dimensional data can include a list of points, and the technique can determine a vector for each field based on the list of points. In some embodiments, the technique can determine values ​​for fields that are interior to an object in the 3D data, as described above. For example, a machine vision system can determine, based on the point cloud, that one or more field cells are associated with an interior portion of the object and set the values ​​to zero.

[0085] Referring to steps 206-212, method 200 can be used as described above to perform a coarse stage of 3D model alignment search within a 3D image. In some embodiments, method 200 can search for an approximate pose of the 3D model within the field, which can be further refined by subsequent steps. The approximate pose can include 3D position data, e.g., (x, y, z) position, and orientation data, e.g., roll, pitch, and / or yaw. Referring to step 206, in some embodiments, the testing includes testing a set of probes of the 3D model against the field. For example, a machine vision system can test a set of probes of the model against the field and determine a score by summing the dot product of each probe and its associated vector within the field. In some embodiments, the score can be based on multiple values ​​(e.g., multiple fields), such as crease edge information, occlusion boundary information, color information, intensity information, etc., as further discussed in connection with FIG. 9 .

[0086] 3 illustrates exemplary three-dimensional image data 300 and associated fields 350, according to some embodiments. Fields 350 can be generated using method 250 of FIG. 2B. 3D image data 300 includes a set of data entries 302A-302N (collectively data entries 302). For each data entry 302, a machine vision system can determine a set of vectors. For example, the machine vision system can determine vectors v1-v2 for data entry 302A, as shown in FIG. 3. n The field 350 contains a set of cells associated with values ​​352A through 352N (collectively cells 352). To convert the 3D image data 300 into the field 350, the machine vision system can perform one or more functions to convert the determined vectors into the associated field values. For example, as shown for value 352A, the machine vision system converts the vectors v1 through v2 of the data entry 302A into the associated field values. nA function operating on the data entry 302A may be executed to generate aggregate information "s" for the data entry 302A. As discussed herein, the resulting value s for this function may be a single vector, a scalar value (e.g., the magnitude of a vector), and / or other aggregate information. This function may be executed for each data entry to populate the associated values ​​of the fields. Figure 3 illustrates the vectors v1-v2 for the data entry 302A. n Although only a set of data points is shown, this is merely for illustrative purposes to explain the technique. As discussed herein, the technique may involve accumulating information about multiple data points that are used to determine an associated value for each field cell.

[0087] As mentioned above, the aggregate information "s" can be converted into a final representative vector (not shown in FIG. 3). For example, in some embodiments, the aggregate information "s" can be converted into a final representative vector by calculating a component-wise average divided by the number of entries accumulated in that cell (e.g., optionally a regularization constant in the denominator). As another example, in some embodiments, eigenvectors can be extracted from the matrix used to accumulate the information.

[0088] The inventors have determined that searching for a model within image data, whether 2D or 3D data, is a time-consuming process due to the need to repeatedly test each pose of the model against the data. For example, when performing a search, there are dimensions of the search space (e.g., image data and / or fields, such as 2D and / or 3D runtime data) and dimensions of the pose space (e.g., x, y, z, roll, pitch, yaw, scale, skew, aspect, perspective, and other nonlinear distortions). The more dimensions there are, the more poses to search for the model within the image data, and the more processing required to search for the model.

[0089] The inventors have developed a technique for processing image data before searching for a model. The processed image data allows the system to eliminate a large portion of the potential pose space during the search. The machine vision system can be configured to perform large spot checks on the search space to refine their scope, significantly increasing processing speed. As discussed further below, the processed image data allows the machine vision system to take arbitrarily large steps as the pose is refined to ensure that the machine vision system does not miss poses that would improve its score (e.g., above a predetermined threshold). This technique contrasts with downsampling techniques, which can be used, for example, to improve search speed but may miss poses that would otherwise be considered refined. In some embodiments, the techniques described herein can provide sampling that is in the same dimension as the image data (e.g., field), reducing a group of data entries in the image data to a single value. The set of values ​​in the processed image data allows the machine vision system to test whether a model can be found for any relevant poses in the image data.

[0090] As discussed further herein, the techniques can generally divide the search into one or more layers. For example, the techniques can generate two distinct layers of image data, including a first layer of processed image data (e.g., layer 1) and a second layer of processed image data (e.g., layer 2), to produce larger search regions of the image data, where each larger search region in the second layer cumulatively represents multiple smaller regions in the first layer. The system can process each larger region in the lower layers to determine whether to search smaller regions in higher layers (e.g., where layer "1" is higher than layer "2") for pose. As discussed herein, the system can use third and / or higher layers, with each lower layer generated using a larger region than the preceding layer. In some embodiments, the larger regions can be analyzed using a maximum score limit for the associated smaller regions. Using this maximum score technique, the system can determine that the model will not obtain a score higher than the value within the region. Thus, the system can use the maximum score technique to determine whether any possible poses in the associated regions in the higher layers obtain sufficiently high scores that they are worthy of further investigation when testing the image data. If not, the system may determine that none of the data entries associated with that region merit further consideration.

[0091] FIG. 4 shows an exemplary method 400 for coarse search, according to some embodiments. In step 402, a machine vision system receives image data of an object (e.g., 2D or 3D data, such as a point cloud, a dense 3D image, etc.). The image data includes a set of data entries, which may include gradient information, vectors, and / or the like. In step 404, the system determines a set of regions of the image data. Each region includes a set of associated neighboring data entries from the image data. In step 406, the system generates processed image data using this set of regions. As discussed further herein, the processed image data includes a set of cells, each with an associated value. The system can generate the processed image data by determining, for each region, the maximum possible score for each data entry in the associated set of neighboring data entries from the image data. In step 408, the system tests the pose of the model using the processed image data and calculates a score for the pose. In step 410, the system determines whether the score meets a predetermined threshold. If the score does not meet the threshold, the method proceeds to step 412, where one or more regions are excluded from subsequent pose testing. If the score reaches a threshold, the method proceeds to step 414 to store the pose (e.g., for subsequent refinement). In step 416, the system determines whether there are more poses to test; if so, the method returns to step 408; if not, the method ends at step 418.

[0092] Referring to step 402, the image data can be a vector field representing any type of data. For example, the vector field may include one or more of: (a) surface normal vectors; (b) fold edge vectors (e.g., actual edges of an object, such as the sides of a book, and / or occlusion edges, such as a silhouette or the edge of a mug, which is not an actual edge of the mug because the mug is cylindrical); and / or (c) color edge vectors, such as edges based on the color of an object (e.g., where one color ends and another begins, such as on a striped object). In some embodiments, this technique can be performed using multiple fields, as discussed further herein. In some embodiments, a machine vision system can determine the vector field based on received image data (e.g., 2D and / or 3D image data).

[0093] Referring to step 404, the machine vision system can determine a set of regions within the image data. For example, the regions can include the same number of data entries from each image data, as further described below in connection with FIGS. 5-8. For example, for 2D data, each region can include 16x16 data entries, 16x8 data entries, 8x8 data entries, 4x4 data entries, 4x3 data entries, etc. As another example, for 3D data, each region can include 16x16x16 data entries, 8x8x8 data entries, 16x8x8 data entries, 5x5x5 data entries, etc.

[0094] In some embodiments, regions overlap with other nearby regions. For example, as described further below in connection with Figures 5-8, a machine vision system can repeatedly move from data entry to data entry and determine regions based on adjacent data entries (e.g., a predetermined number of data entries at adjacent locations), resulting in some overlap between data entries of nearby regions. For example, as described further below in connection with Figures 7-8, if a region is a 2x2x2 region of data entries in 3D data, a first region will overlap its neighboring region by 4 data units.

[0095] In some embodiments, configuring a machine vision system to determine regions with some overlap between adjacent regions can provide better pose testing. For example, using overlapping regions can provide better pose testing than using non-overlapping regions. As an example, assume that a machine vision system uses a model with multiple probes, and that these probes can have different phases relative to each other. Also assume for this example that the probes have integer spacings in data units and / or regions. As a result, for example, a probe will fall within a specific data unit instead of potentially falling into a location shared by multiple data units. If the machine vision system in such an example determines that the regions do not overlap (e.g., a region does not share data entries with adjacent regions), the processed image data will have reduced resolution from the original image data. Therefore, the location of the probe in the original image data may not be testable in the processed image data. As a simple example, assume that two adjacent probes in a model correspond to parallel data entries in the original image data, and the machine vision system determines the processed image data using a 2x2 data entry region. In this example, one of the two probes would lie on a line between two regions in the processed image data at each pose tested (e.g., due to a loss of image data resolution when calculating the processed image data). Thus, in this example, the machine vision system would not be able to test the probe against the maximum score within a 2x2 neighborhood in the processed image data. Because the machine vision system would not be able to test that probe in the processed image data, the machine vision system would not be able to correctly estimate a score for a model determined using, for example, the maximum possible score as discussed herein. This could therefore cause the machine vision system to erroneously ignore a pose for subsequent testing and / or include a pose that does not include a potential location for the pose in subsequent testing.

[0096] Computing regions with some overlap allows the machine vision system to test each pose. If resolution is not reduced, for example, by calculating maximum motion at each data entry (e.g., as described in connection with Figures 5-8), each possible probe of the machine vision system can directly match the calculated values ​​in the processed image data (e.g., the machine vision system can test against the correct 2x2 region of the original image data in this way). In this case, the machine vision system can advantageously shift the model by one or more cells (e.g., two or more cells) for the next test. The model can be shifted in this manner because, due to the structure of the regions and the maximum score technique discussed herein, once the machine vision system has indirectly evaluated a complete 2x2 region in the upper-layer image data in one test, the next location the machine vision system needs to test is the next adjacent (non-overlapping) 2x2 region of the image data. This technique thus allows each probe to skip reevaluating some of the cells it (implicitly) evaluated in the previous test, allowing the system to advantageously shift the model by one or more cells.

[0097] Referring to step 406, the technique can be configured to use the image data (e.g., the vector field described above) to generate processed image data (e.g., a new vector field) of the same or similar resolution, where each value in the processed image data represents the maximum possible score for a model probe that falls within a group of image data entries that encompasses a particular data entry. For example, the scoring scheme may have a maximum possible score achievable for a particular probe and data entry in the image data (e.g., this may be lower depending on the actual data for the probe). To enable the system to perform a conservative search for a model pose within the processed image data, the technique can determine the maximum possible scores for a group of image data entries and, assuming a perfect match for each data entry, take the maximum of these maximum scores as the result value for the processed image data. This allows, for example, a probe being tested against a result value to determine whether the data entries associated with those values ​​potentially have sufficient scores to merit individual testing of each group of data entries used to determine the result value.

[0098] In some embodiments, each probe in the model can be a unit vector, and a particular pose of the model can be scored using a scoring method that involves calculating the squared dot product of the probe and the corresponding image data. This calculates the square of the product of the magnitude and cosine between them, resulting in a higher score for closer alignment. Using the squared dot product, if the probes match exactly, the score will be the square of the length of the data entry (e.g., vector). Therefore, on a data entry basis, the maximum score reflects the square of the length, which is the maximum possible score that a probe of any unit length can achieve. In such an example, each value in the processed image data is populated with the maximum squared value of the magnitude of the data entry (e.g., vector) found in the data entry's region in the input image data. This technique can therefore aggregate individual data entries in the image data (or smaller regions) with the maximum score for the entire region.

[0099] Referring to step 408, the processed image data can be tested as if it were the type of image data used to test the model (e.g., derived vector fields, e.g., 2D gradient fields, and / or normals, edges, 3D color fields as described above). For example, once the processed image data has been determined, the model can be tested for a particular pose by testing a probe of the model and testing the score (e.g., magnitude sum). When the probe is tested for a particular region, if the probe is anywhere within the region, this technique allows the system to determine that the score for the model cannot be higher than the value for that region, because that is the maximum value for the entire region, as discussed above. Therefore, the score cannot be higher for each data entry associated with the region, and therefore the associated value for the region is the upper limit for that region.

[0100] See steps 410-414, which ensure that while the poses tested can be sparse, the calculated score cannot be lower than the actual score of the best pose within the range of poses tested. For example, as discussed further below, if a region is configured to be an 8x8 set of data entries in the 2D image data, if the pose of the model does not reach a particular value threshold for the region, the system can skip testing the remaining values ​​for that region at the next level, as discussed further below in connection with FIG. 6. Thus, if the system tests a set of probes against a model and calculates a score that does not reach the threshold, the system can exclude all values ​​associated with the tested region from further testing. If the pose exceeds the threshold, the system can make a note of these associated data entries to reference later for further refinement. Further refinement can include, for example, performing method 400 with smaller region sizes and / or refined searches, as described herein.

[0101] Referring to step 418, the output of method 400 is a set of poses of the model that can achieve a score above a predetermined threshold. The range of poses that did not achieve a score above the threshold need not be reconsidered. In some embodiments, as described above, method 400 can be applied in a pyramidal scheme, with the output of each stage being the input to the next higher resolution stage. For example, a first stage can use regions of a particular size (e.g., 16 data entries, 8 data entries, etc.), a second stage can use regions of a smaller size (e.g., 12 data entries, 10 data entries, 4 data entries, etc.), and so on. Method 400 allows the machine vision system to perform higher-level searches on the processed image data, which allows the machine vision system to take larger steps when testing poses (e.g., moving multiple squares in each dimension instead of one square), essentially allowing multiple poses to be tested at once, as discussed further below.

[0102] A machine vision system can perform method 400 to process various types of image data, including 2D and / or 3D data. FIGS. 5-6 provide examples of processing 2D image data, and FIGS. 7-8 provide examples of processing 3D image data. Referring to FIG. 5, FIG. 5 illustrates an exemplary portion of two-dimensional image data 500 and associated processed image data 550 according to some embodiments. In this example, the two-dimensional image data 500 includes 2D data entries 502A-502N, collectively referred to as data entries 502 (e.g., cells of a grid). Each data entry 502 has an associated 2D vector denoted v1, v2, etc. (e.g., in this example, data entry 502A has vector v1, and data entry 502N has vector v4). As shown in FIG. 5, the system is configured to process the 2D image data using a 2x2 arrangement of four regions of data entries. Two of these regions are designated 504A and 504B. As illustrated by these exemplary regions, the system can be configured to iteratively pass through the 2D image to identify regions. In this example, as the system moves through each data entry (e.g., in a recursive left-to-right, top-to-bottom fashion), each region contains the current data entry as well as the data entry to the right, the data entry below, and the data entries between the data entry to the right and the data entry below.

[0103] Continuing with reference to FIG. 5, processed image data 550 includes cells having values ​​552A through 552N, collectively referred to as 552. The machine vision system is configured to execute a function that processes the data entries of each region to determine a result value for an associated cell in the processed image data. As discussed herein, the function can be a maximum score function that uses a vector of associated data entries in the region. For example, the system uses this function to process vectors v1, v2, v4, and v5 of associated data entries in region 504A to calculate a result value f1, which the system uses to determine the value of cell 552A. As another example, the system processes vectors v4, v5, v7, and v8 of associated data entries in region 504B to calculate a result value f7, which the system uses to determine the value of cell 552B.

[0104] FIG. 6 illustrates an exemplary exclusion of a set of poses of two-dimensional image data during a coarse search, according to some embodiments. FIG. 6 illustrates exemplary two-dimensional image data 500 from FIG. 5 and associated processed image data 550. As discussed herein, a score for a model is calculated based on multiple probes of the model. For illustrative purposes, FIG. 6 shows that one of the probes for the model, falling in cell 552A, is determined to have an insufficient score when testing the poses of the model. As cell 552A is excluded from consideration at the layer shown in processed image data 550 and the two-dimensional image data 500 moves up one layer, all poses corresponding to the location of the tested probe are excluded from further consideration (e.g., for a refined search operating on the remaining candidate locations in the two-dimensional image data 500).

[0105] 6, when testing at a layer represented by processed image data 550 (e.g., a layer aggregating four data entries for each value in next layer 500), if only one pose is tested at layer 550, the image processing system can step over a layer and essentially test a group of poses in the next layer 500. As shown in the example of FIGS. 5-6, if the system tests a model (with one of possibly many model probes that fall into cell 552A, as previously described) where f1 represents the maximum value for the four data entries in region 504A (the same applies to other probes tested against cells not shown), the system can determine that those poses will not receive a high enough score, and therefore none of the corresponding poses in two-dimensional data 500 will receive a high enough score. Thus, as shown, higher layer searching of the processed image data 550 allows the machine vision system to take larger steps when testing poses (e.g., moving two squares in each dimension instead of one square when using a 2x2 region as shown in Figure 5), essentially allowing multiple poses to be tested at once (e.g., four poses at once when using a 2x2 region).

[0106] 7 illustrates exemplary three-dimensional image data 700 and associated processed image data 750, according to some embodiments. The 3D image data 700 contains a set of 3D data entries 702A-702N, collectively referred to as data entries 702 (e.g., a 3D vector field). Each data entry 702 includes a v 11 , v 12 In this example, each vector in the first layer is v 1X and X refers to a specific data entry in the layer (for example, in this example, data entry 702A and data entry 702N are both in the first layer, so they are represented by vectors v 11 and vector v 147 , the system is configured to process the 3D image data using eight data entry regions that encompass four data entries from one layer and four data entries from the next layer. Two of these regions are shown as 704A and 704B, each encompassing four data entries from the first layer and four corresponding data entries in the second layer. As discussed herein, the system iteratively passes through the 3D image to identify the regions. In this example, as the system traverses each data entry (e.g., row, column, and depth) in the 3D image data 700, the system determines a new region that encompasses the current data entry, as well as the data entry to the right, the data entry below, the data entry between the data entry to the right and the data entry below, and the corresponding four data entries at the same location in the next layer.

[0107] Continuing with reference to FIG. 7, processed image data 750 includes cells 752A-752N having values, collectively referred to as cells 752. The machine vision system is configured to perform a function to process the data entries of each region to determine a resulting value for the associated cell in the processed image data. As discussed herein, the function can be a maximum score function using a vector of associated data entries in the region. For example, the system can use this function to find the vector v of associated data entries in region 704A. 11 , v 12 , v 14 , v 15 , v 21 , v 22 , v 24 and v 25 Processing the result f 11 which the system uses to determine the value of cell 752A. As another example, the system processes vectors v4, v5, v7, and v8 of the associated data entries in region 504B to calculate a result value f7 which the system uses to determine the value of cell 552B.

[0108] FIG. 8 illustrates an exemplary exclusion of a set of poses of three-dimensional image data during a coarse search, according to some embodiments. FIG. 8 illustrates the exemplary exclusion and associated processed image data 750 from FIG. 7. As discussed herein, a score for a model is calculated based on multiple model probes. For illustrative purposes, FIG. 8 shows only one of the model probes falling into cell 752A for illustrative purposes; however, as discussed herein, each of the model probes is tested against its associated cell and determined to have an insufficient pose score. With cell 752A excluded from consideration at the layer shown in processed image data 750, the three-dimensional image data 700 is moved up one layer, and all eight corresponding poses in region 704A are excluded from further consideration (e.g., for a refined search operating on the remaining candidate locations in the three-dimensional image data 700).

[0109] As shown in FIGS. 7 and 8, when applying this technique to 3D image data, the pose space can be quite large due to the number of dimensions. In some 3D images and / or 3D applications, only a small percentage of the large space of potential poses can encompass good candidates (e.g., the object occupies only a small portion of the image). As another example, in some 3D images and / or 3D applications, small rotations away from relevant regions can begin to yield very low scores. This technique can utilize an upper layer search (e.g., using the processed image data 750 shown in FIG. 7) to quickly eliminate a large set of poses that do not yield acceptable scores. Upper layer search allows the machine vision system to take larger steps when testing poses (e.g., moving two squares in each dimension instead of one when using a 2×2×2 region as shown in FIG. 7), essentially enabling multiple poses to be tested at once (e.g., eight poses at once when using a 2×2×2 region).

[0110] The inventors have determined that performing pattern matching using only a single field may not provide sufficient information. For example, some techniques use a single normal field to locate a probe on the surface of an object by trying various pose configurations and scoring each pose across the field. However, using only a surface normal field can be problematic, for example, when a scene contains one or more large areas with the same surface normal that resemble the trained object. For example, if a 3D model is trained on a book and the scene contains a table instead of the book, the book will have many matches across the table using only the surface normal vectors. Therefore, performing an initial coarse search for the approximate location of the model using a normal field may result in insufficient information in the field to eliminate the initial pose from consideration for further refinement. The inventors have developed a technical improvement to machine vision search techniques that uses additional information beyond a single field. As discussed further herein, this technique may include using additional information, such as information about crease edges, occlusion boundaries, color, intensity, and / or the like. The additional information can be stored in one or more separate fields for the search process (e.g., probe normal data can be matched to a normal field, and probe edge data can be matched to an edge field). The machine vision system can test each type of data and use multiple tests to determine a final score for a particular pose (e.g., by summing the individual scores). Using multiple fields allows additional information about the object to be used to improve the efficiency of the search process. For example, by searching both normal and edge information, poses that score high in the normal field but low in the edge field can be eliminated. As another example, this technique can improve the system's ability to search for specific types of objects, such as objects of the same shape. For example, while it may be difficult to search for a specific pose of a can, this technique can include other information about the can, such as color and / or reflectivity, to improve the search (e.g., the shape of the can alone may not be sufficient).

[0111] FIG. 9 shows an example method 900 for performing an initial search for a pose of a three-dimensional model, according to some embodiments. In step 902, a machine vision system stores a three-dimensional model containing a set of probes. In step 904, the system receives 3D data (e.g., a point cloud, a depth image, etc.) of an object containing a set of 3D data entries. In step 906, the system converts the three-dimensional data into two or more fields. The first field contains a first set of values ​​indicating a first characteristic of one or more associated data entries from the 3D data entries, respectively. The second field contains a second set of values ​​indicating a second characteristic of one or more associated data entries from the 3D data entries, respectively. In step 908, the system tests the pose of the 3D model against the set of fields to determine a score for the pose, which includes testing the set of probes against the set of fields. In step 910, the system determines whether the score reaches a predetermined threshold. If the score reaches the threshold, the method proceeds to step 912, where the pose is stored (e.g., for subsequent refinement). If the score does not reach the threshold, the method proceeds to step 914 to determine if there are more postures to test, if so the method returns to step 908, if not the method ends at step 916.

[0112] Referring to step 902, the model can be a trained model as discussed herein. Each probe can include one or more vectors. For example, a probe can represent a position vector (e.g., an (x, y, z) position and an (x, y, z) direction). A probe can represent, for example, normal data, edge data, intensity data, intensity gradient data, and / or other information. Normal data can include, for example, a point on a surface and its normal direction (e.g., a normal probe). Edge data can include, for example, data on a fold in an object or a fold edge of an object and a direction along the fold or fold (e.g., an edge probe). Intensity data can include information related to, for example, intensity, surface reflectance, color, albedo, etc. For example, intensity data can reflect information related to a grayscale image and / or a color image (e.g., coloring of objects, labels, and / or the like).

[0113] Referring to step 906, the first characteristic is different from the second characteristic. Step 906 therefore generates at least two different fields for the 3D data. Similar to the model probe, the values ​​in the fields can include various types of data such as surface normal data (e.g., normal vectors perpendicular to the surface of the object), edge boundary data (e.g., edges, creases, and / or other features in the image), intensity data, intensity gradient data, and / or the like.

[0114] In some embodiments, the technique includes converting the runtime 3D data into one or more dense 3D arrays, called fields. In some embodiments, the system generates a 3D array for each field. The 3D array is indexed using these indices, one for each dimension. The system can be configured to use the indices of the 3D array to include the x, y, and z locations of each value in the array. For example, the x, y, and z indices into the array can be x, y, and z locations and / or can be converted to x, y, and z locations using a transformation. Each value can include, for example, vectors of the same dimension (e.g., one or more). Each such vector can represent a point within or near the associated 3D data entry. The length of the vector can be zero (0), such as when no point is found within or near the associated 3D data entry (e.g., when the data entry is inside an object in the 3D point cloud).

[0115] As discussed herein, with reference to step 908, testing the poses can include transforming the trained probes according to each hypothesized pose. In some embodiments, to determine a score for a pose, the system can calculate the sum of the dot product of each probe and the associated value in the field. The probes are tested against a set of fields generated by the system. The system can be configured to calculate a similarity metric based on the probes' scores for the individual fields. For example, the system can be configured to average the individual scores for each field to determine an overall score for the pose. As another example, the system can be configured to perform more complex operations to combine the individual scores for each field, such as linear weighting (e.g., a*score1 + b*score2), non-linear weighting (e.g., min(score1, score2)), etc.

[0116] The technique can save poses that exceed a threshold for subsequent refinement. In some embodiments, the threshold is configured such that scores that exceed the threshold represent local peaks of associated scores in score space (e.g., pose space). For example, in some embodiments, in addition to checking whether a particular pose reaches a threshold, the system can analyze the score of a particular pose with respect to the scores of neighboring poses. The system can be configured to save a subset of poses if the subset achieves a higher score than its neighbors in pose space.

[0117] Techniques operating according to the principles described herein may be implemented in any suitable manner. The process and decision blocks of the flowcharts above represent steps and acts that may be included in algorithms that perform these various processes. The algorithms derived from these processes may be implemented as software integrated with and directing the operations of one or more special-purpose or general-purpose processors, as functionally equivalent circuitry such as digital signal processing (DSP) circuitry or application-specific integrated circuit devices (ASICs), or in any other suitable manner. It should be understood that the flowcharts contained herein do not depict the syntax or operations of any particular circuitry or any particular programming language or type of programming language. Rather, these flowcharts illustrate functional information that one skilled in the art may use to fabricate circuitry or implement computer software algorithms for processing particular devices that perform the techniques of the types described herein. Unless otherwise indicated herein, it should also be understood that the specific sequence of steps and / or acts depicted in each flowchart is merely illustrative of algorithms that may be implemented, and that implementations and embodiments of the principles described herein may vary.

[0118] Thus, in some embodiments, the techniques described herein may be embodied in computer-executable instructions implemented as software, including application software, system software, firmware, middleware, embedded code, or any other suitable type of computer code. Such computer-executable instructions may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may also be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine.

[0119] When the techniques described herein are embodied as computer-executable instructions, these computer-executable instructions can be implemented in any suitable manner, including as a number of utility functions, each providing one or more operations to complete the execution of an algorithm operating according to these techniques. However, an instantiated "utility function" is a structural element of a computer system that, when integrated with and executed by one or more computers, causes the one or more computers to perform a particular operational role. A utility function can be part or all of a software element. For example, a utility function may be implemented as a function of a process, as a separate process, or as other suitable processing unit. When the techniques described herein are implemented as multiple utility functions, each utility function may be implemented in a unique manner and need not all be implemented in the same manner. Furthermore, these utility functions may be executed in parallel and / or serially as desired, and may pass information between each other using shared memory of the computers on which they are executing, a message-passing protocol, or any other suitable manner.

[0120] Generally, utility functions include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of utility functions can be combined and distributed as desired in the systems in which they operate. In some implementations, one or more utility functions that perform the techniques herein can together form a complete software package. These utility functions may, in alternative embodiments, be adapted to interact with other unrelated utility functions and / or processes to implement a software program application.

[0121] Several exemplary utility functions have been described herein to perform one or more tasks. However, it should be understood that the described utility functions and divisions of tasks are merely illustrative of the types of utility functions that may implement the exemplary techniques described herein, and that embodiments are not limited to any particular number, division, or type of utility functions. In some implementations, all functionality may be implemented in a single utility function. Also, in some implementations, some of the utility functions described herein may be implemented together with other utility functions or separately (i.e., as a single unit or separate units), or some of these utility functions may not be implemented.

[0122] Computer-executable instructions implementing the techniques described herein (whether implemented as one or more convenience features or otherwise) are, in some embodiments, encoded on one or more computer-readable media to provide the media with functionality. Computer-readable media include magnetic media such as hard disk drives, optical media such as compact discs (CDs) or digital versatile discs (DVDs), persistent or non-persistent solid-state memory (e.g., flash memory, magnetic RAM), or other suitable storage media. Such computer-readable media may be implemented in any suitable manner. As used herein, a "computer-readable medium" (also referred to as a "computer-readable storage medium") refers to a tangible storage medium. A tangible storage medium is non-transitory and has at least one physical structural element. As used herein, a "computer-readable medium" refers to at least one physical structural element that has at least one physical characteristic that can be changed in some way during the process of creating a medium with embedded information, recording information on the medium, or encoding the medium with information. For example, the magnetization state of a portion of the physical structure of the computer-readable medium can be changed during the recording process.

[0123] Additionally, some of the technologies described above involve storing information (e.g., data and / or instructions) in a particular manner for use by those technologies. In some implementations of these technologies (e.g., implementations in which the technologies are embodied as computer-executable instructions), the information is encoded on a computer-readable storage medium. Where particular structures are described herein as advantageous formats for storing this information, these structures can be used to provide a physical organization of the information when encoded on the storage medium. These advantageous structures can then impart functionality to the storage medium by affecting the operation of one or more processors that interact with the information, for example, by increasing the efficiency of computer operations performed by the processors.

[0124] In some implementations (but not all implementations) in which the techniques may be embodied as computer-executable instructions, these instructions may be executed by one or more suitable computing devices operating in any suitable computer system, or one or more computing devices (or one or more processors of one or more computing devices) may be programmed to execute the computer-executable instructions. A computing device or processor may be programmed to execute the instructions when the instructions are stored in a manner accessible to the computing device or processor, such as a data store (e.g., on-chip cache or instruction registers, computer-readable storage accessible via a bus, computer-readable storage media accessible via one or more networks, and media accessible by the device / processor, etc.). Utility functions containing these computer-executable instructions may be integrated with and direct the operation of a single general-purpose programmable digital computing device, a cooperative system of two or more general-purpose computing devices that share processing power and jointly perform the techniques described herein, a single computing device or a cooperative system of computing devices (co-located or geographically distributed) dedicated to performing the techniques described herein, one or more field-programmable gate arrays (FPGAs) for performing the techniques described herein, or any other suitable system.

[0125] A computing device may include at least one processor, a network adapter, and a computer-readable storage medium. The computing device may be, for example, a desktop or laptop personal computer, a personal digital assistant (PDA), a smartphone, a mobile phone, a server, or any other suitable computing device. The network adapter may be any suitable hardware and / or software that enables the computing device to communicate wired and / or wirelessly with any other suitable computing device over any suitable computing network. A computing network may include wireless access points, switches, routers, gateways, and / or other networking equipment, as well as any suitable wired and / or wireless communication medium for exchanging data between two or more computers, including the Internet. The computer-readable medium may be adapted to store data to be processed and / or instructions to be executed by the processor. The processor enables the processing of data and execution of instructions. The data and instructions may be stored on the computer-readable storage medium.

[0126] A computing device may further include one or more components and peripherals, including input / output devices. These devices can be used, among other things, to provide a user interface. Examples of output devices that can be used to provide a user interface include a printer or display screen for visually displaying output, and a speaker or other sound-generating device for audibly displaying output. Examples of input devices that can be used in a user interface are keyboards, pointing devices such as mice or touchpads, and digitizing tablets. As another example, a computing device can receive input information via voice recognition or other audible formats.

[0127] The embodiments described above illustrate techniques implemented with circuits and / or computer-executable instructions. It should be understood that some embodiments may be in the form of a method, of which at least one example is provided. Operations performed as part of a method may be ordered in any suitable manner. Thus, while illustrated as sequential operations, embodiments may be configured to perform operations in an order different from that illustrated, including performing some operations simultaneously.

[0128] Various aspects of the above-described embodiments may be used alone, in combination, or in various configurations not specifically discussed in the above-described embodiments, and therefore are not limited in their application to the details and arrangements of components set forth in the foregoing description or illustrated in the drawings. For example, aspects described in one embodiment may be combined in any manner with aspects described in other embodiments.

[0129] The use of ordinal numbers such as "first," "second," "third," etc. to modify a claim element in a claim does not, in itself, imply a priority, precedence, or ranking of one claim element over another, or a chronological order in which method operations are performed, but is merely used as a descriptive term to distinguish a claim element having a particular name from another element having the same name (except for the use of ordinal numbers) to distinguish the claim elements.

[0130] Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "having," "having," "including," "involving," and variations thereof herein is meant to encompass the items listed thereafter, and equivalents thereof, as well as additional items.

[0131] The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Thus, any embodiments, implementations, processes, features, etc. described herein as exemplary are to be understood as illustrative examples and not as preferred or advantageous examples, unless expressly stated otherwise.

[0132] While several aspects of at least one embodiment have been described above, it should be understood that various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the principles described herein. Accordingly, the foregoing description and drawings are by way of example only.

Claims

1. 1. A computerized method for testing the pose of a model in three-dimensional data, comprising: The method includes receiving three-dimensional data for an object, the three-dimensional data including a set of data entries; The method includes converting the three-dimensional data into a field containing a set of cells each having an associated value, the converting including determining, for each cell value, representative data based on one or more data entries from the set of data entries of the three-dimensional data; and The method includes testing a pose of the model with a field to determine a score for the pose. The above method.

2. The method of claim 1 , wherein the step of converting three-dimensional data into a field comprises generating a three-dimensional array of a set of values.

3. The method of claim 1 , wherein the step of converting three-dimensional data into a field includes generating a dense grid, the dense grid containing data for each cell of the grid.

4. the set of data entries for three-dimensional data includes a list of points; determining representative data for each cell based on the one or more data entries includes determining a vector based on one or more associated points in a list of points; The method of claim 1.

5. determining the vector for one or more associated points includes determining based on a list of points, the cells being associated with an interior portion of an object; and determining the vector includes setting the vector to zero; The method of claim 4.

6. The step of testing the pose of the model by the field and determining the score includes:

6. The method of claim 5, comprising testing a set of probes of a model against a field to determine a score, said determining comprising summing a dot product of each probe and an associated vector in the field.

7. said converting the three-dimensional data into a field containing a set of cells each having an associated value, including determining a representative vector for each cell value; generating an accumulation matrix comprising calculating a cross product of each vector of a set of vectors with itself, where the set of vectors is data determined based on one or more data entries from a set of data entries of the three-dimensional data; and the transforming step includes extracting eigenvectors, eigenvalues, or both from the cumulative matrix to determine a representative vector; The method of claim 1.

8. 1. A system for testing a pose of a model in three-dimensional data, the system comprising: one or more processors; the processor is configured to receive three-dimensional data of an object, the three-dimensional data comprising a set of data entries; the processor is further configured to convert the three-dimensional data into a field containing a set of cells each having an associated value, the converting including determining, for each cell value, representative data based on one or more data entries from the set of data entries of the three-dimensional data; and the processor is further configured to test the pose of the model with the field to determine a score for the pose; The above system.

9. The system of claim 8 , wherein the converting three-dimensional data into a field comprises generating a three-dimensional array of a set of values.

10. 9. The system of claim 8, wherein the converting the three-dimensional data into a field includes generating a dense grid, the dense grid containing data for each cell of the grid.

11. the set of data entries for three-dimensional data includes a list of points; determining representative data based on one or more data entries for each cell includes determining a vector based on one or more associated points in a list of points; The system of claim 8.

12. Determining the vector for one or more associated points includes determining based on a list of points, the cells being associated with an interior portion of an object; and determining the vector includes setting the vector to zero; The system of claim 11.

13. Testing the pose of the model against the field to determine the score includes testing a set of probes of the model against the field to determine a score, the determining including a dot product of each probe and a sum of the associated vector in the field. The system of claim 12.

14. said converting the three-dimensional data into a field containing a set of cells each having an associated value, including determining a representative vector for each cell value, includes generating an accumulation matrix including calculating a cross product of each vector of the set of vectors with itself, wherein the set of vectors is determined based on one or more data entries from the set of data entries of the three-dimensional data; and the converting includes extracting eigenvectors, eigenvalues, or both from the cumulative matrix to determine a representative vector; The system of claim 8.

15. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the following operations: the operation is receiving three-dimensional data of an object, the three-dimensional data including a set of data entries; The operations further include converting the three-dimensional data into a field including a set of cells each having an associated value, the converting including determining representative data for each of the cell values ​​based on one or more data entries from the set of data entries of the three-dimensional data; and The operation further comprises testing the pose of the model with the field to determine a score for the pose. The non-transitory computer-readable storage medium.

16. 16. The non-transitory computer-readable storage medium of claim 15, wherein the act of converting the three-dimensional data to a field comprises generating a three-dimensional array of a set of values.

17. 16. The non-transitory computer-readable storage medium of claim 15, wherein the act of converting the three-dimensional data to a field includes generating a dense grid, the dense grid containing data for each of the cells of the grid.

18. 16. The non-transitory computer-readable storage medium of claim 15, wherein the set of data entries for the three-dimensional data includes a list of points, and determining representative data based on one or more data entries for each of the cells includes determining a vector based on one or more associated points in the list of points.

19. determining a vector for the one or more associated points includes determining the vector based on a list of points, the cells being associated with an interior portion of an object; determining the vector further comprises setting the vector to zero; 20. The non-transitory computer-readable storage medium of claim 18.

20. converting the three-dimensional data into a field containing a set of cells each having an associated value, including determining a representative vector for each cell value, includes generating an accumulation matrix including calculating a cross product of each vector of the set of vectors with itself, wherein the set of vectors is determined based on one or more data entries from the set of data entries of the three-dimensional data; and converting to the field includes extracting eigenvectors, eigenvalues, or both from the cumulative matrix to determine representative vectors; 16. The non-transitory computer-readable storage medium of claim 15.

21. 1. A computerized method for testing a pose of a model against image data, comprising: The method includes receiving image data of an object, the image data including a set of data entries; The method further includes determining a set of regions of the image data, each region in the set of regions encompassing an associated set of adjacent data entries in a set of data entries; The method further includes generating processed image data, the processed image data including a set of cells each having an associated value; generating the processed image data includes, for each region in the set of regions, determining a maximum possible score for each data entry in an associated set of adjacent data entries from the image data, and setting one or more values ​​in a set of values ​​based on the determined maximum possible score; and The method further includes testing the pose of a model using the processed image data. The method.

22. receiving the image data includes receiving 2D image data, each data entry containing a 2D vector; determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on a 2D vector within a region associated with the value; 22. The method of claim 21.

23. Testing the pose of the model using the processed data includes: determining that the pose score does not exceed a predetermined threshold and testing a plurality of probes of the model against associated scalar values ​​of the processed data; and excluding from further testing the set of poses associated with each region used to determine the associated scalar value; 23. The method of claim 22, comprising:

24. receiving the image data includes receiving 3D image data, each data entry including a 3D vector; determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on a 3D vector within a region associated with the value; 22. The method of claim 21.

25. Testing the pose of the model using the processed data includes: determining that the pose score does not exceed a predetermined threshold and testing a plurality of probes of the model against associated scalar values ​​of the processed data; and excluding from further testing the set of poses associated with each region used to determine the associated scalar value; 25. The method of claim 24, comprising:

26. Transforming the three-dimensional data into a second field containing a second set of cells each associated with a second value includes: determining, for each second cell value, representative data based on one or more data entries from the set of data entries of the three-dimensional data; and testing the pose of the model with a second field based on testing the pose of the model with the field; 22. The method of claim 21, comprising:

27. 1. A system for testing a pose of a model against image data, the system comprising: one or more processors; the processor is configured to receive image data of an object, the image data including a set of data entries; the processor is further configured to determine a set of regions of the image data, each region in the set of regions encompassing an associated set of adjacent data entries in the set of data entries; the processor is further configured to generate processed image data; the processed image data includes a set of cells each having an associated value; generating the processed image data includes, for each region in the set of regions, determining a maximum possible score for each data entry in an associated set of adjacent data entries from the image data, and setting one or more values ​​in a set of values ​​based on the determined maximum possible score; and the processor is further configured to test the pose of the model using the processed image data. The system.

28. receiving the image data includes receiving 2D image data, each data entry containing a 2D vector; determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on a 2D vector within a region associated with the value; 28. The system of claim 27.

29. Testing the pose of the model using the processed data includes: determining that the pose score does not exceed a predetermined threshold, and testing a plurality of probes of the model against associated scalar values ​​of the processed data; and Excluding from further testing the set of poses associated with each region used to determine the associated scalar value.

30. The system of claim 28, comprising:

30. receiving the image data includes receiving 3D image data, each data entry containing a 3D vector; determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on a 3D vector within a region associated with the value; 28. The system of claim 27.

31. Testing the pose of the model using the processed data includes: determining that the pose score does not exceed a predetermined threshold, and testing a plurality of probes of the model against associated scalar values ​​of the processed data; and excluding from further testing the set of poses associated with each region used to determine the associated scalar value; 31. The system of claim 30, comprising:

32. The one or more processors further include: converting the three-dimensional data into a second field containing a second set of cells each associated with a second value, and determining, for each second cell value, representative data based on one or more data entries from the set of data entries; and testing the pose of the model with a second field based on testing the pose of the model with the field; 28. The system of claim 27, configured to:

33. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the following operations: the operation is receiving image data of an object, the image data including a set of data entries; the operation further being an operation of determining a set of regions of the image data, each region in the set of regions encompassing an associated set of adjacent data entries in the set of data entries; the operation further comprises generating processed image data; the processed image data includes a set of cells each having an associated value; generating the processed image data includes, for each region in the set of regions, determining a maximum possible score for each data entry in an associated set of adjacent data entries from the image data, and setting one or more values ​​in a set of values ​​based on the determined maximum possible score; and the operation further comprising testing the pose of the model using the processed image data. The non-transitory computer-readable storage medium.

34. the act of receiving image data includes receiving 2D image data, each data entry containing a 2D vector; determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on a 2D vector within a region associated with the value; 34. The non-transitory computer-readable storage medium of claim 33.

35. Testing the pose of the model using the processed data includes: determining that the pose score does not exceed a predetermined threshold, and testing a plurality of probes of the model against associated scalar values ​​of the processed data; and excluding from further testing the set of poses associated with each region used to determine the associated scalar value; 35. The non-transitory computer-readable storage medium of claim 34, comprising:

36. the act of receiving image data includes receiving 3D image data, each data entry containing a 3D vector; Determining a maximum possible score for each processed image data value of the set of values ​​includes determining a scalar value based on a 3D vector within a region associated with the value.

34. The non-transitory computer-readable storage medium of claim 33.

37. Testing the pose of the model using the processed data includes: determining that the pose score does not exceed a predetermined threshold, and testing a plurality of probes of the model against associated scalar values ​​of the processed data; and Excluding from further testing the set of poses associated with each region used to determine the associated scalar value.

37. The non-transitory computer-readable storage medium of claim 36, comprising:

38. The instructions further include causing the one or more processors to: an operation of transforming the three-dimensional data into a second field including a second set of cells each associated with a second value, the operation including determining, for each second cell value, representative data based on one or more data entries from the set of data entries of the three-dimensional data; and testing the pose of the model with a second field based on testing the pose of the model with the field; 36. The non-transitory computer-readable storage medium of claim 35, configured to cause execution of:

39. 1. A computerized method for testing the posture of a three-dimensional model, comprising: The method includes storing a three-dimensional model, the three-dimensional model including a set of probes; The method further includes receiving three-dimensional data of the object, the three-dimensional data including a set of data entries; The method further includes transforming the three-dimensional data into a set of fields; the converting includes generating a first field containing a first set of values, each value of the first set of values ​​indicating a first characteristic of one or more associated data entries from a set of data entries; the converting further includes generating a second field containing a second set of values, each value of the second set of values ​​indicating a second characteristic of one or more associated data entries from the set of data entries, the first characteristic being different from the second characteristic; and The method further includes testing a pose of the three-dimensional model with the set of fields, the testing including testing a set of probes against the set of fields, thereby determining a score for the pose. The method.

40. 40. The method of claim 39, wherein generating the first and second fields includes generating a three-dimensional array for each field, each of the three-dimensional arrays consisting of a set of three indices containing an index for each dimension, each of the three-dimensional arrays including an x-position, a y-position, and a z-position of each of the first and second values ​​related by the set of three indices.

41. 40. The method of claim 39, wherein the probe, the first set of values ​​of a first field, and the second set of values ​​of a second field comprise surface normal data, edge boundary data, intensity data, or some combination thereof.

42. 40. The method of claim 39, wherein testing a posture to determine a score for the posture comprises summing a value associated with a dot product for each probe.

43. moreover, testing a plurality of postures to determine a plurality of associated scores; determining which poses of the plurality of poses have a score above a predetermined threshold to generate a set of poses; storing the set of poses for subsequent processing; 40. The method of claim 39, comprising:

44. 44. The method of claim 43, wherein each pose in the set of poses represents a local peak in an associated score, the method further comprising refining the set of poses to determine a best pose of the model.

45. 1. A system for determining parameters for image acquisition, comprising one or more processors, the processor is configured to store a three-dimensional model, the three-dimensional model including a set of probes; the processor is further configured to receive three-dimensional data of the object, the three-dimensional data comprising a set of data entries; the processor is further configured to transform the three-dimensional data into a set of fields; the converting includes generating a first field containing a first set of values, each value in the first set of values ​​indicating a first characteristic of one or more associated data entries from a set of data entries; The converting further includes generating a second field containing a second set of values, each value in the second set of values ​​indicating a second characteristic of one or more associated data entries from the set of data entries, the second characteristic being different from the first characteristic; and the processor is further configured to test the pose of the three-dimensional model against the set of fields, the testing including testing a set of probes against the set of fields, thereby determining a score for the pose. The system.

46. 46. ​​The system of claim 45, wherein generating the first and second fields includes generating a three-dimensional array for each field, each of the three-dimensional arrays consisting of a set of three indices containing an index for each dimension, each of the three-dimensional arrays including an x-position, a y-position, and a z-position of each of the first and second values ​​related by the set of three indices.

47. 46. ​​The system of claim 45, wherein the probe, the first set of values ​​of a first field, and the second set of values ​​of a second field comprise surface normal data, edge boundary data, intensity data, or some combination thereof.

48. 46. ​​The system of claim 45, wherein testing a pose to determine a score for the pose comprises summing a value associated with a dot product for each probe.

49. The one or more processors further Test multiple postures to determine multiple associated scores, determining which poses of the plurality of poses have a score above a predetermined threshold to generate a set of poses; and Save the set of poses for subsequent processing, 46. ​​The system of claim 45, configured to:

50. 50. The system of claim 49, wherein each pose in the set of poses represents a local peak in an associated score, and the processor is configured to further refine the set of poses to determine a best pose for the model.

51. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the following operations: the operation is to save a three-dimensional model, the three-dimensional model including the set of probes; the operation further comprises receiving image data of the object, the image data including a set of data entries; the act further comprising converting the three-dimensional data into a set of fields; the converting includes generating a first field containing a first set of values, each value in the first set of values ​​indicating a first characteristic of one or more associated data entries from a set of data entries; The converting further includes generating a second field containing a second set of values, each value in the second set of values ​​indicating a second characteristic of one or more associated data entries from the set of data entries, the second characteristic being different from the first characteristic; and the operation further includes testing a pose of the three-dimensional model with a set of fields, the testing including testing a set of probes against the set of fields, thereby determining a score for the pose. The non-transitory computer-readable storage medium.

52. 52. The non-transitory computer-readable storage medium of claim 51 , wherein generating the first and second fields includes generating a three-dimensional array for each field, each of the three-dimensional arrays consisting of a set of three indices containing an index for each dimension, each of the three-dimensional arrays including an x-position, a y-position, and a z-position of each of the first and second values ​​related by the set of three indices.

53. 52. The non-transitory computer-readable storage medium of claim 51 , wherein the probe, the first set of values ​​of a first field, and the second set of values ​​of a second field comprise surface normal data, edge boundary data, intensity data, or some combination thereof.

54. 52. The non-transitory computer-readable storage medium of claim 51, wherein the act of testing a pose to determine a score for the pose comprises summing a value associated with a dot product for each probe.

55. Further, the instructions may be transmitted to the one or more processors. the act of testing multiple postures to determine multiple associated scores; determining which poses of the plurality of poses have a score above a predetermined threshold to generate a set of poses; and saving a set of poses for subsequent processing; 52. The system of claim 51, wherein the system causes the execution of the following:

56. 56. The system of claim 55, wherein each pose in the set of poses represents a local peak in an associated score, and the instructions further cause the one or more processors to perform operations of refining the set of poses to determine a best pose for the model.

57. 1. A computerized method for testing a pose of a three-dimensional model against three-dimensional data, comprising: The method includes the steps of storing a three-dimensional model including a set of probes, each probe including data for an associated portion of the three-dimensional model; The method further comprises receiving three-dimensional data of the object, the three-dimensional data including a set of data entries; The method further includes converting the three-dimensional data into a set of fields, including generating a first field and generating a second field: the first field includes a first set of values, each value in the first set of values ​​indicating a first characteristic of an associated one or more data entries from the set of data entries; the second field includes a second set of values, each value in the second set of values ​​indicating a second characteristic of an associated one or more data entries from the set of data entries, the second characteristic being different from the first characteristic; The method further includes testing a pose of the three-dimensional model with the set of fields and determining a score for the pose based on the set of probes and associated values ​​of the first, second, or both fields; The computerized method.

58. 58. The method of claim 57, wherein determining a score for a posture based on the set of probes and associated values ​​of the first, second, or both fields comprises summing dot products of each probe in the set of probes and associated values ​​of the first, second, or both fields.

59. 58. The method of claim 57, wherein generating the first and / or second fields comprises generating a three-dimensional array for each field, each three-dimensional array including a set of three indices, one for each dimension, each said three-dimensional array representing the x, y, z position of each associated value in the first and / or second set of values.

60. 58. The method of claim 57, wherein the set of probes, the first set of values ​​of the first field, and the second set of values ​​of the second field comprise surface normal data, edge boundary data, and / or intensity data.

61. 58. The method of claim 57, further comprising testing a plurality of poses to determine a plurality of associated scores; determining one or more of the plurality of poses that include a score above a predetermined threshold to generate a set of poses; and storing the set of poses for subsequent processing.

62. each pose in the set of poses represents a local peak in a plurality of the associated scores; The method further includes refining the set of poses to determine a best pose of the three-dimensional model.

62. The method of claim 61.

63. 58. The method of claim 57, wherein testing the pose of the three-dimensional model with the set of fields to determine a score for the pose comprises testing the set of probes against the set of fields.

64. Testing the set of probes against the set of fields includes: determining a first score for the set of probes tested against the first set of values ​​of the first field; determining a second score for the set of probes tested against the second set of values ​​of the second field; and determining a score for the pose based on the first and second scores; 64. The method of claim 63, comprising:

65. 1. A system for testing a pose of a three-dimensional model against three-dimensional data, the system comprising: one or more processors; the processor is configured to store a three-dimensional model including a set of probes, each probe including data for an associated portion of the three-dimensional model; The processor is further configured to receive three-dimensional data of an object, the three-dimensional data including a set of data entries; The processor is further configured to convert the three-dimensional data into a set of fields, comprising generating a first field and generating a second field: the first field includes a first set of values, each value in the first set of values ​​indicating a first characteristic of an associated one or more data entries from the set of data entries; the second field includes a second set of values, each value in the second set of values ​​indicating a second characteristic of an associated one or more data entries from the set of data entries, the second characteristic being different from the first characteristic; The processor is further configured to test a pose of the three-dimensional model with the set of fields and determine a score for the pose based on the set of probes and associated values ​​of the first, second, or both fields; The system.

66. 66. The system of claim 65, wherein determining a score for a posture based on the set of probes and associated values ​​of the first, second, or both fields comprises summing dot products of each probe in the set of probes and associated values ​​of the first, second, or both fields.

67. 66. The system of claim 65, wherein generating the first and / or second fields comprises generating a three-dimensional array for each field, each three-dimensional array including a set of three indices, one for each dimension, each three-dimensional array representing the x, y, z location of each associated value in the first and / or second set of values.

68. 66. The system of claim 65, wherein the set of probes, the first set of values ​​of the first field, and the second set of values ​​of the second field comprise surface normal data, edge boundary data, and / or intensity data.

69. 66. The system of claim 65, wherein the processor is further configured to test a plurality of poses to determine a plurality of associated scores, determine one or more of the plurality of poses that include a score above a predetermined threshold to generate a set of poses, and store the set of poses for subsequent processing.

70. each pose in the set of poses represents a local peak in a plurality of the associated scores; the processor is further configured to refine the set of poses to determine a best pose of the three-dimensional model.

70. The system of claim 69.

71. At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform the following operations: the operation is to save a three-dimensional model including a set of probes, each probe including data for an associated portion of the three-dimensional model; The operation further includes receiving three-dimensional data of an object, the three-dimensional data including a set of data entries; The operations further include converting the three-dimensional data into a set of fields, including generating a first field and a second field: the first field includes a first set of values, each value in the first set of values ​​indicating a first characteristic of an associated one or more data entries from the set of data entries; the second field includes a second set of values, each value in the second set of values ​​indicating a second characteristic of an associated one or more data entries from the set of data entries, the second characteristic being different from the first characteristic; The operations further include testing a pose of the three-dimensional model with the set of fields and determining a score for the pose based on the set of probes and associated values ​​of the first, second, or both fields; The non-transitory computer-readable storage medium.

72. 72. The non-transitory computer-readable storage medium of claim 71 , wherein the act of determining a score for a posture based on the set of probes and associated values ​​of the first, second, or both fields comprises summing dot products of each probe in the set of probes and associated values ​​of the first, second, or both fields.

73. 72. The non-transitory computer-readable storage medium of claim 71, wherein generating the first and / or second fields comprises generating a three-dimensional array for each field, each three-dimensional array including a set of three indices, one for each dimension, each three-dimensional array representing an x, y, z position of each associated value in the first and / or second set of values.

74. 72. The non-transitory computer-readable storage medium of claim 71 , wherein the set of probes, the first set of values ​​of the first field, and the second set of values ​​of the second field comprise surface normal data, edge boundary data, and / or intensity data.

75. 72. The non-transitory computer-readable storage medium of claim 71, wherein the operations further include testing a plurality of poses to determine a plurality of associated scores; determining one or more of the plurality of poses that include a score above a predetermined threshold to generate a set of poses; and storing the set of poses for subsequent processing.

76. each pose in the set of poses represents a local peak in a plurality of the associated scores; the operation further comprises refining the set of poses to determine a best pose of the three-dimensional model.

76. The non-transitory computer-readable storage medium of claim 75.

Images