Feature detection by deep learning and vector field estimation

Abstract

A system and method for extracting features from a 2D image of an object using a deep learning neural network and a vector field estimation process. The method includes extracting a plurality of possible feature points, generating a mask image defining pixels in which the object is located in the 2D image, and generating a vector field image for each extracted feature point including an arrow pointing to the extracted feature point. The method also includes generating a vector intersection image by identifying intersection points of arrow crossings for each combination of two pixels in the 2D image. The method assigns a score to each intersection point according to a distance of each pixel of each combination of two pixels to the intersection point and generates a point voting image identifying a feature location from a plurality of clustered points.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to systems and methods for extracting features from images, and more particularly, to systems and methods for extracting features from images using deep learning neural networks and vector field estimation processes. BACKGROUND

[0002] Robots perform multiple tasks including pick and place operations, where the robot picks an object and moves the object from one location, such as a conveyor belt, to another location, such as a collection bin, where the position and orientation of the object, referred to as the 3D pose of the object, is slightly different. Thus, in order for the robot to effectively pick the object, the robot typically needs to know the 3D pose of the object. Various techniques to determine the 3D pose of the object for this purpose use a 3D camera that obtains a 3D image of the object in real time, where the 3D image identifies distance measurements to many points in the field of view of the camera, which are sent to the robot controller. These techniques can use structured light to construct a 3D map of the scene and then search for the object in the map by identifying 3D features of the object. Using a 3D camera to determine the 3D pose of the object for robotic applications has proven to be effective. However, 3D cameras are expensive devices and typically run slower than expected.

[0003] One technique known in the art to obtain the 3D pose of an object for robotic applications employs a 2D camera. This technique takes a single 2D color image of the scene and processes the image using a fully convolutional architecture. An algorithm divides the image into a 2D grid comprising SxS cells and identifies objects in the cells. The algorithm projects a bounding box around each object in the image and predicts the 2D locations of the eight corners of the box. However, this technique for predicting the corners of the bounding box has limitations in accuracy and low robustness to occlusions.

[0004] Some of the processes mentioned above for determining the 3D pose of an object employ artificial intelligence (AI), which is a part of computer science that employs algorithms that allow software applications to learn from their environment and make decisions from it to achieve a specific result. Machine learning is a part of AI that employs software applications that acquire their own knowledge by analyzing large amounts of raw input data in an iterative manner to extract patterns from the data and allow the software applications to learn to perform a task without being specifically programmed to perform that task. Deep learning is a particular type of machine learning that provides stronger learning performance by representing a particular real-world environment as a hierarchy of increasingly complex concepts.

[0005] Deep learning often employs a software structure that includes several layers of neural networks that perform non-linear processing, where each successive layer receives the output from the previous layer. Typically, these layers include an input layer that receives raw data from a sensor, a number of hidden layers that extract abstract features from the data, and an output layer that identifies particular things based on the feature extraction from the hidden layers. The neural network includes neurons or nodes, each of which has a "weight" that is multiplied by the input to the node to obtain a probability that something is correct. More specifically, each node has a weight that is a floating point number that is multiplied by the input to the node to generate the output of the node, which is some proportion of the input. The weights are initially "trained" or set by having the neural network analyze a set of known data under supervised processing and by minimizing a cost function to allow the network to obtain the highest probability of the correct output.

[0006] Deep learning neural networks are often used to provide image feature extraction and transformation for visual detection and classification of objects in images, where a video or image stream can be analyzed by the network to identify and classify objects and learn through the process to better discern objects. Thus, in these types of networks, the system can use the same processing configuration to detect certain objects and different classifications of those objects based on how the algorithm learns to identify the objects. SUMMARY

[0007] The following discussion discloses and describes a system and method for extracting multiple features from a 2D image of an object using a deep learning neural network and vector field estimation processing. The method includes extracting a plurality of possible feature points on the object from the 2D image, generating a mask image that defines pixels in the 2D image in which the object is located, and generating a separate vector field image for each extracted feature point, the vector field image including arrows having an x-direction value and a y-direction value for each pixel of the 2D image, where the direction of each arrow is toward the extracted feature point in the 2D image, and where extracting the possible feature points, generating the mask image, and generating the vector field image are performed in a deep learning neural network. The method also includes generating a vector extraction image by multiplying pixels in the mask image with corresponding pixels in the vector field image such that those arrows in the 2D image that are not associated with pixels having a portion of the object are discarded, and generating a vector intersection image by identifying intersection points of arrow intersections for each combination of two pixels in the 2D image. The method assigns a score to each intersection point according to the distance of each pixel of each combination of two pixels from the intersection point, generates a score grid, and generates a point voting image that identifies a feature location from a plurality of clustered points.

[0008] Additional features of the disclosure will become apparent from the following description of the drawings and claims, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a schematic diagram of a robotic system including a robot that picks up objects from a conveyor belt and places them in a bin, where the system employs a 2D camera for taking 2D images of the objects and a controller for computing 3D poses of the objects using the 2D images;

[0010] Figure 2 is a flowchart showing a process for obtaining a dataset from objects using a 2D camera to train a neural network;

[0011] Figure 3 is a schematic diagram of a neural network including a plurality of nodes, each node having adjustable weights;

[0012] Figure 4 is a flowchart showing a learning-based neural network process for using a trained neural network to estimate 3D poses of objects using 2D images of the objects and the neural network;

[0013] Figure 5 is a schematic diagram of a perspective-n-point (PnP) process for determining a 3D pose estimate of an object in the process shown in Figure 4

[0014] Figure 6 is a schematic diagram of a camera system using a plurality of cameras to image an object;

[0015] Figure 7 is a block diagram of a system for estimating 3D poses of objects using a plurality of 2D cameras;

[0016] Figure 8 is a schematic diagram of a PnP process for estimating a 3D pose estimate of an object in the system shown in Figure 7

[0017] Figure 9 is a flowchart showing a vector field estimation process for predicting feature points on an object. DETAILED DESCRIPTION

[0018] The following discussion of embodiments of the disclosure directed to systems and methods for extracting features from images using deep learning neural networks and vector field estimation processes is merely exemplary in nature and is in no way intended to limit the application or its applications or uses. For example, the systems and methods have applications for determining the position and orientation of objects grasped by a robot. However, the systems and methods can have other applications.

[0019] Figure 1 ​​is a schematic view of a robotic system 10 that includes a robot 12 that picks up objects 14 from a conveyor 16 and places them in a collection bin 18. The system 10 is intended to represent any type of robotic system that can benefit from the discussion herein, where the robot 12 can be any robot suitable for the purpose. As the objects 14 move down the conveyor 16, each object 14 has a different orientation and position on the conveyor 16 relative to the other objects 14. In order for the robot 12 to effectively grasp and pick up the objects 14, it is necessary to be able to place an end effector 20 on the robot 12 in the proper position and orientation before grasping the object 14. To accomplish this, the system 10 includes a 2D camera 22 that is positioned at a desired location relative to the objects 14, which provides 2D images to a robot controller 24 that controls the robot 12 to move the end effector 20. As will be discussed in detail below, the robot controller 24 employs an algorithm to convert the 2D images from the camera 22 into a 3D pose of the object 14 in order to properly direct the end effector 20.

[0020] As will be discussed, in order for the controller 24 to accurately determine the 3D pose of the object 14, in one embodiment a learning-based neural network is employed in the algorithm and software running in the controller 24 that extracts features of the object, where the neural network needs to be trained with data obtained from representative objects. Figure 2 is a flowchart 30 that illustrates a process for obtaining image data to be used to train the neural network from objects 32 that are the same as the objects 14 in the system 10 using a 2D camera 40. The objects 32 are placed on a marker board 34 with an appropriate number of markers 36, and the offset distance between the markers 34 and the objects 32, represented by lines 38, is measured, where the markers 36 have a special detectable design that will be used to identify the true or actual position of the objects 32, with only one marker 36 shown on the board 34.

[0021] The camera 40 provides one or more 2D images 42 of the marker board 34 at different angles, here four images. The process then performs a projection on the images 42 to determine the rotation and position of one of the various markers 36 identified by the coordinate frame 44. The algorithm then adds the previously measured offset between the markers 36 and the object 32 to the rotation and position of the markers 36 to determine the rotation and position of the object 32. Once the true position of the object 32 is determined, a known feature point extraction process can be used to calculate the position of feature points 46 on the object 32, such as corners, where a minimum of four feature points are needed. An algorithm employing, for example, a Gaussian distribution process looks at each feature point 46 identified in the images 42 and generates a "heat map" 48 of that point 46, where each heat map 48 identifies by color intensity the probability that one of the various feature points of the object 32 exists at a certain location. For example, locations in the images 42 where a feature point is thought to exist, such as in area 50, will be assigned a certain color, such as red, and as the probability of the feature point existing at locations further away from the area 50, such as in area 52, decreases, the color changes to, for example, blue.

[0022] The data provided by the 2D images 42 and the corresponding heat maps 48 generated by the process shown in FIG. 30 are then used to train a neural network using known AI training schemes, which will then be used in the system 10 to identify the 3D pose of the object 14. Figure 3 is a schematic diagram of a neural network 60 comprising a plurality of nodes 62, each having adjustable weights W, where the network 60 is intended to represent generally the neural networks discussed herein. The neural network 60 includes an input layer 64 that receives individual pixel data from a 2D image, a plurality of residual block layers 66 and 68 that identify features in the pixel data of the 2D image, and a plurality of convolutional layers 70, where each node 62 in the convolutional layers 70 provides a heat map. Note that the convolutional layers 70 and the residual block layers 66 and 68 are standard modules used in the field of deep learning. These types of convolutional layers are composed of a plurality of filter convolutional kernels that perform a convolution operation on the input of the image or a previous convolutional layer such that edge, texture, or contour information can be extracted from the input. A residual block layer is a structure that contains two convolutional layers with a skip connection, where the output of the second convolutional layer is added to the input of the first convolutional layer to be used as the block output.

[0023] Figure 4is flowchart 80 showing an algorithm running in controller 24 that estimates a 3D pose of object 14 using a trained neural network based on a 2D image 82 of object 14 taken by camera 22 using a learning-based neural network 78. Image 82 is provided to an input layer 84 and a plurality of successive residual block layers 86 and 88 that include a feed-forward loop in neural network 78 running in AI software in controller 24 that uses a filtering process to provide feature extraction of possible feature points on object 14 in image 82, such as gradients, edges, contours, etc. The image including the extracted features is provided to a plurality of successive convolutional layers 90 in neural network 78 that define possible feature points obtained from the extracted features as a series of heat maps 92, one heat map for each feature point, the heat maps showing the likelihood of the presence of that feature point on object 14 based on the color in heat map 92. Image 94 is generated using image 82 of object 14 that includes a plurality of feature points 96 from all of heat maps 92, where each feature point 96 is given a confidence value based on the color of its heat map 92, and where those feature points 96 with confidence values not above a certain threshold are not used.

[0024] Image 94 is then compared in pose estimation processor 98 to a nominal or virtual 3D CAD model of object 14 having the same feature points to provide an estimated 3D pose 100 of object 14. One suitable algorithm for comparing image 94 to a CAD model is known in the art as Perspective-n-Point (PnP). Generally, a PnP process estimates the pose of an object relative to a calibrated camera given a set of n 3D points of the object in a world coordinate system and their corresponding 2D projections in an image from the camera. The pose includes six degrees of freedom (DOF) consisting of a rotation (roll, pitch, and yaw) and 3D translation of the object relative to the camera coordinate system.

[0025] Figure 5is a schematic diagram 104 of how the PnP process is implemented in this example to obtain the 3D pose of the object 14. The schematic diagram 104 shows a 3D object 106 representing the object 14 at a true or real location. The object 106 is observed by a camera 112 representing the camera 22 and is projected onto a 2D image plane 110 as a 2D object image 108, where the object image 108 represents the image 94, and where the points 102 on the image 108 are the feature points predicted by the neural network 78 to represent the points 96. The schematic diagram 104 also shows a virtual 3D CAD model 114 of the object 14 having feature points 132 at the same locations as the feature points 96, which is randomly placed in front of the camera 112 and is projected onto the 2D image plane 110 as a 2D model image 116 also including the projected feature points 118. The CAD model 114 is rotated and translated in front of the camera 112, which rotates and translates the model image 116, to try to minimize the distance between each feature point 118 on the model image 116 and the corresponding feature point 102 on the object image 108, i.e., to align the images 116 and 108. Once the model image 116 is aligned as best as possible with the object image 108, the pose of the CAD model 114 relative to the camera 112 is the estimated 3D pose 100 of the object 14.

[0026] For one of the respective corresponding feature points between the images 108 and 116, the analysis is described by equation (1), where equation (1) is used for all of the feature points of the images 108 and 116.

[0027]

[0028] where V i is one of the respective feature points 132 on the CAD model 114, v i is the corresponding projected feature point 102 in the model image 116, a i is one of the respective feature points 102 on the object image 108, R is the rotation of the CAD model 114 relative to the camera 112, T is the translation of the CAD model 114 relative to the camera 112, the symbol'is the vector transpose, and is a reference to any feature point with index i. By solving equation (1) with an optimization solver, the optimal rotation and translation can be computed, thereby providing an estimate of the 3D pose 100 of the object 14.

[0029] The above process of learning-based neural network estimation of the 3D pose of object 14 provides high accuracy. However, improvements can be made. For example, small pixel errors in the 2D projection of object 14 on the image plane 110 in the x or y axis, which is orthogonal to the optical z-axis of camera 112, result in a relatively large error in the z-axis, which can significantly produce object position uncertainty and degrade 3D pose estimation performance. Accordingly, the present disclosure also proposes a 3D pose estimation process that reduces z-axis uncertainty by using multiple cameras.

[0030] Figure 6 is a schematic diagram of a camera system 120 that includes a first camera 122 and a second camera 124 oriented with respect to an x, y, and z coordinate frame, where the z-axis is along the optical axis of cameras 122 and 124. Shown is an uncertainty region 126 that describes the accuracy limit with which the position of object 14 can be estimated from the image from camera 122, and an uncertainty region 128 that describes the accuracy limit with which the position of object 14 can be estimated from the image from camera 124. As shown, cameras 122 and 124 are highly accurate in the x and y axes, but the accuracy in the z-axis is limited, where the accuracy in all axes depends on the distance between camera 122 or 124 and the imaged object 14. However, by fusing the 3D pose estimates provided by both cameras 122 and 124, the accuracy of the estimated 3D pose of object 14 in the z-axis is significantly improved, as shown by uncertainty region 130. In other words, for this schematic diagram, the z-axis of one of cameras 122 and 124 is the x-axis of the other camera 122 and 124, such that the general uncertainty of the estimation process is the uncertainty in the x-axis direction.

[0031] In this non-limiting embodiment, cameras 122 and 124 are oriented such that the optical axes of cameras 122 and 124 are orthogonal to each other, and have an optimal distance from cameras 122 or 124 to object 14 to provide the smallest uncertainty region 130. However, real systems can be limited in the z-axis positioning of cameras 122 and 124 with respect to each other, and the distance between cameras 122 or 124 and object 14, as well as other limiting factors such as lighting, the type of cameras used, etc., that can result in non-orthogonal optical axes and other limitations. In those systems, it can be necessary to employ more than two cameras to reduce the size of the uncertainty region closer to the smallest uncertainty region 130.

[0032] Figure 7is a block diagram of a system 140 for estimating the 3D pose of an object 14 using 2D cameras 122 and 124 and a learning based neural network of the type described above. Camera 122 provides a 2D image 142 of the object 14 and camera 124 provides a 2D image 144 of the object 14 at their respective angles. Image 142 is provided to a processor 146 and image 144 is provided to a processor 148, which generate feature point images 150 and 152, respectively, similar to image 94 from their angles, where the processors 146 and 148 use, for example, an input layer 84, a plurality of successive residual block layers 86 and 88, and a plurality of successive convolutional layers 90 to generate heat maps which are then used to generate images 150 and 152 in the manner described above. Images 150 and 152 are fused in a pose estimation block 154 which generates a more accurate estimate of the 3D pose 156 of the object 14 than can be provided by a single camera 22, where the pose estimation process also uses a PnP algorithm as described above.

[0033] Figure 8 is a schematic 160 similar to schematic 104, where like elements are identified by like reference numerals, which describes the PnP process as discussed in block 154 for image fusion. In schematic 160, camera 162 represents camera 122 and camera 164 represents camera 124. Schematic 160 shows a 2D model image 166 with feature points 172 which is a projection of CAD model 114 onto a 2D image plane 170 of camera 164, and a 2D object image 168 with feature points 174 which is a projection of 3D object 106 onto image plane 170 of camera 164. CAD model 114 is rotated and translated in front of cameras 162 and 164 to obtain the best model pose for both cameras 162 and 164 simultaneously, where feature points 118 on model image 116 on image plane 110 are as close as possible to feature points 102 on object image 108, and feature points 172 on model image 166 on image plane 170 are as close as possible to feature points 174 on object image 168. Once this is obtained, the best orientation of 3D model 114 is obtained which represents the 3D pose of object 14.

[0034] For one of the respective corresponding feature points between images 108 and 116 and images 166 and 168, the analysis is described by equation (2), where equation (2) is used for all of the feature points of images 108 and 116 and images 166 and 168.

[0035]

[0036]

[0037]

[0038] where u i is one of the individual feature points 172 on the model image 166, b i is the corresponding feature point 174 on the object image 168.

[0039] The above-described technique for extracting features from 2D images using a learning-based neural network to obtain a 3D pose estimate of the object 14 is one suitable feature extraction technique. There are other methods. The following discussion describes a feature extraction technique employing deep learning and vector field estimation that can also be used in place of the learning-based technique with satisfactory results. As will be discussed in detail below, the vector field estimation process replaces the neural network 78 including the input layer 84, the plurality of successive residual block layers 86 and 88, and the plurality of successive convolutional layers 90 that produce the heat map 92 with vector field estimation elements. However, it is emphasized that although the vector field estimation process is described herein for obtaining a 3D pose of an object, the process will apply to any suitable process that requires features to be extracted from an image.

[0040] Figure 9 is a flowchart 180 illustrating a vector field estimation process operating in the controller 24 for generating a 3D pose of the object 14 using a 2D image 82 of the object 14 taken by the camera 22, where like elements are identified by like reference numerals. The image input is a three-channel RGB (red-green-blue) image defined by a three-dimensional matrix N*N*3, where N is the height and width of the image 82. The image 82 is provided to an encoder / decoder neural network 182 that includes an encoder network 184 that extracts possible key features in the image 82 and a decoder network 186 that generates an image from the extracted features to identify possible feature points on the object 14, as described below, where both networks 184 and 186 include a plurality of neural network layers of the type described above. In particular, in one non-limiting embodiment, the neural network layers in the encoder network 184 include convolutional layers 188, batch normalization (BN) layers 190, and rectified linear unit (ReLu) layers 192, and the neural network layers in the decoder network 186 include pooling layers 194, up-sampling layers 196, and softmax layers 198.

[0041] The network 182 processes the three-dimensional matrix of the image 82 to generate a one-dimensional (N*N*1) mask image 200 and a two-dimensional (N*N*2) vector field image 202 for each of the identified feature points from the three channels. To generate the mask image 200, the network 182 assigns each pixel in the image 82 a number 1 or 0 depending on whether a portion of the object 14 is present in the particular pixel, where pixels in bright areas of the mask image 200 have a number 1 and pixels in dark areas of the mask image 200 have a number 0. Each vector field image 202 includes a plurality of arrows, each defined by an x-direction vector value and a y-direction vector value provided for each pixel in the image 82, where the direction of the arrow is toward one of the respective feature points in the image 82 and thus each pixel in the image 82 has the ability to predict a direction to that feature. The value of the respective pixel in the mask image 200 is multiplied by the value of the corresponding pixel in the vector field image 202 such that those arrows that are not associated with a pixel having a portion of the object 14 and that do not provide a useful direction to a feature point are discarded, as shown in the vector field extraction image 204.

[0042] Each arrow in the vector field extraction image 204 provides a direction to one of the plurality of feature points, but not a distance to that feature point. Thus, for each combination of two object pixels p i and p j in the vector field extraction image 204, the process identifies their vectors v i and v j will intersect at a point x ij , as shown in the vector intersection image 206. Each point x i is assigned a score according to the distance between the pixels p ij and the point x j , the distance between the pixels p ij and the point x i , and the gradient magnitude at the pixels p j and p ij . For example, smaller distances between the pixels p i and the point x ij and between the pixels p j and the point x ij provide higher scores, and higher gradient magnitudes at the pixels p i and p j provide higher scores. The result is a grid of many points x ij , each with a score, where the areas in the image with the most points x ij will identify the likely feature point locations, as shown in the point voting image 208.

[0043] The vector field image 202 shows a vector arrow for one of the individual possible feature points identified by the network 182. The network 182 will also generate a separate vector field image 202 for each possible feature point, multiplied by the mask 200. All of the feature points of all of the point voting images 208 are combined into a single image, such as the image 94 including the feature points 96, where each individual feature point 96 is provided by one of the individual point voting images 208. The image 94 can then undergo a PnP process in the pose estimation processor 98 to generate the 3D pose 100 in the manner described above. In addition, multiple cameras can be employed, with each of the individual images from those cameras being processed by the vector field estimation process, due to the uncertainty along the optical axis of an individual camera as described above.

[0044] As mentioned, the vector field estimation process can be applied to other processes that require feature point extraction. For example, one application in robotics is to find the center of an object being picked up by a robot, where the orientation of the object is not necessary. Another application can be to determine a measurement of an object for quality assurance purposes.

[0045] As will be readily appreciated by those skilled in the art, the several and various steps and processes discussed herein in connection with the description of the present disclosure can be embodied in various forms and methods, and are not limited to the specific structures and arrangements described herein. Rather, the present disclosure is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure as defined by the appended claims.

[0046] The foregoing discussion discloses and describes exemplary embodiments of the present disclosure. One skilled in the art will readily recognize from such discussion and from the accompanying drawings and claims that various changes, modifications and variations can be made therein without departing from the spirit and scope of the disclosure as defined by the following claims.

Claims

1. A method for extracting features from an image of an object, the method comprising: obtaining a 2D image having a plurality of pixels of the object using a 2D camera; extracting a plurality of possible feature points on the object from the 2D image; generating a mask image defining a plurality of pixels in the 2D image in which the object is located; generating a separate vector field image for each extracted feature point, the vector field image including an arrow having an x-direction value and a y-direction value for each pixel in the 2D image, wherein the direction of each arrow is toward the extracted feature point in the 2D image; generating a separate vector extraction image for each extracted feature point by multiplying the plurality of pixels in the mask image with corresponding pixels in the vector field image such that those arrows not associated with pixels having portions of the object in the 2D image are discarded; generating a vector intersection image for each extracted feature point by identifying intersection points of arrows for each combination of two pixels in the 2D image; assigning a score to each intersection point for each extracted feature point according to the distance of each pixel in each combination of two pixels from the intersection point and generating a score grid for all pixel combinations; and generating a point voting image for each extracted feature point identifying a feature location from a plurality of clustered points. The extracting of the possible feature points, the generating of the mask image, and the generating of the vector field image are performed in a deep learning neural network.

2. The method of claim 1, wherein, Assigning a score to each intersection point further includes using gradient magnitude at the plurality of pixels.

3. The method of claim 1, wherein, Smaller distances between the pixels and the intersection points provide higher scores, and higher gradient magnitudes at the plurality of pixels provide higher scores.

4. The method of claim 3, wherein, The point voting images for all extracted feature points are combined into a combined point voting image.

5. The method of claim 1, wherein, The combined point voting image is used to estimate a 3D pose of the object.

6. The method of claim 5, wherein, Estimating the 3D pose of the object includes comparing the feature location to a 3D virtual model of the object.

7. The method of claim 6, wherein, Estimating the 3D pose of the actual object includes using a perspective-n-point algorithm that compares a 2D projection of the 3D virtual model onto an image to a 2D projection of the actual object having the feature location on the image.

8. The method of claim 7, wherein, The method is used in a robotic system, and the object is being picked up by a robot.

9. The method of claim 7, wherein, The 2D image is a three-channel RGB (red-green-blue) image defined by a three-dimensional matrix.

10. The method of claim 1, wherein, 11. A method for extracting features from an image having a plurality of pixels of an object using a camera, the method comprising: extracting a plurality of possible feature points on the object from the image using a deep learning neural network; generating a separate vector field image for each extracted feature point using a deep learning neural network, the vector field image including an arrow having an x-direction value and a y-direction value for each pixel in the image, wherein the direction of each arrow is toward the extracted feature point in the image; and identifying intersection points of the arrows using a vector field estimation process to identify a feature location for each separate vector field image. ​ 12. The method of claim 11, further comprising providing a combined image as a combination of all identified feature locations.

13. The method of claim 11, wherein, The image is a three-channel RGB (red-green-blue) image defined by a three- dimensional matrix.

14. A system for extracting features from an image of an object, the system comprising: a 2D camera for obtaining a 2D image having a plurality of pixels of the object; means for extracting a plurality of possible feature points on the object from the 2D image; means for generating a mask image defining a plurality of pixels in the 2D image in which the object is located; means for generating, for each extracted feature point, a separate vector field image comprising, for each pixel in the 2D image, an arrow having an x-direction value and a y- direction value, wherein the direction of each arrow is towards the extracted feature point in the 2D image; means for generating, for each extracted feature point, a separate vector extraction image by multiplying the plurality of pixels in the mask image with corresponding pixels in the vector field image, such that those arrows not associated with pixels in the 2D image having parts of the object are discarded; means for generating, for each extracted feature point, a vector intersection image by identifying intersection points of arrows of each combination of two pixels in the 2D image; means for assigning a score to each intersection point of each extracted feature point according to the distance of each pixel in each combination of two pixels from the intersection point, and generating a score grid for all pixel combinations; and means for generating, for each extracted feature point, a point voting image identifying a feature location from a plurality of clustered points.

15. The system of claim 14, wherein, The means for extracting the plurality of possible feature points, the means for generating the mask image and the means for generating the vector field image are deep learning neural networks.

16. The system of claim 14, wherein, The means for assigning a score to each intersection point also uses gradient magnitude at the plurality of pixels.

17. The system of claim 16, wherein, Smaller distances between the pixels and the intersection points provide higher scores, and higher gradient magnitudes at the plurality of pixels provide higher scores.

18. The system of claim 14, wherein, The means for generating a point voting image combines a plurality of point voting images for a plurality of feature locations.

19. The system of claim 18, wherein, The combined point voting image is used to estimate a 3D pose of the object.

20. The system of claim 14, wherein, The system is used in robot control, and the object is being picked up by a robot.

Images