Robot grasping prediction using neural networks and geometrically-aware object representations

Abstract

A deep machine learning method and system, some of which relate to determining grasp outcome predictions for candidate grasp poses of an end effector of a robot. Some implementations are directed to training and utilization of a geometry network and a grasp outcome prediction network. A trained geometry network can be used to generate geometry output(s) based on two-dimensional or two-and-a-half-dimensional images, the geometry output(s) being geometry-aware and representing (e.g., high-dimensional) three-dimensional features captured by the image(s). In some implementations, the geometry output(s) include at least an encoding generated based on a trained encoding neural network trained to generate an encoding representing three-dimensional features (e.g., shapes). A trained grasp outcome prediction network can be used to generate grasp outcome predictions for candidate grasp poses based on applying the geometry output(s) and additional data as input(s) to the network.

Description

[0001] This case is a divisional application of the invention patent application filed on June 18, 2018, with application number 201880035890.6 and invention title "Robot Grasping Prediction Using Neural Networks and Geometric Perceptual Object Representation". Technical Field

[0002] Embodiments of this disclosure relate to deep machine learning methods and apparatus for determining robot grasping predictions using neural networks and geometrically perceived object representations. Background Technology

[0003] Many robots utilize one or more end effectors to grasp various objects. For example, a robot can use a grasping end effector such as an "impact" gripper or an "invasive" gripper (e.g., physically penetrating the object using a pin, needle, etc.) to pick up an object from a first position, move the object to a second position, and place the object at the second position. Some additional examples of robotic end effectors that can grasp objects include "constraint" end effectors (e.g., using suction or vacuum to pick up objects) and one or more "contact" end effectors (e.g., using surface tension, freezing, or adhesive to pick up objects), to name just a few. Summary of the Invention

[0004] Various embodiments of this specification generally pertain to deep machine learning methods and apparatuses relating to prediction of grasping outcomes for determining candidate grasping postures of a robot's end effector. The grasping outcome prediction can be a metric indicating the probability of successfully grasping an object if the end effector attempts to grasp it while it is in a candidate grasping posture. In some embodiments, if the grasping outcome prediction is determined to meet one or more criteria (e.g., a probability threshold), control commands can be provided to one or more actuators of the robot to cause the end effector to attempt to grasp an object in its candidate grasping posture.

[0005] Some implementations address the training and utilization of a geometric network and a grasping outcome prediction network. Both the geometric network and the grasping outcome prediction network may each include at least one trained neural network model. The trained geometric network can be used to generate multiple geometric outputs based on multiple two-dimensional (2D) or two-and-a-half-dimensional (2.5D) images (2D with depth), wherein the multiple geometric outputs are geometrically aware and represent the three-dimensional (3D) geometric features (e.g., shape, position, orientation) of multiple objects captured by the multiple images. The multiple images may be captured by a robot's vision sensors (e.g., vision sensors that capture 2D or 2.5D vision sensor data). For example, the geometric features of the objects represented by the multiple geometric outputs may include at least the 3D shape of the object.

[0006] The (multiple) geometric outputs may include global geometric representations, such as encodings of high-dimensional geometric representations. This encoding may be generated based on applying (or providing) an image to a neural network encoder (e.g., a convolutional neural network (CNN) encoder). As described herein, the neural network encoder can be trained such that the encoded representation generated by the neural network encoder represents the 3D features of (multiple) objects captured by the image. Furthermore, the (multiple) geometric outputs may additionally or optionally include local geometric representations, wherein the local geometric representations are geometrically aware and locally focused on regions of the object near the candidate grasping pose. For example, the local geometric representations may be generated based on dense sampling of the encoded global geometric representation, wherein the dense sampling is from the angles of the candidate grasping pose from the end effector (e.g., the dense sampling is from a reference frame of the candidate grasping pose). In these and other ways, the local geometric representations can represent the 3D features of the object from the angles of the candidate grasping pose.

[0007] The trained grasp outcome prediction network can be used to generate grasp outcome predictions based on applying (multiple) geometric outputs and additional data as inputs to the network. For example, the additional data applied to the grasp outcome prediction network may include candidate grasp poses, pre-grab images, and / or pre-grab visual sensor poses.

[0008] Some implementations described herein can utilize training instances generated through user-guided grasping attempts in a virtual reality environment to train geometric networks and / or grasping outcome prediction networks. For example, the virtual reality (VR) environment can render a virtual environment including representations of objects and optionally multiple additional scene objects, as well as representations of an end effector of a robot and optionally multiple additional robot components. Through interaction with multiple user interface input devices (e.g., VR controllers), a user can manipulate the pose of the end effector in the VR environment to attempt to grasp an object once the user has manipulated the end effector into the desired grasping pose. For example, when a user determines that the end effector is in the desired pose for grasping an object rendered in the virtual environment, the user can provide user interface input (e.g., clicks, verbal commands) to attempt grasping. The VR environment's physics engine can be used to determine whether the attempted grasping was successful. Various data from the VR environment and user interactions via that environment can be used to generate training instances. For example, multiple images generated from the VR environment, the desired pose, and the actual grasping outcome can be used to generate training instances.

[0009] In some of those implementations, additional training instances are automatically generated based on (e.g., without requiring additional user-guided grasping attempts) user-guided grasping attempts. For example, when generating additional training instances, a new grasping pose can be determined based on the grasping pose from the user-guided grasping attempts. For each instance, a new grasping pose can be selected that is within the expected variance of the grasping poses from the user-guided grasping attempts (e.g., position of + / - X cm and / or orientation of + / - X degrees). Furthermore, the object and scene from the user-guided grasping attempts can be utilized (e.g., the same object pose can be utilized), and new grasping attempts can be attempted in a VR environment using the new grasping pose. The physics engine of the VR environment can be used to determine whether the new grasping attempt was successful. Various data from the new grasping attempts can be used to generate additional training instances. For example, the new grasping pose and the new grasping result can be used to generate additional training instances. Images from the new grasping attempts and / or user-guided attempts can also be included in the additional training instances. In these and other ways, a large number of additional training instances can be generated and used to train the geometry network and / or the grasping outcome prediction network. Furthermore, many of these additional training instances will be negative training instances (i.e., grasping failures) because the selection of new grasping poses will not be user-guided (but will be automatically selected using previous user-guided grasping attempts as guidance).

[0010] The various implementations described above and elsewhere in this document can achieve one or more technical advantages. For example, compared to various techniques that do not utilize such geometric output(s), the generation of geometric output(s) described herein and applying such geometric output(s) as input to a grasping outcome prediction network can improve the accuracy of grasping outcome prediction and / or make the grasping outcome prediction more robust (e.g., applicable to untrained objects). Furthermore, for example, the various neural network architectures described herein can improve performance compared to various other grasping prediction architectures. For example, the architectures described herein can improve the success rate of grasping target objects. Moreover, for example, automatically generating additional training instances using the techniques described herein can be more computationally efficient than generating additional training instances using further user-guided grasping attempts, and / or can make the training of the various neural network models described herein more robust.

[0011] In some implementations, a method is provided that includes identifying a current image captured by a robot's vision sensor. The current image captures the robot's end effector and an object to be grasped by the robot using the end effector. The method also includes generating an encoding of the current image or an additional image capturing the object. The generated encoding encodes geometric features of the object, and generating the encoding includes processing the current image or the additional image using a trained neural network encoder. The method also includes determining candidate grasping poses for the end effector. The method further includes: applying the current image, candidate grasping poses, and encoding as one or more inputs to a grasping outcome prediction network, which is a trained neural network model; and generating a grasping outcome prediction on the grasping outcome prediction network based on the applied one or more inputs. The method further includes: determining that the grasping outcome prediction meets one or more criteria; and, in response to the grasping outcome prediction meeting one or more criteria, providing control commands to one or more actuators of the robot to cause the end effector to attempt to grasp the object in the end effector's candidate grasping pose.

[0012] These and other implementations of the technology may include one or more of the following features.

[0013] In some implementations, the method further includes: generating a local geometric representation using the encoding and candidate grasping poses of the end effector; and including the local geometric representation in one or more inputs applied to the grasping outcome prediction network when generating a grasping outcome prediction. In some of those implementations, generating the local geometric representation includes performing dense sampling on the encoding from a reference frame of the candidate grasping pose of the end effector (e.g., from the angle of the candidate grasping pose).

[0014] In some implementations, the encoding is invariant to any visual sensor pose of the visual sensor, and / or the trained neural network encoder is jointly trained with the 3D neural network decoder.

[0015] In some implementations, the trained neural network encoder is trained based on applying the training codes generated on the neural network encoder to the 3D decoder, using the 3D decoder to generate training decoders, determining the loss based at least in part on the training decoders, and updating the neural network encoder based on the loss.

[0016] In some implementations, the geometric features encoded by coding include the three-dimensional shape features of the object.

[0017] In some embodiments, applying one or more inputs to a grasping result prediction network includes: applying the current image as at least a portion of a first input to one or more first layers; and generating a first layer output on the first layer based on the applied first input. In some of those embodiments, applying one or more inputs to a grasping result prediction network further includes: applying the encoding and the first layer output as at least a portion of a second input to one or more second layers; and generating a second layer output on the second layer based on the applied second input. In some versions of those embodiments, applying one or more inputs to a grasping result prediction network further includes: applying the end effector pose and the second layer output as at least a portion of a third input to one or more third layers; and generating a third layer output on the third layer based on the applied third input. In some further versions of those embodiments, the method further includes: generating a local geometric representation using the encoding and the candidate grasping pose of the end effector; applying the third layer output and the local geometric representation as at least a portion of a fourth input to one or more fourth layers; and generating a grasping result prediction on the fourth layer based on the applied fourth input.

[0018] In some implementations, determining candidate grasping poses includes: generating a plurality of candidate grasping poses; and performing one or more iterations of cross-entropy optimization on the plurality of candidate grasping poses to select a candidate grasping pose from the plurality of candidate grasping poses.

[0019] In some implementations, the method further includes: identifying the current visual sensor pose of the visual sensor; and including the current visual sensor pose in one or more of the inputs applied to the grasping result prediction network.

[0020] In some implementations, a method is provided that includes: identifying a current image captured by a robot's vision sensor; and generating an encoding of the current image or an additional image of a captured object. The current image captures the robot's end effector and the object to be grasped by the robot using the end effector. The generated encoding encodes geometric features of the object, and generating the encoding includes processing the current image or additional image using a trained neural network encoder. The method further includes: applying the current image and the encoding as one or more inputs to a prediction network, wherein the prediction network is a trained neural network model; generating a prediction on the prediction network based on the application of the one or more inputs; determining that the resulting prediction satisfies one or more criteria; and providing prediction-based control commands to one or more actuators of the robot in response to the prediction satisfying the one or more criteria.

[0021] Other implementations may include a non-transitory computer-readable storage medium storing instructions executable by one or more processors (e.g., a central processing unit (CPU), a graphics processing unit (GPU), and / or a tensor processing unit (TPU)) to perform methods such as those described above and / or elsewhere herein. Other implementations may include a system of one or more computers and / or one or more robots, the computers including one or more processors operable to execute stored instructions to perform methods such as those described above and / or elsewhere herein.

[0022] In some implementations, a method implemented by one or more processors is provided, comprising: generating training instances based on user-guided robotic grasping attempts of objects in a virtual reality environment, the training instances including: one or more rendered images of the user-guided robotic grasping attempts, the one or more rendered images being generated by the virtual reality environment and capturing the object; grasping poses of the user-guided robotic grasping attempts, wherein the grasping poses are based on virtual poses of virtual robot end effectors in the virtual environment during the user-guided robotic grasping attempts; and grasping result measurements indicating whether the user-guided robotic grasping attempts of the object were successful; based on user... Guided robot grasping attempts to automatically generate multiple additional training instances, each of which includes: determining a new grasping pose within the variance of grasping poses from user-guided robot grasping attempts; attempting a new grasping attempt on the object using the new grasping pose; determining whether the new grasping attempt was successful; generating additional training instances to include the new grasping pose, one or more new images from the new grasping attempt, and a new grasping result measurement based on whether the new grasping attempt was successful; and training a neural network model using the training instances and additional training instances.

[0023] In some embodiments, a system is provided comprising: one or more computers, the one or more computers including one or more processors operable to execute stored instructions to perform a method, the method comprising: generating training instances based on user-guided robot grasping attempts in a virtual reality environment, the training instances including: one or more rendered images of the user-guided robot grasping attempts, the one or more rendered images being generated by the virtual reality environment and capturing the object, grasping poses of the user-guided robot grasping attempts, wherein the grasping poses are based on virtual poses of a virtual robot end effector in the virtual environment during the user-guided robot grasping attempts, and grasping result measurements. The method indicates whether the user-guided robot grasping attempt of the indicated object was successful; it automatically generates multiple additional training instances based on the user-guided robot grasping attempt, wherein each of the multiple additional training instances automatically generated based on the user-guided robot grasping attempt includes: determining a new grasping pose based on the new grasping pose within the variance of the grasping poses from the user-guided robot grasping attempt; attempting a new grasping attempt of the object using the new grasping pose; determining whether the new grasping attempt was successful; and generating additional training instances to include the new grasping pose, one or more new images from the new grasping attempt, and a new grasping result measurement based on whether the new grasping attempt was successful; and training a neural network model using the training instances and additional training instances.

[0024] It should be understood that all combinations of the foregoing and additional concepts described in more detail herein are considered part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are considered part of the subject matter disclosed herein. Attached Figure Description

[0025] Figure 1 An example environment in which the implementations disclosed herein can be carried out is shown.

[0026] Figure 2 The diagram illustrates a geometric network and a crawling outcome prediction network, along with examples of data that can be applied to the network to generate crawling outcome predictions.

[0027] Figure 3 It shows Figure 2 Specific implementations of geometric networks and crawling result prediction networks, and how to... Figure 2 Data is applied to specific implementations of the network to generate Figure 2 An example of predicting crawling results.

[0028] Figure 4 Examples of how to train a geometry network and a grasping result prediction network are shown.

[0029] Figure 5 This is a flowchart illustrating an example method for generating training instances based on user-guided crawling attempts.

[0030] Figure 6 This is a flowchart illustrating an example method for pre-training a geometric network based on training instances.

[0031] Figure 7 This is a flowchart illustrating an example method for training a grasping result prediction network and an optional geometric network based on training instances.

[0032] Figure 8 An example method is shown that uses a geometric network and a grasp result prediction network to determine when to attempt a grasp based on candidate grasp poses.

[0033] Figure 9 An example architecture of the robot is schematically depicted.

[0034] Figure 10 An example architecture of a computer system is schematically depicted. Detailed Implementation

[0035] Figure 1An example environment in which the embodiments disclosed herein can be implemented is shown. The example environment includes a robot 180, a computing device 105, a training instance generation system 110, a training instance database 170, a training engine 130, a geometric network 120, and a grasping result prediction network 140.

[0036] The training instance generation system 110 generates training instances based on user-guided grasping attempt demonstrations. The training instance generation system 110 stores the generated training instances in a training instance database 170. For example, a user-guided grasping attempt demonstration may include: physical manipulation of the robot 180 by the user to kinesthetically demonstrate grasping; and / or interaction between the user and computing device 105 and / or (multiple) associated components to demonstrate the grasping attempt in a VR environment rendered by computing device 105.

[0037] Training engine 130 uses training instances from training instance database 170 to train geometry network 120 and grasping result prediction network 140. Once trained, geometry network 120 and grasping result prediction network 140 can be used by robot 180 and / or other robots(s) to determine how to grasp objects using corresponding end effectors. For example, robot 180 can use networks 120 and 140 to grasp spatula 189A using grasping end effector 182A. Reference is now made to the various accompanying figures provided herein. Figure 1 Additional descriptions of the various components.

[0038] Robot 180 is a "robotic arm" with multiple degrees of freedom, enabling the grasping end effector 182A to traverse any of a plurality of potential paths to position the grasping end effector 182A in a desired pose. As used herein, "pose" can refer to the position and / or orientation of the corresponding component it refers to. For example, "pose" can define the position and orientation of the corresponding component in all six dimensions of a six-dimensional space (e.g., Euclidean space). The pose can optionally be defined by six values, each representing a corresponding value in six-dimensional space.

[0039] Robot 180 controls two opposing "paws" of its grasping end effector 182A to actuate the claws between at least one open position and one closed position (and / or optionally multiple "partially closed" positions). Example vision component 184 is also included. Figure 1 As shown in [the image]. Figure 1In this embodiment, vision component 184 is mounted in a fixed posture relative to the base or other stationary reference point of robot 180. The vision component includes one or more vision sensors that can generate images relating to the color, depth, and / or other characteristics of objects within the line of sight of the vision sensors. For example, vision component 184 may include a thematic camera (and optionally a depth sensor), a stereo camera, or a laser scanner. When vision component 184 is a thematic camera with a depth sensor, it can capture 2.5D images, such as images with multiple color channels (e.g., “R, G, B” channels captured by a 2D camera sensor) and a depth channel captured by the depth sensor. In some embodiments, a stereo camera may include two or more sensors, each located in a different advantageous position. In some of those embodiments, the stereo camera generates images based on features sensed by two sensors, each image including multiple data points defining depth values ​​and color and / or grayscale values. For example, a stereo camera may generate images including a depth channel and red, blue, and / or green channels. A laser scanner includes one or more lasers that emit light and one or more sensors that collect 2.5D data points relating to the reflection of the emitted light. Such instances of 2.5D data points are considered images, as the term is used herein. For example, the laser scanner could be a time-of-flight laser scanner or a triangulation-based laser scanner, and could include a position-sensitive detector (PSD) or other optical position sensors. The vision component 184 has a field of view of at least a portion of the workspace of the robot 180, such as including a portion of the workspace of the example spatula 189A. Although the resting surface of the spatula 189A is not in... Figure 1 As shown, it can be placed on a table, tray and / or other (multiple) surfaces.

[0040] Robot 180 is provided as an example of a robot that can be utilized in the various techniques described herein. For example, a user can physically manipulate robot 180 to kinesthetically demonstrate grasping, and during the kinesthetic demonstration, data from the sensors of robot 180 and images(s) associated with the kinesthetic demonstration captured by vision component 184 can be utilized by training instance generation system 110 to generate one or more training instances.

[0041] As another example, one or more processors of robot 180 may additionally and / or alternatively utilize a trained version of geometric network 120 and grasping outcome prediction network 140 to grasp objects such as spatula 189A using end effector 182A.

[0042] refer to Figure 2This provides a specific example of grasping an object using a trained version of the geometry network 120 and the grasping result prediction network 140. A static image 291A (and optionally a static vision sensor pose 292A) is applied as one or more inputs to the geometry network 120, or a pre-grasped image 291B (and optionally a pre-grasped vision sensor pose 292B) is applied as one or more inputs to the geometry network 120. For example, one or more processors of the robot 180 may apply image 291A or 291B, and optionally the corresponding vision sensor pose 292A or 292B.

[0043] Images 291A and 291B can each be 2D or 2.5D images captured by the vision component 184 of robot 180. The static vision sensor pose 292A can be the pose of the vision component 184 when capturing static image 291A (e.g., the pose of the vision sensor of the vision component), and the pre-grasp vision sensor pose 292B can be the pose of the vision component 184 when capturing pre-grasp image 291B. The vision sensor pose can be fixed (e.g., in the case of a stationary vision sensor) or can be determined based on a localization technique that relies on one or more inputs (e.g., inputs from multiple sensors of robot 180, such as inputs from an inertial measurement unit (IMU)). For example, static image 291A can be an image of spatula 189A captured when end effector 182A is positioned, such that end effector 182A does not occlude spatula 189A in static image 291A and / or does not occlude spatula 189A in static image 291A. In other cases, the end effector 182A may occlude the spatula 189A in and / or in the still image 291A. The pre-captured image 291B may be "later in time" than the still image 291A and may be, for example, the current image, such as a recently captured (e.g., recently captured) image.

[0044] One or more processors of the robot 180 may generate a global geometric representation 223 and a local geometric representation 225 based on applying an input to the geometric network 120. The global geometric representation 223 may be an encoding of a high-dimensional geometric representation. In some implementations, one or more trained neural network models of the geometric network 120 may be utilized to generate the encoding based on processing of a static image 291A or a pre-grasp image 291B (and optionally the corresponding visual sensor pose 292A or 292B). The local geometric representation 225 may be geometry-aware and locally focused on a region proximate to a candidate grasp pose 293. With the candidate grasp pose 293 considered, the local geometric representation 225 may be generated based on processing the global geometric representation 223 and / or the static image 291A (or the pre-grasp image 291B). The candidate grasp pose 293 may be a candidate end effector pose that is considered for grasping, and may be generated by one or more other components.

[0045] In some implementations, the candidate grasp pose 293 is generated by sampling a set of candidate end effector poses and selecting the candidate grasp pose 293 from the sampled group. In some of those implementations, the candidate grasp pose 293 is generated using an optimization technique such as a derivative-free optimization algorithm. For example, a derivative-free optimization technique such as the cross-entropy method (CEM) may be utilized in the selection. CEM is a derivative-free optimization algorithm that samples a batch of N values at each iteration, fits a Gaussian distribution to N<M of these samples, and then samples a new batch of N values from the Gaussian distribution. For example, CEM may be utilized with values of M=64 and N=6, and three iterations of CEM may be performed to determine the best available (according to CEM) candidate grasp pose 293. For example, the optimization may take into account a grasp outcome prediction network (e.g., the grasp outcome prediction network may be utilized to evaluate which sampled value (candidate end effector pose) is "best" at each iteration). For example, when generating the candidate grasp pose 293, a plurality of candidate end effector poses may be considered with the optimization technique taken into account, and the best (according to the optimization technique) sampled candidate grasp pose may be selected. With the processing using the grasp outcome prediction network taken into account, the best sampled candidate grasp pose may be the pose that yields the best grasp outcome prediction. In some other implementations, a separate upstream trained neural network model may be utilized to determine the candidate grasp pose 293.

[0046] Local geometric representation 225, global geometric representation 223, candidate grasping pose 293, pre-grasp image 291B, and optional pre-grasp visual sensor pose 292B are applied (or provided) as one or more inputs to grasping result prediction network 140. Based on the application of one or more inputs, a grasping result prediction 247 is generated on grasping result prediction network 140. For example, one or more processors of robot 180 can apply the inputs and generate grasping result prediction 247. Grasp result prediction 247 can be a metric indicating the probability that end effector 182A will successfully grasp an object if it attempts to grasp the object when it is in a candidate grasping pose. In some implementations, if it is determined that grasping result prediction 247 meets one or more criteria (e.g., probability thresholds), control commands can be provided to one or more actuators of the robot to cause end effector 182A to attempt to grasp an object in its candidate grasping pose 293. For example, a control command may cause the end effector 182A to traverse to candidate grasping poses 293 and then attempt to grasp. In some implementations, if it is determined that the grasping outcome prediction 247 does not meet one or more criteria (e.g., probability thresholds), a new candidate grasping pose may be selected, and a new grasping outcome prediction may be determined based on the new candidate grasping pose (and optionally based on the poses of the new(multiple) images and / or(multiple) visual sensors).

[0047] Figure 3 It shows Figure 2 One implementation of the geometric network and the crawling result prediction network. Figure 3 It shows the relationship with Figure 2 The same data 291A, 291B, 292A, 292B, and 293 are shown, and are also shown to be consistent with... Figure 2 The same global geometric representation 223, local geometric representation 225, and crawling result prediction 247 are shown. However, Figure 3 Components of the geometric network 120 and the crawling result prediction network 140, which can be utilized in some embodiments, are shown. Furthermore, Figure 3 The static image 291A and optional static vision sensor pose 292A, which are applied as input to the geometry network 120, are shown, as well as the separate pre-grasp image 291B and optional pre-grasp vision sensor pose 292B, which are applied as input to the grasping result prediction network 140.

[0048] exist Figure 3In this process, a static image 291A and an optional static visual sensor pose 292A are used as input to a trained CNN encoder 122 to generate a global geometric representation 223. The global geometric representation 223 is an encoding of a high-dimensional geometric representation and is generated using the trained CNN encoder 122 based on the processing of the static image 291A and the optional static visual sensor pose 292A. In other words, the global geometric representation 223 is an encoding of the static image 291A and the optional static visual sensor pose 292A, as generated based on the trained CNN encoder 122. As described herein (e.g., with...), Figure 4 and Figure 6 (As described above), a CNN encoder 122 can be trained such that a global geometric representation 223 generated using the CNN encoder 122 represents the 3D features (e.g., 3D shape) of the object(s) captured by the static image 291A. In some of those embodiments, the global geometric representation 223 is an encoding and is view-invariant (e.g., an identity unit). Furthermore, in some of those embodiments, the global geometric representation 223 is high-dimensional because it is not a complete 3D representation. Instead, the global geometric representation 223 encodes 3D features in an encoding that is a much smaller data structure than the data structure required for a complete 3D representation.

[0049] Local shape sampler 124 of geometric network 120 is used to generate local geometric representation 225. Local geometric representation 225 is generated based on global geometric representation 223 and candidate grasping poses 293 of end effectors. Local shape sampler 124 can generate local geometric representation 225 from the angle (e.g., reference frame) of candidate grasping poses 293 of end effectors, based on dense sampling of global geometric representation 223. In these and other ways, local geometric representation can represent the 3D features of an object from the angle of candidate grasping poses 293.

[0050] The grasping result prediction network 140 includes a CNN 142, a combination layer 143, a first multilayer perception (MLP) neural network 144, and a second MLP neural network 146. A pre-grasp image 291B, optionally along with a pre-grasp visual sensor pose 292B, is applied as input to the CNN 142. An output is generated on the CNN 142 based on the applied pre-grasp image 291B and the optional pre-grasp visual sensor pose 292B. The output generated on the CNN 142 effectively encodes the current state of the robot 180's environment, and this output, along with a global geometric representation 223, is applied as input to the combination layer 143. An output is generated on the combination layer 143 based on the output generated on the CNN 142 and the global geometric representation. The output from the combination layer 143 and the candidate grasping pose 293 are applied as input to the first MLP 144. An output is generated on the first MLP 144 based on the applied output from the combination layer 143 and the candidate grasping pose 293. The local geometric representation 225, along with the output from the first MLP 144, is also applied as input to the second MLP 146. Based on the applied output from the first MLP 144 and the local geometric representation 225, a crawling result prediction 247 is generated on the second MLP 146.

[0051] Therefore, in Figure 3 In this implementation, the grasping result prediction network 140 includes multiple stacked trained neural networks 142, 143, 144, and 146, each network including one or more network layers. CNN 142 is used to efficiently encode the current state of the robot 180's environment, regardless of candidate grasping poses 293 or global geometric representations 223 or local geometric representations 225. A combination layer 143 is used to inject the global geometric representation 223 along with the encoding of the current state. Taking into account the current state encoding of CNN 142 and the global geometric representation 223 (via applying the output of layer 143 to a first MLP 144), and considering the candidate grasping poses 293, the first MLP 144 is used to generate an output indicating the grasping result prediction. Taking into account the globally geometrically influenced output of the first MLP 144, and considering the local geometric representation 225, a second MLP 146 is used to generate a final grasping result prediction 247.

[0052] Despite Figure 1 A specific robot 180 is shown in the figure, and reference is made to the various figures in this document (e.g., Figure 2 and Figure 3The description is provided, but additional and / or alternative robots may be used, including additional robotic arms similar to Robot 180, robots with other robotic arm forms, robots with humanoid forms, robots with animal forms, robots that move via one or more wheels (e.g., self-balancing robots), diving robots, unmanned aerial vehicles (UAVs), and so on. Furthermore, although in Figure 1 The specific grasping end effector 182A is shown, but the robot 180 may use additional and / or alternative end effectors, such as alternative impact-type grasping end effectors (e.g., those with grasping “plates”, those with more or fewer “fingers” / “daws”), “intrusive” grasping end effectors, “restrictive” grasping end effectors, or “contact” grasping end effectors, or non-grasping end effectors. Additionally, although in Figure 1 A specific mounting of the vision component 184 is shown, but additional and / or alternative mountings may be used. For example, in some embodiments, the vision component may be mounted directly to the robot, such as on a non-actuable component of the robot or on an actuable component of the robot (e.g., on or near an end effector). Furthermore, for example, in some embodiments, the vision component may be mounted on a non-stationary structure separate from its associated robot, and / or may be mounted in a non-stationary manner on a structure separate from its associated robot.

[0053] Figure 1 The diagram also shows a computing device 105. For example, computing device 105 can be: a virtual reality headset and optional external components; a tablet computer; a mobile phone; a desktop computing device, etc. Users can interact with one or more user interface input devices of computing device 105 to demonstrate grabbing attempts in a VR environment rendered by computing device 105 and / or by one or more additional computing devices communicating with computing device 105 via a network.

[0054] For example, in Figure 1In this embodiment, the display screen 106 of the computing device shows a rendered VR environment, which includes a representation of glasses 189B, a representation (unnumbered) of the surface on which the glasses rest, and a representation of the robot's end effector 182B. In other embodiments, other components of the robot, such as links controlling the position of the end effector 182B, may also be rendered. Through interaction with user interface input devices (e.g., the display screen 106 (if it is touch-sensitive), and individual controllers), a user can manipulate the pose of the end effector 182B in the VR environment and attempt to grasp the glasses 189B when the pose is in a desired pose. In some embodiments, the physics engine of the VR environment may be used to determine whether the attempted grasp is successful. In some embodiments, a grasp can be determined to be successful if at least a portion of the glasses 189B is positioned between the grasping members after the grasping members of the end effector 182B are actuated to a closed or "partially closed" state in the virtual reality environment. Various data from the VR environment and user interactions via that environment can be provided to the training instance generation system 110 and used to generate training instances. For example, the training instance generation system 110 can generate training instances using multiple images generated from the VR environment, the desired pose, and the actual grasping results.

[0055] The training instance generation system 110 includes an image engine 112, a grasping pose engine 114, a grasping result engine 116, and a supplementary instance engine 118. When generating training instances from user-guided grasping attempts, for each training instance, the image engine 112 determines one or more static images and / or pre-grabbed images based on data from the user-guided grasping attempts. The image engine 112 can also determine the visual sensor pose corresponding to the image. For each training instance, the grasping pose engine 114 determines the grasping pose of the end effector based on the grasping pose used in the user-guided grasping attempts. For each training instance, the grasping result engine 116 determines a metric indicating the grasping result of the user-guided grasping attempts. The training instance generation system 110 can store the determined images(s), grasping poses, and grasping results from the user-guided grasping attempts as training instances in a training instance database 170. Supplementary instance engine 118 can automatically generate one or more additional training instances based on corresponding user-guided crawling attempts, and can optionally interact with one or more of other engines 112, 114, and 116 when generating additional training instances. Training instance generation system 110 can store the additional training instances in training instance database 170. Additional descriptions of each engine 112, 114, 116, and 118 are now provided.

[0056] When generating training instances from user-guided grabbing attempts, the image engine 112 determines one or more 2D or 2.5D images associated with the grabbing attempt for each training instance. The image engine 112 can generate multiple images when the grabbing attempt is performed via a VR environment. When generating images from a virtual environment, the image engine 112 can utilize ray tracing and / or other image generation techniques.

[0057] As an example, for a training instance, image engine 112 can generate one or more static images, each capable of capturing at least a portion of the glasses 189B and the surface on which the glasses 189B rests. When generating static images, image engine 112 can generate static images from corresponding virtual vision sensor poses, and the corresponding static virtual vision sensor poses can be stored in the training instance in association with the static images. For example, the static images can be generated from virtual vision sensor poses... Figure 1 Glasses 189B are the target or approach Figure 1 The image engine 112 is positioned at a distance (e.g., between 35 and 45 cm) from the glasses 189B, and at a corresponding azimuth and elevation. In some embodiments, the representation of the end effector 182B may be omitted from one or more of the still images. In some embodiments, the image engine 112 may generate multiple still images corresponding to the grasping attempt, wherein each still image is captured from a virtual camera sensor pose that is unique (e.g., at a unique position and / or orientation) for the other (multiple) still images of the grasping attempt. For example, a first still image may be captured from a virtual visual sensor pose targeting the glasses 189B, and this virtual visual sensor pose: has a pseudo-randomly selected position within a first distance range from the glasses 189B; and has an elevation and / or orientation selected from multiple predetermined elevations and / or orientations.

[0058] As another example, for a training instance, image engine 112 can generate one or more pre-grasp images, each capturing glasses 189B and end effector 182B. In some cases, a pre-grasp image can be captured when the end effector 182B is in a grasping pose, or when the end effector 182B is in a pose prior to the grasping pose, which precedes the grasping pose and occurs within at least a threshold time duration of the grasping pose. Similar to static images, pre-grasp images can be generated from the pose of a virtual vision sensor, which... Figure 1 Glasses 189B are the target or approach Figure 1The image is positioned at a distance corresponding to the glasses 189B, and at the corresponding orientation and height. The corresponding pre-captured virtual sensor pose can be stored in the training instance in association with the pre-captured image.

[0059] In the case where the user-guided grasping attempt is physically manipulated by robot 180, image engine 112 can determine the images of training instances based on images captured by vision component 184. For example, image engine 112 can utilize images(s) omitting end effector 182A as static images, and images captured by vision component 184 that are temporally close to the grasping attempt as pre-grasping images. The pose of the corresponding vision component 184 can be stored in association with the static images and pre-grasping images. For example, the pose of vision component 184 for a given image can be determined by robot 180 using one or more localization techniques.

[0060] For training instances, the grasping posture engine 114 determines the grasping posture of the end effector based on the grasping posture used in user-guided grasping attempts. For example, for a VR-based user-guided grasping attempt, the grasping posture can be determined based on the virtual posture of the end effector 182B in the virtual environment during the grasping attempt. For a user-guided grasping attempt physically manipulated by the robot 180, the grasping posture can be determined based on the posture of the end effector 182A during the grasping attempt. For example, this grasping posture can be determined by the robot 180 using one or more of the robot's sensors (e.g., based on a dynamic model of the robot 180 that applies sensor data).

[0061] For training instances, the crawling results engine 116 determines a metric indicating the crawling result of a user-guided crawling attempt. In some implementations, the metric indicating the crawling result is a binary label, such as "0 / success" or "1 / failure". In some implementations, the metric indicating the crawling result can be selected from more than two options, such as 0, 1, and one or more values ​​between 0 and 1. For example, "0" can indicate a confirmed "failed crawl", "1" can indicate a confirmed successful crawl, "0.25" can indicate "most likely unsuccessful crawl", and "0.75" can indicate "most likely successful crawl".

[0062] As an example, for a VR-based user-guided grasping attempt, if at least a portion of the glasses 189B is positioned between the grasping members after the grasping member of the end effector 182B is actuated to a closed or "partially closed" state in the virtual reality environment, the grasping result engine 116 can determine a metric indicating a successful grasping result.

[0063] As another example, for a user-guided grasping attempt physically manipulated by robot 180, grasping result engine 116 can determine the success of the grasp based on various sensor data from robot 180. For example, sensor data may include position readings, torque readings, and / or other readings from grasping end effector 182A. In such an example, grasping result engine 116 can determine a grasping success label based on (multiple) readings. For example, if the reading is a position reading, grasping result engine 116 can determine a "successful grasp" label if the reading is greater than a certain threshold (e.g., 1 cm), and a "failed grasp" label if the reading is less than a certain threshold (e.g., 1 cm). As another example, sensor data may include an image of the end effector grasping an object in the grasping attempt. For example, this image may be captured by vision component 184 after end effector 182A and any object that can be grasped by the end effector have moved to a position close to vision component 184. In such an example, grasping result engine 116 can use the captured image to determine whether the grasping attempt was successful. For example, the crawling results engine 116 can provide the captured images to the computing devices of one or more human reviewers, who can then use their computing devices to indicate whether the crawling was successful based on the captured images.

[0064] The supplementary instance engine 118 can automatically generate one or more additional training instances based on corresponding user-guided grasping attempts (virtual or physical), and can interact with one or more of other engines 112, 114, and 116 during the generation of additional training instances. The training instance generation system 110 can store the additional training instances in the training instance database 170. For example, when generating additional training instances, the supplementary instance engine 118 can determine a new grasping pose based on the grasping pose from the user-guided grasping attempt. For an instance, a new grasping pose can be selected that is within the expected variance of the grasping pose from the user-guided grasping attempt (e.g., position of + / - X centimeters, and / or orientation of + / - X degrees). Furthermore, the object and scene from the user-guided grasping attempt can be utilized (e.g., the same object pose can be utilized), and new grasping attempts can be made using the new grasping pose (in a VR environment or a physical environment). Various techniques, such as those described herein, can be used to determine whether the new grasping attempt is successful. Various data from the new grasping attempt can be used to generate additional training instances. For example, new grasping poses and new grasping results can be used to generate additional training instances. Images from new grasping attempts and / or from user-guided attempts can also be included in the additional training instances (e.g., static images from user-guided attempts can be reused and new pre-grab images can be determined based on the additional training instances).

[0065] although Figure 1 Specific objects 189A and 189B are shown, but in many implementations, training instances are generated based on user-guided grasping attempts associated with various objects (physical or virtual) and / or various scenarios. For example, for virtual user-guided grasping attempts, one or more users may use corresponding computing devices to attempt to virtually grasp a cup placed on a first surface, one or more users may use corresponding computing devices to attempt to virtually grasp a bowl placed on a second surface, one or more users may use corresponding computing devices to attempt to virtually grasp a stapler placed on a third surface, and so on. Such diversity of target objects and / or scenarios can lead to more diverse training instances, thereby improving the performance of networks 120 and / or 140 trained based on such training instances.

[0066] Training engine 130 uses training instances from training instance database 170 to train multiple neural network models of geometric network 120 and / or train multiple neural network models of grasping result prediction network. Training engine 130 may include one or more GPUs, TPUs, and / or CPUs.

[0067] refer to Figure 4 This provides examples of how to train the geometry network 120 and the grasping result prediction network 140. Although Figure 4 Training engine 130 is not shown, but training engine 130 can perform [the following actions]. Figure 4 Example training. For example, training engine 130 can be applied. Figure 4 The training instances in the dataset are generated. Figure 4 Multiple losses are considered, and updates are made based on these multiple losses. Figure 4 Various neural networks in [the context]. In [the context]... Figure 4 For simplicity, a single training instance is shown here, and a description of that single training instance is provided below. However, it should be understood that... Figure 4 The training of various neural networks will be based on a large number (e.g., tens of thousands or more) of training instances, and these neural networks will be updated based on these training instances.

[0068] exist Figure 4 In, it is shown Figure 3 Example geometric network 120. Furthermore, in Figure 4 The diagram illustrates a CNN 3D decoder 126, a global shape sampler, a geometric loss engine 175, and a grasping loss engine 177. In some implementations, loss engines 175 and 177 may be components of the training engine 130.

[0069] exist Figure 4In this process, a still image 171A1 and a static visual sensor pose 172A1 are applied as input to a CNN encoder 122. Both the still image 171A1 and the static visual sensor pose 172A1 are training instances from a training instance database 170. Based on the applied still image 171A1 and static visual sensor pose 172A1, a global geometric representation 423 is generated using the CNN encoder 122. The global geometric representation 423 is an encoding. For example, the encoding can be a 1×1×512 value matrix.

[0070] Global geometric representation 423 is applied as input to CNN 3D decoder 126, which may be a deconvolutional neural network. Decoding of global geometric representation 423 is generated using CNN 3D decoder 126, and the decoding of global geometric representation 423 is in the form of 3D voxel representation 427. For example, voxel representation 427 may be a 32×32×32 (or other dimensional) value matrix, where each value indicates multiple features (e.g., occupied / unoccupied) of the corresponding voxel. Voxel representation 427 is provided to global shape sampler 128, which converts voxel representation 427 into a predicted image 428 with the same resolution as static image 171A1. Global shape sampler 128 uses static visual sensor pose 172A1 to convert voxels into predicted image 428, such that predicted image 428 is a predicted image with the same pose as static visual sensor pose 172A1.

[0071] The geometric loss engine 175 generates a loss 176 based on a comparison of the predicted image 428 and the still image 171A1. Therefore, the generated loss 176 is a function of the difference between the predicted image 428 and the still image 171A1. In some implementations, the generated loss 176 is a function of the difference in the depth dimension of the two images 428 and 171A1, and optionally a function of the difference in the "object mask" between the two images 428 and 171A1. The "object mask" of an image can be the 2D contour of an object captured in the image. The loss 176 is used to update the CNN 3D decoder 126 and / or the CNN encoder 122. For example, the loss can be backpropagated through both the CNN 3D decoder 126 and the CNN encoder 122.

[0072] Therefore, through the generation of loss 176 as described above, CNN encoder 122 is updated to encourage the generation of encodings representing rich geometric features of objects(s) in an image based on 2D or 2.5D images—because encoding 423 is being applied to CNN3D encoder 126 to generate 3D representations in voxel form, and the accuracy of the generated 3D representation is indirectly determined as a function of loss 176 by generating prediction image 428 and comparing prediction image 428 with static image 171A1.

[0073] It is worth noting that the CNN 3D decoder 126 and the global shape sampler 128 are in Figure 4 They are used to train the CNN encoder 122. However, in many implementations, they are not utilized during the inference phase, such as... Figure 3 As illustrated in the example. In other words, these encodings are used for training to enable the training of a CNN encoder 122, which can then be used to generate geometrically aware encodings of images. As described herein, those encodings can then be applied as direct input to a grab result prediction network 140, and the grab result prediction network 140 can be used to determine grab result predictions based on these encodings. Thus, in those implementations, the grab result prediction network 140 can accept high-dimensional encodings of geometric features and generate outputs based on these high-dimensional encodings of geometric features, instead of low-dimensional (and computationally more burdensome) full 3D representations.

[0074] exist Figure 4 In this process, the local shape sampler 124 uses the global geometric representation 423 and the grasping pose 173A of the training instance to generate the local geometric representation 425.

[0075] Global geometric representation 423, local geometric representation 425, and grasping pose 173A are applied as inputs to the grasping result prediction network 140. Furthermore, pre-grasped images 171A2 of training instances and optional pre-grasping visual sensor poses 172A2 are also applied as inputs to the grasping result prediction network 140. As described herein (e.g., Figure 3 In some implementations, the grasping result prediction network 140 may include multiple individual neural networks (each neural network including multiple layers), and each input may be applied to the corresponding neural network(s). For example, the pre-grabbed image 171A2 and the pre-grabbed visual sensor pose 172A2 may be used as inputs to CNN 142 (…). Figure 3 The output from CNN 142 and global geometry 423 are applied to the combination layer 143, and the output from the combination layer 143 ( Figure 3 ) and grasping posture 473A as input applied to the first MLP 144 ( Figure 3), and the output from the first MLP 144 and the local geometry 425 are applied as input to the second MLP 144 ( Figure 3 ).

[0076] Based on the applied input, crawl result prediction network 140 is used to generate crawl result prediction 447.

[0077] The grasping loss engine 177 generates a loss 178 based on a comparison between the grasping result prediction 447 and the actual grasping result 177A2 of the training instance. Therefore, the generated loss 178 is a function of the difference between the grasping result prediction 447 and the actual grasping result 177A2. The loss 178 is used to update the grasping result prediction network 140. For example, the loss can be backpropagated through the grasping result prediction network 140. In some implementations, the loss 178 can also be used to update the CNN encoder 122 and / or the CNN 3D decoder 126.

[0078] Figure 4 An example is shown of training both the CNN encoder 122 and the grab result prediction network 140 based on training instances. However, in some implementations, the CNN encoder 122 can be "pre-trained" at least independently of any training of the grab result prediction network 140. For example, the CNN encoder 122 can be pre-trained using still images and still camera poses from training instances, and utilizing the CNN 3D decoder 126, global shape sampler 128, and geometric loss engine 175 as described above. The grab result prediction network 140 can then be trained together with the CNN encoder 122 (e.g., using the global geometric representation generated by the CNN encoder 122 based on the training instances). In some of those implementations, the CNN encoder 122 can continue to be updated during such subsequent training. For example, the CNN encoder 122 can continue to be updated based on the loss generated by the grab loss engine 177 and / or based on the loss generated by the geometric loss engine 175 (if engine 175 and / or the CNN 3D decoder 126 and global shape sampler 128 are utilized in such subsequent training). Furthermore, Figure 4 An example of a single still image 171A1 and a loss 176 determined based on the single still image 171A1 is shown. However, in many implementations, multiple still images from the scene (e.g., each still image captured from a different visual sensor pose) can be utilized, a predicted image can be determined for each still image, and the loss 176 can be a multiple loss based on the differences between all predicted images and their corresponding still images.

[0079] In some implementations, when converting the voxel representation 427 into a predicted image 428, the global shape sampler 128 performs a projection operation that uses a vision sensor transformation matrix based on the static vision sensor pose 172A1 to convert the voxel representation 427 into a 2.5D image (2D with depth mapping). This may include: performing dense sampling from the voxel representation 427 (in a 3D world frame) to the output volume (in normalized coordinates); and flattening the 3D spatial output in one dimension. Each 3D point from the voxel representation 427 and each corresponding point in the output volume may be associated by the global shape sampler 128 using the vision sensor transformation matrix.

[0080] Turn now Figure 5 -8 provides additional descriptions of the various methods described in this paper.

[0081] Figure 5 This is a flowchart illustrating an example method 500 for generating training instances based on user-guided crawling attempts. For convenience, the operations of the flowchart are described with reference to the system performing these operations. This system may include one or more components of one or more computing systems (e.g., computing device 105 and / or training instance generation system 110). Furthermore, although the operations of method 500 are shown in a specific order, this is not intended to be restrictive. One or more operations may be reordered, omitted, or added.

[0082] In box 552, the user-guided crawling attempt begins.

[0083] In box 554, the system stores one or more still images of the scene and corresponding visual sensor poses(s). For example, in the case where the user-guided grasping attempt is via a VR environment, the system can render one or more still images, each rendered from a virtual visual sensor pose that is targeting or near the target object of the grasping attempt, and that the virtual visual sensor pose is at a corresponding distance from the target object and at a corresponding orientation and height. In some implementations, one or more of the still images may omit any robot end effector.

[0084] In box 556, the system stores the grasping posture of a user-guided grasping attempt. For example, for a VR-based user-guided grasping attempt, the system can determine the grasping posture based on the virtual posture of the end effector in the virtual environment at the time of the grasping attempt.

[0085] In box 558, the system determines and stores the actual grasping result of the user-guided grasping attempt, such as a metric indicating the actual grasping result. As an example, for a VR-based user-guided grasping attempt, if at least a portion of the target object is positioned between the grasping members after the grasping member of the end effector is actuated to a closed or “partially closed” state in the virtual reality environment, the system can determine a metric indicating a successful grasping result.

[0086] In box 560, the system generates a training instance, which includes: (multiple) static images and corresponding (multiple) static visual sensor poses stored in box 554, a grasping pose stored in box 556, a pre-grab image captured before the grasping attempt, a pre-grab visual sensor pose of an optional pre-grab image, and the actual grasping result in box 558. The pre-grab image and optional pre-grab visual sensor pose capture the target object and the end effector. In some cases, a pre-grab image can be captured when the end effector is in the grasping pose, or when the end effector is in a pose prior to the grasping pose, which occurs within at least a threshold time duration of the grasping pose.

[0087] In box 562, the system automatically generates additional training instances based on user-guided crawling attempts. In some implementations, subboxes 562A-562A are executed when generating additional training instances based on user-guided crawling attempts.

[0088] In box 562A, the system determines a new grasping pose for a grasping attempt based on the stored grasping poses of user-guided grasping attempts (grasping poses in box 556). For example, the system may determine the new grasping pose based on its expected variance within the expected variance of the grasping poses of user-guided grasping attempts. For example, the system may determine the new grasping pose based on pseudo-random selection of a new grasping pose from a normal distribution of poses, where the normal distribution of poses has the mean and expected variance of the stored grasping poses (e.g., 5 cm as the standard deviation of position and 20 degrees as the standard deviation of orientation).

[0089] In box 562B, the system attempts to grab the object with a new grabbing posture.

[0090] In box 562C, the system determines and stores new actual crawl results, such as metrics indicating the new actual crawl results.

[0091] In box 562D, the system generates additional training instances, which include: static images (e.g., the image in box 554 and / or (multiple) additional static images) and corresponding static visual sensor poses, a new grasping pose in box 562A, a new pre-grasp image and optionally a new pre-grasp visual sensor pose, and a new actual grasping result in box 562C. In some cases, a new pre-grasp image can be captured when the end effector is in a new grasping pose, or when the end effector is in a pose prior to the new grasping pose.

[0092] Figure 6 This is a flowchart illustrating an example method for pre-training a geometric network based on training instances. For convenience, the operations of the flowchart are described with reference to the system performing these operations. This system may include one or more components of one or more computational systems (e.g., training engine 130). Furthermore, although the operations of method 600 are shown in a specific order, this is not intended to be restrictive. One or more operations may be reordered, omitted, or added.

[0093] In box 654, the system selects a training instance.

[0094] In box 656, the system generates encodings based on applying static images of training instances to the CNN encoder of the geometric network.

[0095] In box 658, the system generates voxel representation outputs based on applying encoding to the CNN 3D decoder.

[0096] In box 660, the system generates a predicted image based on the projection of the voxel output representation, where the projection of the voxel output representation is based on the visual sensor pose of a static image of a training instance.

[0097] In box 662, the system updates the geometric network based on the predicted image and the still image. For example, the system may generate a loss based on a comparison of the predicted image and the still image, and backpropagate this loss through the CNN encoder and / or CNN 3D decoder. In some implementations, the system generates a loss that is a function of the difference in the depth dimension between the predicted image and the still image, and optionally a function of the difference in the “object mask” difference between the two images.

[0098] In box 664, the system determines whether additional training instances should be processed. If so, the system proceeds to box 654 and selects additional training instances. If not, the system proceeds to box 666, and the geometric network pre-training ends. In some implementations, determining whether additional training instances should be processed may include: determining whether additional unprocessed training instances are available and / or whether one or more training criteria (e.g., multiple epoch criteria, training duration criteria) have been met.

[0099] Figure 7 This is a flowchart illustrating an example method 700 for training a grasping result prediction network and an optional geometric network based on training instances. For convenience, the operations of the flowchart are described with reference to the system performing these operations. This system may include one or more components of one or more computational systems (e.g., training engine 130). Furthermore, although the operations of method 700 are shown in a specific order, this is not intended to be restrictive. One or more operations may be reordered, omitted, or added.

[0100] In box 752, the system begins training.

[0101] In box 754, the system selects a training instance.

[0102] In box 756, the system generates encodings based on applying static images of training instances to the CNN encoder of the geometric network.

[0103] In box 758, the system generates a local geometric representation based on applying the encoding and end effector pose to the local shape sampler.

[0104] In box 760, the system generates a predicted grasping result by applying the following to the grasping result prediction network: the encoding of box 756; the local geometric representation of box 758; and the grasping pose of the training instance, the pre-grab image, and optionally the pre-grab visual sensor pose.

[0105] In box 762, the system performs backpropagation based on the predicted grasping result from box 760 and the actual grasping result of the training instance. For example, the system may generate a loss based on a comparison between the predicted and actual grasping results, and backpropagate this loss through grasping result prediction network 140 and optionally through CNN encoder 122.

[0106] Figure 8 An example method is illustrated that utilizes a geometric network and a grasping result prediction network to determine when to attempt a grasp based on candidate grasping poses. For convenience, the operations of the flowchart are described with reference to a system performing these operations. This system may include one or more processors of a robot, such as one or more processors of robot 180, robot 925, and / or other robots, and / or a robot control system. When implementing one or more boxes of method 800, the system may operate using a trained neural network model, for example, which may be stored locally on the robot and / or remotely. Furthermore, although the operations of method 800 are shown in a specific order, this is not intended to be restrictive. One or more operations may be reordered, omitted, or added.

[0107] In box 852, the system determines candidate grasping poses for the robot's end effector. For example, the system can use optimization techniques such as CEM and / or other optimization techniques to determine candidate grasping poses.

[0108] In box 854, the system identifies the current image captured by the robot's vision sensors. The current image can be a recently captured image or an image captured within a threshold time period of the current time.

[0109] In box 856, the system uses the trained encoder neural network to generate an encoding for the current image or an additional image. When the encoding is for an additional image, the additional image can be an image captured earlier than the current image, and the robot's end effector can optionally be omitted, or the end effector can be included in the pose of an object in the additional image without occluding it. When generating the encoding using the trained encoder neural network, the visual sensor pose of the corresponding image can also optionally be utilized.

[0110] In box 858, the system applies the current image of box 854, the candidate grasping pose of box 852, and the encoding of box 856 as one or more inputs to the trained grasping outcome prediction network. In some implementations, the system also applies a local geometric representation (e.g., a local geometric representation generated from the perspective of the candidate grasping pose based on dense sampling of the encoding of box 856) to the trained grasping outcome prediction network.

[0111] In box 860, the system generates a grab result prediction using a trained grab result prediction network based on one or more inputs.

[0112] In block 862, the system determines whether the grasping result prediction generated in block 860 meets one or more criteria (e.g., thresholds). If it does, the system proceeds to block 864 and causes the end effector to attempt grasping in a candidate grasping pose. For example, the system may provide one or more control commands to the robot's actuators to cause the end effector to move to a candidate grasping pose (if it is not already in a candidate grasping pose) and attempt grasping in the candidate grasping pose (e.g., closing one or more actuated members).

[0113] If the conditions are not met, the system returns to box 852 and determines additional candidate grasping poses. The system can then use the additional candidate grasping poses to perform another iteration of boxes 858, 860, and 862 (the system can reuse the encoding from previous iterations or generate new encoding (e.g., if the scene has changed)).

[0114] In some implementations, execution can be parallel. Figure 8Multiple iterations of one or more boxes. For example, in some implementations, multiple candidate grasping poses may be considered in parallel (e.g., multiple iterations of boxes 858 and 860 are performed using the same encoding), and box 862 may include determining whether any of the multiple candidates meets one or more criteria. In some implementations, multiple iterations of boxes 852, 854, 856, 858, and 860 may be used to sample (according to optimization techniques) multiple candidate end effector poses in a given iteration. In some of those implementations, the sampled candidate end effector pose that has the best grasping result prediction at box 860 can be selected from those sampled candidate end effector poses as the determined grasping pose to be implemented. Thus, in some versions of those implementations, multiple candidate grasping poses may be sampled in each iteration according to optimization techniques, and in each iteration, only the best (according to optimization techniques) sampled candidate grasping pose may be selected for actual implementation (e.g., for controlling the end effector). In these and other ways, multiple candidate grasping poses are sampled / considered in each iteration, but only one candidate grasping pose is selected and implemented in each iteration.

[0115] Figure 9 An example architecture of robot 925 is schematically depicted. Robot 925 includes a robot control system 960, one or more operating components 925a-925n, and one or more sensors 942a-942m. For example, sensors 942a-942m may include vision sensors, light sensors, pressure sensors, pressure wave sensors (e.g., microphones), proximity sensors, accelerometers, gyroscopes, thermometers, barometers, etc. Although sensors 942a-m are depicted as integrated with robot 925, this is not intended to be limiting. In some embodiments, sensors 942a-m may be located external to robot 925, for example, as separate units.

[0116] For example, operating components 925a-925n may include one or more end effectors and / or one or more servo motors or other actuators to achieve movement of one or more parts of the robot. For example, robot 925 may have multiple degrees of freedom, and each actuator may control the actuation of robot 925 in one or more of these degrees of freedom in response to control commands. As used herein, the term actuator includes mechanical or electrical devices (e.g., electric motors) capable of generating motion, except for any driver(s) that may be associated with an actuator and translate received control commands into signals for driving one or more actuators. Therefore, providing control commands to an actuator may include providing control commands to a driver(s) that translate the control commands into appropriate signals for driving electrical or mechanical devices to produce the desired motion.

[0117] The robot control system 960 may be implemented in one or more processors, such as the CPU, GPU, and / or other controller(s) of the robot 925. In some embodiments, the robot 925 may include a “brain box,” which may include all or aspects of the control system 960. For example, the brain box may provide real-time bursts of data to the operating components 925a-n, each real-time burst including a set of one or more control commands that specifically define motion parameters (if any) for each of one or more of the operating components 925a-n. In some embodiments, the robot control system 960 may perform one or more aspects of the method 800 described herein.

[0118] As described herein, in some embodiments, all or aspects of the control commands generated by the control system 960 when positioning the end effector to grasp an object can be based on end effector commands, wherein the end effector commands are generated based on grasp outcome predictions(a) generated using a trained grasp outcome prediction network. For example, the vision sensor of sensor 942a-m can capture a current image and a still image (before the current image), and the robot control system 960 can generate candidate grasp poses. The robot control system 960 can generate encoding based on applying the still image and the corresponding still vision sensor pose as input to a trained CNN encoder, as described herein. Furthermore, the robot control system 960 can apply the encoding, candidate grasp poses, and the current image and the corresponding current vision sensor pose as input to a trained grasp outcome prediction network to determine a grasp success metric using the candidate grasp poses. If the grasp success metric meets a threshold and / or other criteria, the control system 960 can provide control commands to cause the robot's end effector to attempt grasping with the candidate grasp poses. Although the control system 960 in Figure 9 While shown as a component of robot 925, in some embodiments, all or aspects of control system 960 may be implemented in a component separate from but communicating with robot 925. For example, all or aspects of control system 960 may be implemented on one or more computing devices (e.g., computing device 1010) that communicate with robot 925 via wired and / or wireless communication.

[0119] Figure 10This is a block diagram of an example computing device 1010, which may optionally be used to perform one or more aspects of the techniques described herein. For example, in some embodiments, computing device 1010 may be used to train various neural network models, generate training instances, and / or render VR environments. Computing device 1010 typically includes at least one processor 1014 that communicates with a plurality of peripheral devices via a bus subsystem 1012. These peripheral devices may include a storage subsystem 1024 (e.g., including a memory subsystem 1025 and a file storage subsystem 1026), a user interface output device 1020, a user interface input device 1022, and a network interface subsystem 1016. The input and output devices allow users to interact with computing device 1010. The network interface subsystem 1016 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0120] User interface input device 1022 may include a keyboard, pointing device (e.g., mouse, trackball, touchpad, or graphics tablet), scanner, touchscreen integrated into the display, audio input device (e.g., speech recognition system, microphone), and / or other types of input devices. Generally, the term "input device" is used to include all possible types of devices and methods for inputting information into computing device 1010 or a communication network.

[0121] User interface output device 1020 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LED), a projection device, or other mechanisms for creating visual images. The display subsystem may also provide non-visual displays, such as via an audio output device. Generally, the term "output device" is used to encompass all possible types of devices and methods for outputting information from computing device 1010 to a user or another machine or computing device.

[0122] Storage subsystem 1024 stores the programming and data structures that provide the functionality of some or all of the modules described herein. For example, storage subsystem 1024 may include execution... Figure 5 , 6 The logic of the selected aspects of methods 7 and / or 8.

[0123] These software modules are typically executed by processor 1014 alone or in combination with other processors. The memory 1025 used in storage subsystem 1024 may include multiple memories, including main random access memory (RAM) 1030 for storing instructions and data during program execution and read-only memory (ROM) 1032 for storing fixed instructions. File storage subsystem 1026 can provide persistent storage for program and data files and may include hard disk drives, floppy disk drives, and associated removable media, CD-ROM drives, optical disk drives, or removable media cartridges. Modules implementing the functionality of certain embodiments may be stored by file storage subsystem 1026 in storage subsystem 1024 or in other machines accessible to processor(s) 1014.

[0124] The bus subsystem 1012 provides a mechanism for allowing the various components and subsystems of the computing device 1010 to communicate with each other as intended. Although the bus subsystem 1012 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0125] The computing device 1010 can be of various types, including workstations, servers, computing clusters, blade servers, server groups, or any other data processing system or computing device. Due to the constantly evolving nature of computers and networks, Figure 10 The description of the computing device 1010 depicted herein is intended only as a specific example to illustrate some implementation methods. Many other configurations of the computing device 1010 may have... Figure 10 The computing device shown has more or fewer components.

[0126] While several embodiments have been described and illustrated herein, various other means and / or structures may be utilized for performing functions and / or obtaining results and / or one or more advantages described herein, and each of these variations and / or modifications is considered to be within the scope of the embodiments described herein. More generally, all parameters, dimensions, materials, and configurations described herein are exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications to which these teachings are applied. Those skilled in the art will recognize, or be able to determine, many equivalents of the specific embodiments described herein using only conventional experimentation. Therefore, it should be understood that the foregoing embodiments are given by way of example only, and that embodiments may be practiced in ways other than those specifically described and claimed within the scope of the appended claims and their equivalents. Embodiments of this disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods that are not inconsistent with each other is included within the scope of this disclosure.

Claims

1. A method implemented by one or more processors, comprising: User-guided robot grasping attempts based on objects in a virtual reality environment generate training instances, which include: A user-guided robot attempts to grasp one or more rendered images, which are generated by a virtual reality environment and capture objects. The grasping posture of a user-guided robot grasping attempt, wherein the grasping posture is based on the virtual posture of the virtual robot end effector in the virtual environment during the user-guided robot grasping attempt, and The grasping result is measured to indicate whether the user-guided robot grasping attempt on the object was successful. Multiple additional training instances are automatically generated based on user-guided robot grasping attempts, each of which includes: The new grasping posture is determined within the variance of the grasping postures from user-guided robot grasping attempts. Utilize the new grasping posture to attempt new grasping methods for objects; Determine whether the new crawling attempt was successful; and Generate additional training instances to include the new grasping pose, one or more new images from the new grasping attempt, and new grasping outcome measurements based on whether the new grasping attempt was successful; and The neural network model is trained using training instances and additional training instances.

2. The method according to claim 1, wherein, During a user-guided robot grasping attempt on an object, the virtual reality environment rendering includes the virtual environment of the object, the virtual robot end effector, and one or more additional scene objects.

3. The method according to claim 2, wherein, During the user-guided robot grasping attempt on the object, the virtual reality environment also renders one or more additional robot components in addition to the virtual robot end effector.

4. The method according to claim 1, wherein, Try new grabbing techniques in a virtual reality environment.

5. The method according to claim 4, wherein, The new grasping attempt utilizes an object in the same object pose as the object in the user-guided robot grasping attempt.

6. The method according to claim 4, wherein, Determining whether a new grabbing attempt is successful involves utilizing the physics engine of the virtual reality environment.

7. The method according to claim 1, wherein, Determining the new grasping pose based on the variance of grasping poses from user-guided robot grasping attempts includes: The new grasping posture is determined by pseudo-randomly selecting a new grasping posture from a normal distribution of postures with mean and variance.

8. The method according to claim 1, wherein, The neural network model is a neural network for predicting the results of data capture.

9. The method according to claim 8, further comprising: After training the neural network model using the training instances and additional training instances: Store the neural network model locally on the robot; The robot uses a neural network model to locate its end effector in order to grasp an attached object.

10. The method according to claim 1, wherein, The variance includes positional variance and orientation variance.

11. A system for predicting robot grasping, comprising: One or more computers, the one or more computers including one or more processors, the one or more processors being operable to execute stored instructions to perform a method, the method comprising: User-guided robot grasping attempts based on objects in a virtual reality environment generate training instances, which include: A user-guided robot attempts to grasp one or more rendered images, which are generated by a virtual reality environment and capture objects. The grasping posture of a user-guided robot grasping attempt, wherein the grasping posture is based on the virtual posture of the virtual robot's end effector in the virtual environment during the user-guided robot grasping attempt, and The grasping result is measured to indicate whether the user-guided robot grasping attempt on the object was successful. Multiple additional training instances are automatically generated based on user-guided robot grasping attempts, each of which includes: The new grasping posture is determined within the variance of the grasping postures from user-guided robot grasping attempts. Utilize the new grasping posture to attempt new grasping methods for objects; Determine whether the new crawling attempt was successful; and Generate additional training instances to include the new grasping pose, one or more new images from the new grasping attempt, and new grasping outcome measurements based on whether the new grasping attempt was successful; and The neural network model is trained using training instances and additional training instances.

12. The system according to claim 11, wherein, During a user-guided robot grasping attempt on an object, the virtual reality environment rendering includes the virtual environment of the object, the virtual robot end effector, and one or more additional scene objects.

13. The system according to claim 12, wherein, During the user-guided robot grasping attempt on the object, the virtual reality environment also renders one or more additional robot components in addition to the virtual robot end effector.

14. The system according to claim 13, wherein, Try new grabbing techniques in a virtual reality environment.

15. The system according to claim 14, wherein, The new grasping attempt utilizes an object in the same object pose as the object in the user-guided robot grasping attempt.

16. The system according to claim 14, wherein, Determining whether a new grabbing attempt is successful involves utilizing the physics engine of the virtual reality environment.

17. The system according to claim 11, wherein, Determining the new grasping pose based on the variance of grasping poses from user-guided robot grasping attempts includes: The new grasping posture is determined by pseudo-randomly selecting a new grasping posture from a normal distribution of postures with mean and variance.

18. The system according to claim 11, wherein, The neural network model is a neural network for predicting the results of data capture.

19. The system according to claim 18, wherein, The method also includes: after training the neural network model using training instances and additional training instances: This allows the neural network model to be stored locally on the robot.

20. The system according to claim 11, wherein, The variance includes positional variance and orientation variance.

Images