Hybrid machine learning based system and method for training an object picking robot using real performance data and simulated performance data
Through a hybrid machine learning system combining real and simulated data, evaluating and optimizing object picking positions, the problem of limited data sources in the existing technology is solved, the efficiency and accuracy of robot training is improved, and efficient object picking and industrial applications are achieved.
Patent Information
- Application Number
- CN202080024548.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-03-27
- Filing Date
- 2020-03-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-03-24
AI Technical Summary
In the prior art, machine learning-based robot training systems only utilize limited data sources, resulting in reduced efficiency, speed and accuracy of machine learning models, and lack of multi-source collaborative utilization of object pickup performance data, resulting in improved industrial process efficiency. Loss of opportunity.
The hybrid machine learning system is adopted, combining data from real and simulated environments, and the quality of the grab is evaluated through sensor feedback system, and the running time and fine-tuning of the robot grab is performed through online learning to optimize the picking position of the object.
Improve the accuracy, speed and efficiency of object picking, reduce the error rate, reduce the demand for CPU clock cycles, continuously evaluate and monitor the grab performance, enhance the accuracy and robustness of the ML model, and improve computing and storage , network bandwidth and power resource utilization rate.
Smart Images

Figure CN113646138B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This patent application claims the benefit of U.S. Patent Application No. 16 / 366,700, filed on March 27, 2019, which is incorporated herein by reference for all purposes. Technical Field
[0003] The present disclosure relates to robotic grasping of objects, and more particularly to systems and methods for machine learning-based training of object picking robots. Background Art
[0004] In at least some known systems and methods for machine learning-based robot training, limited data sources are utilized, which may reduce the efficiency, speed, and accuracy of the resulting machine learning models. Known systems and methods may utilize only data from real-world robot picking experiments. Likewise, known robot training systems and methods may utilize only synthetic data generated in a virtual environment based on the physical properties of objects. Regardless, the lack of coordinated utilization of numerous sources of robot grasping performance data from actual and simulated experiments and in training and run-time machine learning environments may result in lost efficiency improvement opportunities for object picking robot training and the various industrial processes in which such systems are applied. Summary of the invention
[0005] The systems and methods disclosed herein for hybrid machine learning (ML) based training of object picking robots using real grasping performance data and simulated grasping performance data present a new and improved processing pipeline for solving the robotic picking problem. A hybrid machine learning engine is used to train the robot to learn grasp positions in simulated and real world environments. A sensor feedback system fuses multiple sensor signals to assess grasp quality. The disclosed systems and methods also enable run-time / online self-correction and fine-tuning of robotic grasping through online learning.
[0006] In one aspect, a method for training an object picking robot using real grasping performance data and simulated grasping performance data is provided. The method includes assigning a plurality of grasping locations on an object based on known or estimated physical properties of the object. The method includes performing a first simulation experiment for a robot to grasp an object using a first set of assigned grasping locations from a plurality of assigned grasping locations. The method includes evaluating a grasp quality of a simulated object grasped by the robot for each assigned grasping location in the first set of assigned grasping locations based on a first set of simulation data from the first simulation experiment. The method includes determining a first set of candidate grasping locations on the object based on data representing the simulated grasping quality obtained in an evaluation step for simulating grasping quality of the object. The method includes evaluating an actual object grasping quality of a grasped object by the robot for each candidate grasping location in the first set of candidate grasping locations based on a first set of grasping quality sensor data, the first set of grasping quality sensor data coming from a first actual experiment for a robot to grasp an object using each candidate grasping location in the first set of candidate grasping locations. The method includes determining, for each candidate grasp position in a first set of candidate grasp positions, convergence of actual object grasp quality and simulated object grasp quality based on a first set of grasp quality sensor data from a first actual experiment and data representing simulated grasp quality.
[0007] In another aspect, a system for training an object picking robot using real grasping performance data and simulated grasping performance data is provided. The system includes one or more memory devices, and one or more processors in communication with the one or more memory devices and the object picking robot. The one or more processors are programmed to assign a plurality of grasping positions on an object based on known or estimated physical properties of the object. The one or more processors are programmed to perform a first simulation experiment for the robot to grasp an object using a first set of assigned grasping positions from a plurality of assigned grasping positions. The one or more processors are programmed to evaluate the grasp quality of a simulated object grasped by the robot for each assigned grasping position in the first set of assigned grasping positions based on a first set of simulation data from the first simulation experiment. The one or more processors are programmed to determine a first set of candidate grasping positions on the object based on data representing the simulated grasping quality obtained when evaluating the simulated grasping quality. One or more processors are programmed to evaluate the actual object grasp quality of the robot grasping for each candidate grasping position in a first set of candidate grasping positions based on a first set of sensor data, the first set of sensor data being from a first actual experiment in which the robot grasps an object using each candidate grasping position in the first set of candidate grasping positions.
[0008] In yet another aspect, a non-transitory computer-readable storage medium storing processor-executable instructions for training an object picking robot using real grasping performance data and simulated grasping performance data is provided. The processor-executable instructions, when executed by one or more processors, cause the one or more processors to: (a) assign multiple grasping locations on an object based on known or estimated physical properties of the object; (b) perform a first simulation experiment for a robot to grasp an object using a first set of assigned grasping locations from a plurality of assigned grasping locations; (c) evaluate the grasp quality of a simulated object grasped by the robot for each assigned grasping location in the first set of assigned grasping locations based on a first set of simulation data from the first simulation experiment; (d) determine a first set of candidate grasping locations on the object based on data representing the simulated grasping quality obtained when evaluating the simulated grasping quality; and (e) evaluate the grasp quality of an actual object grasped by the robot for each candidate grasping location in the first set of candidate grasping locations based on a first set of grasping quality sensor data, the first set of grasping quality sensor data being from the first actual experiment.
[0009] Compared to known systems and methods, the systems and methods disclosed herein for hybrid ML-based training of object picking robots using real grasping performance data and simulated grasping performance data provide users with many beneficial technical effects and achieve various advantages. Such benefits include, but are not limited to, enabling the picking and placing of (one or more) objects with improved accuracy, speed, efficiency, and reduced error rates compared to known processes. The disclosed systems and methods are used for hybrid ML-based training of object picking robots, thereby reducing the required number of CPU clock cycles per object required by (one or more) processors in both training and run-time environments. The systems and methods for hybrid ML-based training of object picking robots described herein enable continuous evaluation and monitoring of object grasping performance, so that hybrid ML operations can be implemented as needed to fine-tune and enhance the accuracy and robustness of ML models, including across multiple robots involved in unit operations. Therefore, the disclosed systems and methods for hybrid ML-based training of object picking robots enable efficient and effective training of object picking robots in various industrial applications where it is desired to improve the utilization of computing, memory, network bandwidth, power, and / or human resources.
[0010] Additional and alternative aspects and features of the disclosed principles will be understood from the following detailed description and accompanying drawings. It will be appreciated that the principles associated with the disclosed system and method for hybrid machine learning-based training of an object picking robot using real grasping performance data and simulated grasping performance data can be implemented in other different embodiments and can be modified in various aspects. Therefore, it should be understood that the foregoing summary and the following detailed description are exemplary and illustrative only and do not limit the scope of the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 is a schematic diagram of a system for hybrid machine learning-based training of an object picking robot using real grasping performance data and simulated grasping performance data according to an embodiment of the present disclosure.
[0012] Figure 2 The use of the embodiments according to the present disclosure Figure 1 A flow chart of a method of the system shown for hybrid machine learning-based training of an object picking robot using real grasping performance data and simulated grasping performance data.
[0013] Figure 3 According to the embodiment of the present disclosure, the Figure 1 Schematic diagram of objects in the simulated or actual workspace of the robot of the system shown.
[0014] Figure 4 The use of the embodiments according to the present disclosure Figure 1 The system shown and Figure 2 A schematic diagram of the process of the method shown for hybrid machine learning-based training of an object picking robot using real grasping performance data and simulated grasping performance data.
[0015] Figure 5 According to the embodiment of the present disclosure Figure 1 A block diagram of the memory data structure of the system is shown.
[0016] Figure 6 According to an embodiment of the present disclosure, Figure 2 Block diagram of the software architecture of the method shown.
[0017] Figure 7 According to the embodiment of the present disclosure Figure 2 Flowcharts illustrating aspects of the method. DETAILED DESCRIPTION
[0018] Reference will now be made in detail to specific embodiments or features, examples of which are shown in the accompanying drawings. Where possible, corresponding or similar reference numerals will be used throughout the accompanying drawings to refer to identical or corresponding parts. In addition, when there may be more than one element of the same type, references to the various elements described herein are made jointly or individually. However, such references are merely exemplary in nature. It may be noted that, unless expressly provided in the appended claims, any reference to an element in the singular may also be interpreted as involving the plural, and vice versa, without limiting the scope of the present disclosure to the exact number or type of such elements.
[0019] Figure 1 is a schematic diagram of a system (1) for hybrid machine learning (ML)-based training of an object picking robot (2) using real grasping performance data and simulated grasping performance data according to an embodiment of the present disclosure. Figure 2 is a flow chart of a method (200) for hybrid ML-based training of an object picking robot (2) using real grasping performance data and simulated grasping performance data according to an embodiment of the present disclosure. In the example shown, Figure 2 The method (200) shown uses at least in part Figure 1 is implemented by the system (1).
[0020] refer to Figure 1 , the system (1) includes one or more memory devices (5), also collectively referred to herein as memory (5). The memory (5) includes an ML model (68) stored therein. The ML model (68) can be stored in the memory (5) at least in part as a database. The system (1) includes one or more processors (3) in communication with the memory (5). The system (1) includes an object picking robot (2), the object picking robot (2) including at least one robotic arm (12) in communication with (one or more) processors (3). The object picking robot (2) is also collectively referred to herein as the robot (2). The (one or more) processors (3) include at least one transceiver (55) in communication with one or more transceivers (54) of the robot (2). In one example, the memory (5) also communicates with the robot (2). The robot (2) includes at least one object gripper device (30) operably coupled to the (one or more) robotic arms (12). The (one or more) gripper devices (30) communicate with the (one or more) processors (3). In Figure 1In the illustrated embodiment, the system (1) includes a plurality of robots (71). As used herein, "operably coupled" refers to two or more functionally related components being coupled to each other for the purpose of (one or more) collaborative mechanical motion and / or for the purpose of current flow and / or data signal flow. When such operable coupling of two or more components is for the purpose of data stream flow, the two or more components may be operably coupled via (one or more) wired connections and / or via wireless connections. The two or more components so coupled via wired and / or wireless connections may be close to each other (e.g., a first component and a second component are in the same room or in the same component housing), or they may be separated by a certain distance in physical space (e.g., a first component is located in a building different from the location of a second component).
[0021] The system (1) includes one or more sensors (22) in communication with the (one or more) processors (3). The (one or more) sensors (22) are positioned on the (one or more) robotic arms (12) and / or on the (one or more) grippers (30) and / or are positioned in the (one or more) robotic arms (12) and / or in the (one or more) grippers (30) and are operably coupled to the (one or more) robotic arms (12) and / or the (one or more) grippers (30). The (one or more) sensors (22) may be positioned and / or mounted on a structural frame of the robot (2) (e.g., a fixed camera fixed to a robot cell, a compartment and / or a safety cage, Figure 1), alternatively or additionally positioned on the (one or more) robotic arms (12) and / or on the (one or more) grippers (30) and / or positioned in the (one or more) robotic arms (12) and / or in the (one or more) grippers (30). In one example, the (one or more) sensors (22) are or include the (one or more) grip quality sensors (22). The (one or more) grip quality sensors (22) are also collectively referred to herein as the (one or more) sensors (22). The sensor(s) (22) include one or more of the following: (one or more) multi-degree-of-freedom (DOF) force-torque sensors, (one or more) cameras for monitoring and collecting visual data during robotic manipulation tasks (such as grasping (one or more) objects (11)), (one or more) multi-DOF inertial measurement sensors for measuring mechanical dynamic aspects of object (11) handling, and (one or more) directional and / or rotational motion sensors (e.g., (one or more) motor encoders, (one or more) cameras and / or (one or more) tactile sensors) for monitoring and / or detecting relative motion between the object (11) and the gripper (30). In one example, the multi-DOF force-torque sensors and / or (one or more) multi-DOF inertial measurement sensors are mounted and / or otherwise operably coupled to and / or between the robotic arm (12) and the gripper (30) for measuring the static and / or dynamic state of the object (11) relative to the gripper (30) and the grasping position on the object (11). Likewise, in this example, the sensor(s) (22) may include fixed (to ground) grip mass sensor(s) for facilitating measurement of static and / or dynamic states of the object(s) (11).
[0022] The system (1) includes a training ML environment (74) and a runtime ML environment (80) for a robot (2). The training ML environment (74) and the runtime ML environment (134) include at least one pick location (10) and at least one placement location (42). In one example, the training ML environment (74) and / or the runtime ML environment (80) are simulated computing and robot (2) (one or more) manipulation environments. In another example, the training ML environment (74) and / or the runtime ML environment (80) are actual (e.g., "real world") computing and robot (2) (one or more) manipulation environments. In yet another example, the training ML environment (74) and / or the runtime ML environment (80) are both simulated and actual computing and robot (2) (one or more) manipulation environments. One or more objects (11) are transported to and / or otherwise brought to the pick location (10) for simulated and / or actual manipulation by the system (1) including the robot (2). Figure 1 In the illustrated embodiment, there are multiple objects (11) (e.g., a first object (82) and at least a second object (94)) in the (one or more) training ML environments (74) and / or the (one or more) runtime ML environments (80), each of the first object (82) and at least the second object (94) having multiple physical characteristics, such as shape, (one or more) construction materials, weight, density, center of mass, mass distribution, height, width and / or length. The multiple object (11) physical characteristics may additionally include object (11) identification and images depicting (one or more) appearances of the (one or more) objects (11) in various poses.
[0023] The processor(s) (3) may be located in the training ML environment(s) (74) and / or the runtime ML environment(s) (80). The processor(s) (3) may be remote from the training ML environment(s) (74) and / or the runtime ML environment(s) (80). The processor(s) (3) may be collocated with the robot (2). The processor(s) (3) may be programmed to at least partially implement and / or otherwise perform the method (200) (e.g., Figure 2One or more of the disclosed steps of the system (shown), including but not limited to using the system (1). The (one or more) processors (3) are capable of performing multiple functions in the system (1). The (one or more) processors (3) include robot (12) control functions and fixture (30) control functions. The (one or more) processors (3) include ML functions. In one example, the ML functions of the (one or more) processors (3) are at least partially implemented and / or otherwise performed in the system (1) using one or more artificial intelligence and / or deep learning computing and processing schemes.
[0024] In one example, the memory device(s) (5) include at least one non-transitory computer-readable medium (600). The non-transitory computer-readable medium (600) stores processor (3) executable instructions as software (604) for training a robot (2) using real (e.g., actual, "real world") and simulated grasping performance data, including but not limited to in a system (1). In one example, the processor (3) executable instructions stored as software (604) include one or more software modules (608). When executed by a processor (3) in communication with the memory (5), the robot (12), the gripper (30), and the sensor (22), the processor (3) executable instructions cause the processor (3) to at least partially implement and / or otherwise perform one or more of the disclosed steps of the method (200), including but not limited to using the system (1).
[0025] In the system (1), (one or more) processors (3), memory (5), (one or more) mechanical arms (12), (one or more) grippers (30), and (one or more) sensors (22) communicate with each other via a network (52), and communicate with each other using signals (e.g., encoded data signals) sent and / or received through the network (52). Communication among or between the (one or more) processors (3), memory (5), (one or more) mechanical arms (12), (one or more) grippers (30), and (one or more) sensors (22) is facilitated by transceivers (54, 55). In one example, the system (1) communication using the network (52) includes wireless communication devices and protocols. In another example, the system (1) communication using the network (52) includes wired communication devices and protocols. In yet another example, the system (1) communication using the network (52) includes a combination of wireless and wired communication devices and protocols. In one example, the system (1) communication includes wireless and / or wired communication devices and protocols for utilizing cloud-based processing, data storage, and / or communication resources. In one example, the system (1) communicates using the Internet, including but not limited to Internet of Things (IoT) protocols, practices and / or standards.
[0026] Figure 3 is positioned in multiple positions (67) according to an embodiment of the present disclosure Figure 1 Schematic diagram of an object (11) in a simulated or actual workspace (75) of a robot (2) of the system (1) shown. Figure 4 It is a schematic diagram of a process (300) for hybrid ML-based training of a robot (2) using real grasping performance data and simulated grasping performance data using a system (1) and a method (200) according to an embodiment of the present disclosure.
[0027] refer to Figure 3 , the object (11) has known or estimated physical properties (6), including a center of mass (COM) (70) (e.g., at Figure 3 denoted by "x" in the figure). The COM (70) may be known or may be assigned to the object (11) based on other known or estimated physical properties (6) such as mass, size, material, and distribution of mass over the volume of the object (11). As further described herein, a plurality of grasping locations (8) (e.g., eight locations, Figure 3The object may exist in a training ML environment (74) and / or a runtime ML environment (80) in a plurality of different poses (e.g., a first pose (67a), a second pose (67b), and a third pose (67c)). In this embodiment, the grasping location (8) assigned on the object (11) does not change based on the (one or more) specific poses (67) of the object (11) encountered by the system (1). As described herein, the disclosed system (1) and method (200) include determining a candidate grasping location (20) from a plurality of assigned grasping locations (8). In Figure 3 In the example shown, two of the eight assigned grasping locations (8) are determined as candidate grasping locations (20) (e.g., in Figure 3 Indicated by a circled asterisk in the figure).
[0028] refer to Figure 1-4 , a process (300) is used for hybrid ML-based training of (one or more) robots (2) for performing picking of a specific object (11). In one example, the object (11) can be a family of objects (11) having the same or similar physical properties (6). The process (300) can be implemented separately for (one or more) robots (2) for unique objects (11) having different physical properties (6). In this embodiment, the process (300) starts at box (301). In box (301), the process (300) trains the robot (2) in a simulator. From box (301), the process (300) proceeds to box (304). In box (304), the process (300) determines a candidate grasping location (20) from a plurality of assigned grasping locations (8). From box (304), the process (300) proceeds to box (307). In box (307), simulations and actual experiments for the robot (2) are performed using the determined candidate grasping locations (20) on the object (10).
[0029] From box (307), the process (300) proceeds to box (310). In box (310), the process (300) obtains data from the (one or more) grasping quality sensor (22) regarding the grasping performance of the robot (2) grasping the object (11). From box (310), the process (300) proceeds to box (313). In box (313), the process (300) evaluates the grasping quality of the robot (2) for the object (11) based on the feedback data from the (one or more) sensors (22). From box (313), the process (300) proceeds to logic decision (316). In logic decision (316), the process (300) determines the existence of convergence of the actual object (11) grasping quality (e.g., determined based on the actual robot action in the actual experiment of box (307)) and the simulated object (11) grasping quality (e.g., determined based on the simulated robot action in the simulated experiment of box (307)). As used in this article, "convergence" means that as the algorithm iterates, the computed grasp gets closer and closer to the true object grasp position. It is the opposite of "divergence", in which its output will go through increasingly large oscillations, never approaching a useful result.
[0030] Based on the result of the logical decision (316), if the process (300) determines that convergence is lacking, the process (300) proceeds to box (319). In box (319), the process (300) converts the grasp quality assessment from box (313) into a measure of grasp success probability. Upon completion of box (319), the process (300) returns to box (301) and the process (300) applies the grasp success probability to the simulator to retrain the robot (2) to grasp the object (11). In one embodiment, boxes (301), (304), (307), (310), (313) and (319) of the process (300) and the logical decision (316) are performed in the training ML environment (74).
[0031] For the result of the logical decision (316) indicating the existence of convergence, the process (300) proceeds to box (322) instead of proceeding to box (319). In box (322), the process (300) implements the runtime performance of the robot (2) action for picking up (one or more) objects (11) using at least one of the candidate grasping locations (20) determined in box (304). In one embodiment, box (322) also includes at least in part the above-mentioned functionality of box (310). From box (322), the process (300) proceeds to box (325). In box (325), the process (300) monitors the grasping quality of the robot (2) grasping the object (11). In the process (300), the monitoring in box (325) at least in part includes the functionality of box (328), or the process (300) proceeds to box (328). In box (328), the process (300) implements a hybrid ML engine (51) to facilitate continuous monitoring and evaluation of the quality of grasping of an object (11) in the training ML environment (74) and the runtime ML environment (80), and across multiple robots (71) where applicable. As further described herein, the monitoring and hybrid ML engines (51) implemented in boxes (325) and boxes (328), respectively, facilitate determining the need to retrain the robot (2) to grasp the object (11) based on the runtime object (11) grasping performance and its grasping quality. Thus, for example, if in boxes (325) and / or boxes (328), the process (300) determines that the robot (2) needs to be retrained to grasp the object (11), the process (300) proceeds from boxes (325) and / or boxes (328) to box (301). In one embodiment, blocks (322), (325), and (328) of process (300) are performed in a runtime ML environment (80).
[0032] Figure 5 is a block diagram of a data structure (502) of a memory (5) of a system (1) according to an embodiment of the present disclosure. Figure 6 is a block diagram of the software architecture of the method (200) according to an embodiment of the present disclosure. Figure 1-6The method (200) includes allocating (203) and storing (206) by a processor (3) in a memory (5) a plurality of grasping locations (8) on an object (11) based on known or estimated physical properties (6) of the object (11). In one example, the allocating (203) and / or storing (206) facilitates setting up (one or more) simulations and / or actual experiments for a robot (2) and an object (11) to be grasped. In one example, the plurality of grasping locations (8) on the object (11) are allocated (203) and / or stored (206) by or under the direction of at least one user (60) of the system (1). In another example, the plurality of grasping locations (8) on the object (11) are allocated (203) and / or stored (206) by or under the direction of at least one administrator (48) of the system (1). In yet another example, a plurality of grasping locations (8) on an object (11) are allocated (203) and / or stored (206) by or under the direction of the processor(s) (3). In embodiments where the allocation (203) and / or storage (206) steps are implemented and / or otherwise facilitated by software (604), the processor(s) (3) executes processor (3) executable instructions stored in a module (603) and / or storage (606). In one example, a plurality of grasping locations (8) on an object (11) are stored in and read from a data structure (502) and / or elsewhere in a memory (5) by or under the direction of the processor(s) (3).
[0033] The method (200) includes performing (209) by a processor (3) a first simulation (212) of a robot (2) grasping an object (11) using a first group (14) of a plurality of assigned (203) grasping locations (8). In one example, the first simulation (212) is performed (209) in a training ML environment (74). In an embodiment where the performing (209) step is implemented and / or otherwise facilitated by software (604), the processor (3) executes processor (3) executable instructions stored in an execution module (609). In one example, the first group (14) of the plurality of assigned (203) grasping locations (8) is stored in a data structure (502) and / or elsewhere in a memory (5) and read therefrom by or under the direction of the processor (3).
[0034] The method (200) includes evaluating, by a processor (or processors) (3), for each assigned grasping position (8) in a first set (14) of assigned grasping positions (203), a grasping quality of a simulated object (11) grasped (215) by a robot (2). In this embodiment, the grasping quality of the simulated object (11) is evaluated (218) based on a first set (15) of simulation data (17) from a first simulation experiment (212). The evaluating (218) step thereby facilitates determining the grasping (215) performance of the robot (2) in the first simulation experiment (212). In embodiments where the evaluating (218) step is implemented and / or otherwise facilitated by software (604), the processor (or processors) (3) execute processor (3) executable instructions stored in an evaluation module (618). In one example, a first set (15) of simulation data (17) from a first simulation experiment (212) is stored in and read from a data structure (502) and / or elsewhere in a memory (5) by or under the direction of the processor(s) (3).
[0035] The method (200) includes determining (221) by a processor (or processors) (3) a first set (16) of candidate grasping locations (20) on an object (11). In this embodiment, the first set (16) of candidate grasping locations (20) on the object (11) is determined (221) based on data (18) representing simulated grasping qualities obtained (224) in an evaluation (218) step. The evaluation (218) step thereby facilitates generating a list of candidate grasping locations for an actual experiment based on the evaluation (218) and judgment of grasping (215) performance of the robot (2) in the first simulation experiment (212). Thus, the first set (16) of candidate grasping locations (20) on the object (11) is selected from a plurality of assigned (203) grasping locations (8) on the object (11). In embodiments where the determining (221) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) executes processor(s) (3) executable instructions stored in the determining module (621). In one example, the first set (16) of candidate grasp locations (20) on the object (11) are stored in the data structure (502) and / or elsewhere in the memory (5) and read therefrom by or under the direction of the processor(s) (3).
[0036] The method (200) may include a first actual experiment (230) performed (227) by the processor(s) (3) and the robot (2) for the robot (2) to grasp (215) an object (11) using each candidate grasp position in the first set (16) of candidate grasp positions (20) determined (221). In one example, the first actual experiment (230) is performed (227) in a runtime ML environment (80). In embodiments where the execution (227) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) executes processor (3) executable instructions stored in an execution module (627). However, it should be understood that the steps of the disclosed method 2 described as being performed by the robot (2) in conjunction with the processor(s) (3) (e.g., the execution step (227)) may be performed without such close collaboration between the robot (2) and the processor(s) (3). For example, the processor(s) (3) executing the disclosed method (2) may receive data for processing from the sensor(s) (22) that are not necessarily coupled or mounted to the robot (2), but still provide (e.g., transmit) the sensor (8) data to the processor(s) (3) for use in the disclosed method (2). This scenario illustrates the flexibility of the disclosed systems, methods, and software, as they are beneficially applicable to a variety of robots, gripper devices, and sensors.
[0037] The method (200) includes evaluating (233) by a processor (or processors) (3) for each candidate grasping position in a first set (16) of candidate grasping positions (20) determined (221) an actual object (11) grasping quality performed by a robot (2) for grasping (215). In this embodiment, the actual object (11) grasping quality performed by a robot (2) for grasping (215) is evaluated (233) for each candidate grasping position in a first set (16) of candidate grasping positions (20) determined (221) based on a first set (19) of grasping quality sensor (22) data (26) from a first actual experiment (230). The evaluating (233) step thereby facilitates determining the grasping (215) performance of the robot (2) in the first actual experiment (230). In embodiments where the evaluation (233) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) executes processor (3) executable instructions stored in the evaluation module (633). In one example, a first set (19) of grip quality sensor (22) data (26) from a first actual experiment (230) is stored in and read from the data structure (502) and / or elsewhere in the memory (5) by or under the direction of the processor(s) (3).
[0038] The method (200) includes determining (236), by a processor (or processors) (3), convergence of grasping quality of an actual object (11) and grasping quality of a simulated object (11). In this embodiment, convergence is determined (236) based on: a first set (19) of grasping quality sensor (22) data (26) from a first actual experiment (230), and data (18) representing a simulated grasping quality. In one example, the determining (236) step includes determining (236) convergence of grasping quality of an actual object (11) and grasping quality of a simulated object (11) for each candidate grasping position in the first set (16) of candidate grasping positions (20) determined (221). In an embodiment where the determining (236) step is implemented and / or otherwise facilitated by software (604), the processor (or processors) (3) execute processor (3) executable instructions stored in a determination module (636). In one example, the determined (236) convergence state (66) is stored in and read from a data structure (502) and / or elsewhere in memory (5) by or under the direction of the processor(s) (3).
[0039] Figure 7 is a flow chart of various aspects of a method (200) according to an embodiment of the present disclosure. Figure 1-7 In one embodiment, the method (200) includes determining (239), by the processor(s) (3), a grasp success probability value (29). In this embodiment, for each candidate grasp position in the determined (221) first set (16) of candidate grasp positions (20), the grasp success probability value (29) is determined (239) based on a first set (19) of grasp quality sensor (22) data (26) acquired (241) in an evaluation (233) step. In one embodiment, the grasp success probability value (29) is determined (239) in response to determining (236) that convergence (e.g., a negative convergence state (66)) of grasp qualities of the actual and simulated objects (11) for at least one candidate grasp position in the determined (221) first set (16) of candidate grasp positions (20). The determining (239) step thereby facilitates efficient use of computational, memory, bandwidth, and / or power resources in the event that at least one, but not all, of the determined (221) first set (16) of candidate grasp locations (20) lack convergence (e.g., lack a positive convergence state (66)). Thus, in this embodiment, grasp success probability values (29) are determined (239) only for candidate grasp locations (20) that lack convergence. In embodiments where the determining (239) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) execute processor (3) executable instructions stored in the determining module (639).
[0040] In one embodiment, the method (200) includes allocating (242) and storing (245) in memory (5) by the processor (s) a grasp success probability value (32) determined (239) for each candidate grasp position in the determined (221) first set (16) of candidate grasp positions (20). In embodiments where the allocating (242) and / or storing (245) steps are implemented and / or otherwise facilitated by software (604), the processor (s) (3) executes processor (3) executable instructions stored in the allocating (642) and / or storing (645) modules. In one example, the determined (239) grasp success probability values (29) and / or grasp success probability values (32) are stored and read from the data structure (502) and / or elsewhere in memory (5) by or under the direction of the processor (s) (3).
[0041] In one embodiment, the determining (239) step of the method (200) includes transforming (248) by the processor(s) (3) each grip quality sensor (22) data in the first set (19) grip quality sensor (22) data (26) into discrete values (35). In embodiments where the transforming (248) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) executes processor (3) executable instructions stored in a transform module (648). In one example, the discrete value(s) (35) produced by the transforming (248) step is stored in and read from a data structure (502) and / or elsewhere in a memory (5) by or under the direction of the processor(s) (3). In this embodiment, the determining (239) step includes scoring (251) each candidate grasping position in the determined (221) first set (16) of candidate grasping positions (20) based on the corresponding transformed (248) discrete value (35) by the (one or more) processors (3). In this embodiment, the score value (38) of each candidate grasping position in the determined (221) first set (16) of candidate grasping positions (20) is proportional to the grasping success probability value (29) determined (239). In an embodiment where the scoring (251) step is implemented and / or otherwise facilitated by software (604), the (one or more) processors (3) execute processor (3) executable instructions stored in the scoring module (651). In one example, the (one or more) score values (38) generated by the scoring (251) step are stored in the data structure (502) and / or elsewhere in the memory (5) and read therefrom by or under the direction of the (one or more) processors (3).
[0042] In one embodiment, the transforming (248) step of the method (200) includes determining (254) a discrete value (35) based on a plurality of grip quality sensor (22) readings (41) for each candidate grip position in the determined (221) first set (16) of candidate grip positions (20). In embodiments where the determining (254) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) executes processor (3) executable instructions stored in a determination module (654). In one example, the plurality of grip quality sensor (22) readings (41) used to determine (254) the discrete value are stored in and read from a data structure (502) and / or elsewhere in a memory (5) by or under the direction of the processor(s) (3).
[0043] In one embodiment, the method (200) includes executing (209), evaluating (218), determining (221), executing (227), evaluating (233), and determining (236) steps for at least one candidate grasping position (20) iteration (257) by the processor(s) (3) and the robot(s) (2). In this embodiment, the iteration (257) step is performed for at least one iteration (266) in the method (200). In one example, the iteration (257) step is performed in response to determining (236) the absence of convergence (e.g., a negative convergence state (66)) of actual and simulated object (11) grasp qualities for at least one determined (221) first set (16) of candidate grasping positions (20). The iterating (257) step thereby facilitates efficient use of computational, memory, bandwidth and / or power resources in the event that at least one of the first set (16) of candidate grasping locations (20) determined (221) lacks convergence (e.g., lacks a positive convergence state (66)). Thus, in this embodiment, the executing (209), evaluating (218), determining (221), executing (227), evaluating (233) and determining (236) steps are iterated (257) only for the candidate grasping locations (20) that lack convergence. Thus, the iterating (257) step directs the method (200) back to the simulated experiment (e.g., at least the second simulated experiment (265)) and / or actual experiment (e.g., at least the second actual experiment (267)) for retraining the robot (2) to grasp (215) for at least one candidate grasping location (20), which ensures that the ML model (68) maintains accurate data for the corresponding candidate grasping location (20) for the object (11). In embodiments where the iteration (257) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) execute processor (3) executable instructions stored in the iteration module (657).
[0044] In one embodiment, the iterating (257) step of the method (200) includes, for at least one iteration (266), determining (260) a grasping success probability distribution (44) by (one or more) processors (3) for the determined (221) first set (16) of candidate grasping positions (20). In this embodiment, the grasping success probability distribution (44) is determined (260) based on the determined (239) grasping success probability values (29). In this embodiment, the iterating (257) step includes: for at least one iteration (266), applying (263) by (one or more) processors (3) the determined (260) grasping success probability distribution (44) to at least one second simulation experiment (265) for (one or more) robots (2) to grasp (215) an object (11) using the determined (221) first set (16) of candidate grasping positions (20). Thus, the determining (260) and applying (263) steps facilitate retraining the robot (2) to grasp (215) an object (11) for at least one candidate grasping location (20) by incorporating the grasping success probability distribution (44) into the ML model (68), which improves its accuracy and robustness. In embodiments where the determining (260) and / or applying (263) steps are implemented and / or otherwise facilitated by software (604), the processor (3) executes processor (3) executable instructions stored in the determining (660) and / or applying (663) modules (604). In one example, the determined (260) and applied (263) grasping success probability distribution (44) is stored in and read from a data structure (502) and / or elsewhere in a memory (5) by or under the direction of the processor (3).
[0045] In one embodiment, the iterating (257) step of the method (200) includes, for at least one iteration (266), determining (269) by the processor(s) (3) at least a second set (47) of the plurality of assigned (203) grasping locations (8). In this embodiment, the at least second set (47) of the plurality of assigned (203) grasping locations (8) is determined (269) based on a grasping success probability distribution (44) applied (263) to at least a second simulation (265). In one example, at least the second set (47) differs from the first set (14) by the addition or removal of at least the assigned (203) grasping locations (8) on the object (11). In this embodiment, the iterating (257) step includes, for at least one iteration (266), performing (272) by the processor(s) (3) at least a second simulation (265) using the at least second set (47) of grasping locations (8) assigned (203). Thus, the determining (269) and executing (272) steps facilitate performing subsequent simulations and actual experiments to retrain the robot (2) to grasp (215) the object (11) by using at least a second set (47) of the plurality of assigned (203) grasp locations (8) determined (269) based on the applied (263) grasp success probability distribution (44), which refines the assigned (203) grasp locations (8) and further improves the accuracy and robustness of the ML model (68). In embodiments where the determining (269) and / or executing (272) steps are implemented and / or otherwise facilitated by software (604), the processor (3) executes processor (3) executable instructions stored in the determining (669) and / or executing (672) modules. In one example, at least a second group (47) of the determined (269) plurality of assigned (203) grab locations (8) are stored in and read from a data structure (502) and / or elsewhere in a memory (5) by or under the direction of the processor(s) (3).
[0046] In one embodiment, the iterating (257) step of the method (200) includes determining (275) by the processor (or processors) (3) for each grasping position in at least a second set (47) of assigned (203) grasping positions (8) for at least one iteration (266). In this embodiment, the MLLS (50) is determined (275) based on the grasping success probability distribution (44) determined (260). In this embodiment, the iterating (257) step includes determining (278) by the processor (or processors) (3) for at least one iteration (266) at least a second set (53) of candidate grasping positions (20). In this embodiment, the at least second set (53) of candidate grasping positions (20) is determined (278) based on the MLLS values (56) determined (275) for each grasping position in at least a second set (47) of assigned (203) grasping positions (8). In one example, at least the second set (53) differs from the first set (16) by the addition or removal of at least one candidate grasping location (20). Thus, the determining (275 and 278) steps facilitate retraining the robot (2) to grasp (215) the object (11) by performing subsequent simulations and actual experiments using at least the second set (53) of candidate grasping locations (20) determined (278) based on the MLLS values (56) determined (275), respectively, which refines the candidate grasping locations (20) and further increases the accuracy and robustness of the ML model (68). In embodiments where the determining (275 and / or 278) steps are implemented and / or otherwise facilitated by software (604), the processor (3) executes processor (3) executable instructions stored in the determining module (675 and / or 678). In one example, the determined (275) MLLS (50) and / or the determined (278) MLLS value (56) is stored in and read from a data structure (502) and / or elsewhere in a memory (5) by or under the direction of the processor(s) (3).
[0047] In one embodiment, the iterating (257) step of the method (200) includes, for at least one iteration (266), evaluating (281) by the processor(s) (3) a simulated grasping quality of a grasping (215) by the robot (2) for each grasping position in at least a second set (47) of assigned (203) grasping positions (8) on the object (11). In this embodiment, the simulated grasping quality of a grasping (215) by the robot (2) for each grasping position in at least a second set (47) of assigned (203) grasping positions (8) is evaluated (281) based on at least a second set (59) of simulation data (24) from at least a second simulation experiment (265). In this embodiment, the determining (275) step of the method (200) further comprises determining (284) the MLLS (50) based on data (27) representing a simulated grasp quality acquired (277) by the (one or more) processors (3) in the evaluating (281) step from at least a second set (59) of simulated data (24) from at least a second simulation experiment (265). In embodiments where the evaluating (281) and / or determining (284) steps are implemented and / or otherwise facilitated by software (604), the (one or more) processors (3) execute processor (3) executable instructions stored in the evaluating (681) and / or determining (684) modules. In one example, at least a second set (59) of simulated data (24) from at least a second simulation experiment (265) and / or data (27) representing a simulated grasping quality acquired (277) by the (one or more) processors (3) in an evaluation (281) step are stored in a data structure (502) and / or elsewhere in a memory (5) and read therefrom by or under the direction of the (one or more) processors (3).
[0048] In one embodiment, the method (200) includes allocating (285) and storing (287) by the processor (s) (3) in a memory (5) hyperparameters (65) of an ML model (68). As used herein, "hyperparameter" refers to one or more settings that can be adjusted to control the behavior of a machine learning algorithm. The hyperparameters (65) represent a simulated grasping quality of at least a second simulation experiment (265) for grasping (215) an object (11) by the robot (s) (2). In an embodiment where the system (1) includes a plurality of object picking robots (71) (e.g., a first robot (2) and at least a second robot (2)), the method (200) further includes sharing (290) the hyperparameters (65) allocated (285) to the first robot (2) with at least the second robot (2). The allocating (285) and sharing (290) steps thereby facilitate efficient use of computation, memory, bandwidth, and / or power resources when using a plurality of robots (71) in a runtime ML environment (80) and / or a training ML environment (74). Thus, in this embodiment, only one robot (2) (e.g., the first robot (2)) of the plurality of robots (71) needs to be trained and / or retrained, and the ML model (68) is developed for this purpose according to the disclosed method (200). Thus, additional robot(s) (2) (e.g., at least a second robot (2) having the same or similar design specifications as the first robot (2) and / or meeting the same or similar functional requirements) may not need to be trained and / or retrained to grasp (215) the object (11). In embodiments where the (one or more) allocation (285), storage (287), and / or sharing (290) steps are implemented and / or otherwise facilitated by software (604), the (one or more) processors (3) execute processor (3) executable instructions stored in the (one or more) allocation (685), storage (687), and / or sharing (690) modules. In one example, hyperparameters (65) assigned (285) to the ML model (68) are stored (287) in and read from a data structure (502) and / or elsewhere in memory (5) by or under the direction of the processor(s) (3).
[0049] In one embodiment, the executing (209), evaluating (218), determining (221), executing (227), evaluating (233), and determining (236) steps of the method (200) are performed in an ML training environment (74). In this embodiment, the method (200) includes monitoring (293), by the processor(s) (3), a runtime grasp quality of the (one or more) robots (2) grasping (215) an object (11) using at least one candidate grasp position (20). In this embodiment, the runtime grasp quality of the (one or more) robots (2) grasping (215) an object (11) using at least one candidate grasp position (20) is monitored (293) based on a runtime set (77) of grasp quality sensor (22) data (26) acquired (295) from the (one or more) robots (2) grasping (215) the object (11) in an ML runtime (e.g., actual, "real world") environment (80). In this embodiment, in response to determining (236) the presence of convergence (e.g., a positive convergence state (66)) of actual and simulated object (11) grasp qualities for at least one of the determined (221) first set (16) of candidate grasp locations (20), a monitoring (293) step is performed. The monitoring (293) step thereby facilitates efficient use of computational, memory, bandwidth, and / or power resources in the event that at least one of the determined (221) first set (16) of candidate grasp locations (20) has convergence (e.g., no negative convergence state (66)). Thus, in this embodiment, only candidate grasp locations (20) with convergence proceed to the runtime ML environment (80) and are monitored (293) therein, which helps ensure that the ML model (68) is maintained with accurate, up-to-date data for both the runtime ML environment (80) and the training ML environment (74). In embodiments where the monitoring (293) step is implemented and / or otherwise facilitated by software (604), the processor(s) (3) execute processor (3) executable instructions stored in the monitoring (693) module. In one example, the runtime set (77) of grip quality sensor (22) data (26) is stored (287) in the data structure (502) and / or elsewhere in the memory (5) and read therefrom by or under the direction of the processor(s) (3).
[0050] In one embodiment, the method (200) includes executing (209), evaluating (218), determining (221), executing (227), evaluating (233), and determining (236) steps by the processor(s) (3) and the robot(s) (2) for at least one candidate grasping position (20) iteration (296). In this embodiment, the iteration (296) step is performed for at least one iteration (273) in the method (200). In one example, the iteration (296) step is performed in response to the monitored (293) runtime grasping quality falling below or otherwise not meeting a user (60) predetermined quality threshold (83). In one example, the user (60) predetermined quality threshold (83) is an event of the object (11) falling out of the gripper (30) (e.g., as sensed by the camera sensor(s) (22)). In another example, the user (60) predetermined quality threshold (83) is an event in which the object (11) exhibits sway and / or vibration when grasped (215) by the fixture (30) and / or carried in the fixture (30) (e.g., as sensed by multi-degree-of-freedom (DOF) force torque, multi-DOF inertial measurement, and / or (one or more) tactile sensors (22)), wherein such sway and / or vibration undesirably exceeds levels typically observed during operation of the robot (2). In yet another example, the user (60) predetermined quality threshold (83) is an event in which the object (11) exhibits relative motion and / or physical dynamics relative to the fixture (30) and / or compared to the fixture (30) that is out of tolerance compared to an expected or user (60) specified relative motion amount (e.g., as sensed by (one or more) sensors (22)).
[0051] In this embodiment, the iterating (296) step thereby facilitates efficient use of computational, memory, bandwidth, and / or power resources in situations where, based on the monitored (293) runtime grasp quality, optimization of the ML model (68) is required to ensure consistent object (11) grasping (215) performance during runtime. The monitoring (293) and iterating (296) steps, together with one or more steps of the method (200) described above, constitute a hybrid ML engine (51) that facilitates continuous monitoring and evaluation of object (11) grasp quality across a training ML environment (74) and a runtime ML environment (80), and where applicable, across multiple robots (71). Furthermore, in a use case, the monitored (293) runtime grasp quality of at least one robot (2) may not meet a user (60) predetermined quality threshold (83) due to a mechanical issue with that robot (2). In this use case, mechanical problems can be identified and corrected so that normal unit operation can be restored after retraining the corresponding robot (2) as needed, including via the iteration (296) step of the method (200). In embodiments where the iteration (296) step is implemented and / or otherwise facilitated by software (604), the (one or more) processors (3) execute processor (3) executable instructions stored in the iteration (696) module. In one example, the quality threshold (83) predetermined by the user (60) is stored (287) in the data structure (502) and / or elsewhere in the memory (5) and read therefrom by or under the direction of the (one or more) processors (3).
[0052] Compared to known systems and methods, the disclosed system and method for hybrid ML-based training of an object picking robot using real grasping performance data and simulated grasping performance data provides users with many beneficial technical effects and achieves various advantages. Such benefits include, but are not limited to, the ability to pick up and place (one or more) objects with improved accuracy, speed, efficiency, and reduced error rates compared to known processes. Compared to known systems and methods, the disclosed system and method for hybrid ML-based training of an object picking robot reduces the required number of CPU clock cycles per object required by (one or more) processors in both training and run-time environments. The hybrid ML-based training of the above-mentioned object picking robot enables continuous evaluation and monitoring of object grasping performance, so that hybrid ML operations can be implemented as needed to fine-tune and enhance the accuracy and robustness of the ML model, including across multiple robots involved in a unit operation. Therefore, the disclosed system and method for hybrid ML-based training of an object picking robot can efficiently and effectively train object picking robots in various industrial applications where it is desired to improve the utilization of computing, memory, network bandwidth, power, and / or human resources.
[0053] These and other substantial and numerous technical benefits and advantageous effects that may be appreciated by those of ordinary skill in the art are particularly evident compared to known systems and methods in applications involving high-volume and fast-paced industrial operations. These improvements over known systems and methods are not achieved solely by utilizing conventional and conventional processing systems and methods. Even where such improvements can be quantified in terms of time reduction per object (e.g., measured in seconds or fractions thereof), over relevant time periods (e.g., from hours to years), and compared to known processes, the disclosed systems and methods for hybrid machine learning-based training of object picking robots using real grasping performance data and simulated grasping performance data utilize computing, network, memory, power, personnel, etc. resources with significantly higher efficiency to provide throughput improvements and overall cost reductions for various industrial unit operations involving robotic picking and placement of objects.
[0054] The various embodiments disclosed herein should be considered illustrative and explanatory and should not be construed as limiting the present disclosure in any way.
[0055] Although various aspects of the present disclosure have been specifically shown and described with reference to the above embodiments, it will be appreciated by those skilled in the art that various additional embodiments may be conceived by modifying the disclosed devices, systems, and methods without departing from the spirit and scope of the disclosed content. Such embodiments should be understood to fall within the scope of the present disclosure as determined based on the claims and any equivalents thereof.
Claims
1. A method for training an object picking robot using real grasping performance data and simulated grasping performance data, comprising: (a) allocating, by a processor, a plurality of grasping locations on the object based on known or estimated physical properties of the object; (b) performing, by the processor, a first simulation experiment for the robot to grasp the object using a first set of assigned grasping positions from among a plurality of assigned grasping positions; (c) evaluating, by the processor and based on a first set of simulation data from the first simulation experiment, a grasp quality of a simulated object grasped by the robot for each assigned grasping position in the first set of assigned grasping positions; (d) determining, by the processor, a first set of candidate grasp locations on the object based on data representing the simulated grasp quality obtained in (c); (e) evaluating, by the processor and based on a first set of grasp quality sensor data, for each candidate grasp position in the first set of candidate grasp positions, an actual grasp quality of the object grasped by the robot, the first set of grasp quality sensor data being from a first actual experiment for the robot grasping the object using each candidate grasp position in the first set of candidate grasp positions; as well as (f) for each candidate grasp position in the first set of candidate grasp positions, determining, by the processor, convergence of the actual object grasp quality and the simulated object grasp quality based on the first set of grasp quality sensor data from the first actual experiment and data representing the simulated grasp quality.
2. The method according to claim 1, further comprising: (g) in response to determining in (f) that the convergence of the actual object grasp quality and the simulated object grasp quality is absent for at least one candidate grasp position in the first set of candidate grasp positions, determining, by the processor, a grasp success probability value for each candidate grasp position in the first set of candidate grasp positions based on the first set of grasp quality sensor data acquired in (e).
3. The method according to claim 2, further comprising: (h) Assigning, by the processor, to each candidate grasping position in the first set of candidate grasping positions the grasping success probability value determined in (g).
4. The method according to claim 2, wherein determining the grasping success probability value comprises: (I) transforming, by the processor, each grip quality sensor data in the first set of grip quality sensor data into a discrete value; as well as (II) scoring, by the processor, each candidate grasping position in the first set of candidate grasping positions based on the corresponding discrete value from (I), wherein the score value of each candidate grasping position in the first set of candidate grasping positions is proportional to the grasping success probability value determined in (g). 5 . The method of claim 4 , wherein the transforming comprises determining the discrete value based on a plurality of grip quality sensor readings for each candidate grip position in the first set of candidate grip positions.
6. The method of claim 1, wherein (b)-(f) are performed in a machine learning training environment, and wherein the method further comprises: (g) in response to determining the existence of the convergence of the actual object grasp quality and the simulated object grasp quality for at least one candidate grasp position in the first set of candidate grasp positions in (f), monitoring, by the processor, a runtime grasp quality of grasping by the robot using the at least one candidate grasp position based on a runtime set of grasp quality sensor data acquired from grasping by the robot in a machine learning runtime computing environment.
7. The method according to claim 6, further comprising: (h) in response to the runtime grasp quality monitored in (g) falling below or otherwise failing to meet a user-predetermined quality threshold, iteratively performing (b)-(f) for the at least one candidate grasp position by the processor for at least one iteration.
8. A system for training an object picking robot using real grasping performance data and simulated grasping performance data, comprising: one or more memory devices; as well as one or more processors in communication with the one or more memory devices and the object picking robot, wherein the one or more processors are programmed to: (a) assigning a plurality of grasping locations on the object based on known or estimated physical properties of the object; (b) performing a first simulation experiment for the robot to grasp the object using a first set of assigned grasping positions from among a plurality of assigned grasping positions; (c) evaluating a grasp quality of a simulated object grasped by the robot for each assigned grasping position in the first set of assigned grasping positions based on a first set of simulation data from the first simulation experiment; (d) determining a first set of candidate grasp locations on the object based on data representing the simulated grasp quality obtained in (c); and (e) evaluating, based on a first set of sensor data, for each candidate grasping position in the first set of candidate grasping positions, a grasping quality of an actual object grasped by the robot, wherein the first set of sensor data is from a first actual experiment, wherein the first actual experiment is for the robot to grasp the object using each candidate grasping position in the first set of candidate grasping positions; (f) for each candidate grasp position in the first set of candidate grasp positions, determining convergence of the actual object grasp quality and the simulated object grasp quality based on the first set of grasp quality sensor data from the first actual experiment and data representing the simulated grasp quality.
9. The system of claim 8, wherein the one or more processors are further programmed to: (g) in response to determining in (f) that there is no convergence of the actual object grasp quality and the simulated object grasp quality for at least one candidate grasp position in the first set of candidate grasp positions, determining a grasp success probability value for each candidate grasp position in the first set of candidate grasp positions based on the first set of sensor data acquired in (e).
10. The system of claim 9, wherein the one or more processors are further programmed to: (h) further in response to determining in (f) that the convergence of the actual object grasp quality and the simulated object grasp quality is absent for the at least one candidate grasp position in the first set of candidate grasp positions, iteratively performing (b)-(f) for the at least one candidate grasp position for at least one iteration.
11. The system of claim 10, wherein to iteratively perform (b)-(f) for the at least one candidate grasping position for the at least one iteration, the one or more processors are further programmed to: (I) determining a grasping success probability distribution for the first set of candidate grasping positions based on the grasping success probability values determined in (g); and (II) applying the grasping success probability distribution determined in (I) to at least a second simulation experiment, wherein the at least second simulation experiment is for the robot to grasp the object using the first set of candidate grasping positions.
12. The system of claim 11, wherein to iteratively perform (b)-(f) for the at least one candidate grasping position for the at least one iteration, the one or more processors are further programmed to: (III) determining at least a second set of assigned grasping locations among the plurality of assigned grasping locations based on the grasping success probability distribution applied to the at least second simulation experiment in (II); and (IV) performing the at least second simulation experiment using the at least second set of assigned grasping positions.
13. The system of claim 12, wherein to iteratively perform (b)-(f) for the at least one candidate grasping position for the at least one iteration, the one or more processors are further programmed to: (V) determining a maximum success log-likelihood for each assigned grasping location in the at least a second set of assigned grasping locations based on the grasping success probability distribution determined in (I); and (VI) determining at least a second set of candidate grasping positions based on the maximum success log-likelihood value determined in (V) for each assigned grasping position in the at least second set of assigned grasping positions, respectively.
14. The system of claim 13, wherein to iteratively perform (b)-(f) for the at least one candidate grasping position for the at least one iteration, the one or more processors are further programmed to: (VII) evaluating, for each assigned grasping location in the at least a second set of assigned grasping locations on the object, a simulated grasping quality of the grasping by the robot based on at least a second set of simulation data from the at least a second simulation experiment.
15. The system of claim 14, wherein to determine the maximum log-likelihood of success, the one or more processors are further programmed to: The maximum log-likelihood of success is further determined based on data representing the simulated grasp quality obtained in (VII).
16. The system of claim 10, wherein the one or more processors are further programmed to: (i) assigning hyperparameters of a machine learning model representing simulated grasping quality for at least a second simulated experiment of the robot grasping an object.
17. The system of claim 16, wherein to train a plurality of object picking robots including a first robot and at least a second robot, the one or more processors are further programmed to: (j) sharing the hyperparameters assigned to the first robot in (i) with at least a second robot.
18. A non-transitory computer-readable storage medium storing processor-executable instructions for training an object picking robot using real grasping performance data and simulated grasping performance data, wherein the processor-executable instructions, when executed by one or more processors, cause the one or more processors to: (a) assigning a plurality of grasping locations on the object based on known or estimated physical properties of the object; (b) performing a first simulation experiment for the robot to grasp the object using a first set of assigned grasping positions from among a plurality of assigned grasping positions; (c) evaluating a grasp quality of a simulated object grasped by the robot for each assigned grasping position in the first set of assigned grasping positions based on a first set of simulation data from the first simulation experiment; (d) determining a first set of candidate grasp locations on the object based on data representing the simulated grasp quality obtained in (c); and (e) evaluating, for each candidate grasping position in the first set of candidate grasping positions, an actual grasping quality of the object grasped by the robot based on a first set of grasping quality sensor data, the first set of grasping quality sensor data being from a first actual experiment, the first actual experiment being for the robot to grasp the object using each candidate grasping position in the first set of candidate grasping positions; (f) for each candidate grasp position in the first set of candidate grasp positions, determining convergence of the actual object grasp quality and the simulated object grasp quality based on the first set of grasp quality sensor data from the first actual experiment and data representing the simulated grasp quality.
19. The non-transitory computer readable storage medium of claim 18, wherein the processor executable instructions, when executed by the one or more processors, further cause the one or more processors to: (g) in response to determining in (f) an absence of the convergence of the actual object grasp quality and the simulated object grasp quality for at least one candidate grasp position in the first set of candidate grasp positions, determining a grasp success probability value for each candidate grasp position in the first set of candidate grasp positions based on the first set of grasp quality sensor data acquired in (e); and (h) assigning the grasping success probability value determined in (g) to each candidate grasping position in the first group of candidate grasping positions.
