Controlling multiple robots for collaborative item pick-and-place

Abstract

A robotic system for controlling multiple robots to collaboratively pick and place objects is disclosed. [Solution] In various embodiments, the robot system comprises a first robot arm having a first end effector, a second robot arm having a second end effector, and a control computer configured to pick and place multiple objects using the first and second robot arms, which includes working collaboratively to pick and place one or more of a plurality of objects using the first and second robot arms.

Description

Cross-reference to other applications

[0001] This application claims priority based on U.S. Provisional Patent Application No. 63 / 274,465, filed on November 01, 2021, "CONTROLLING MULTIPLE ROBOTS TO COOPERATIVELY PICK AND PLACE ITEMS", which provisional patent application is incorporated herein by reference for all purposes. BACKGROUND OF THE INVENTION

[0002] Robots have been provided to perform various tasks such as manipulating objects. For example, a robotic arm having an end effector can be utilized to pick and place items. Examples of commercial uses of such robots include sorting, kitting, palletization, depalletization, loading and unloading trucks or containers, and the like.

[0003] In some contexts, the objects to be manipulated vary considerably in size, weight, packaging, and other attributes. Typically, robotic arms are rated to accommodate up to the maximum size, weight, etc. of the objects. In some contexts, conventional approaches may require a robotic arm that can manipulate the largest, heaviest, and / or otherwise most difficult objects that an operation may be required for. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] In the following detailed description and the accompanying drawings, various embodiments of the present invention are disclosed.

[0005] [Figure 1] A block diagram showing one embodiment of a robot system configured to control multiple robots to cooperatively perform a task.

[0006] [Figure 2A]A figure illustrating an example of a collaborative pick-and-place task performed in one embodiment of the robotic system described herein. [Figure 2B] A figure illustrating an example of a collaborative pick-and-place task performed in one embodiment of the robotic system described herein. [Figure 2C] A figure illustrating an example of a collaborative pick-and-place task performed in one embodiment of the robotic system described herein.

[0007] [Figure 3] A block diagram showing one embodiment of a robot control system.

[0008] [Figure 4] A state diagram illustrating one embodiment of a robotic system configured to control multiple robots to collaboratively perform a task.

[0009] [Figure 5A] A flowchart illustrating one embodiment of the process for collaboratively performing a task as a "leader" robot in one embodiment of a robotic system described herein.

[0010] [Figure 5B] A flowchart illustrating one embodiment of a process for collaboratively performing a task as a "follower" robot in one embodiment of a robotic system described herein.

[0011] [Figure 6A] A diagram illustrating one embodiment of a robotic system configured to collaboratively pick and place objects using two or more robots.

[0012] [Figure 6B] A diagram illustrating one embodiment of a robotic system configured to collaboratively pick and place objects using two or more robots.

[0013] [Figure 7] A flowchart illustrating one embodiment of a process for collaboratively picking and placing objects using two or more robots. [Modes for carrying out the invention]

[0014] The present invention can be implemented in various forms, including processes, apparatus, systems, compositions of materials, computer program products embodied on computer-readable storage media, and / or processors (processors configured to execute instructions stored and / or provided by memory connected to the processor). In this specification, these embodiments or any other forms the present invention may take may be referred to as "technologies." Generally, the order of the processes of the disclosed processes may be modified within the scope of the invention. Unless otherwise specified, components such as processors or memory described as configured to perform a task may be implemented as general components temporarily configured to perform a task at a given time, or as specific components manufactured to perform a task. In this specification, the term "processor" refers to one or more devices, circuits, and / or processing cores configured to process data such as computer program instructions.

[0015] The following provides a detailed description of one or more embodiments of the present invention, with reference to drawings illustrating the principles of the present invention. While the present invention is described in relation to such embodiments, it is not limited to any of these embodiments. The scope of the present invention is limited only by the claims, and the present invention includes many substitutes, variations, and equivalents. The following description includes many specific details to provide a complete understanding of the present invention. These details are illustrative, and the present invention can be implemented in accordance with the claims without some or all of these specific details. For simplicity, technical matters well known in the art related to the present invention are not described in detail, so as not to complicate the present invention unnecessarily.

[0016] A system for coordinating and controlling the use of multiple robots to collaboratively pick and place packages is disclosed. In various embodiments, the system disclosed herein may have one or more of the following technical features: The system collaboratively detects objects to be picked. • An integrated computer vision system for identifying obstacles and prioritizing the picking of packages from a stack, enabling safe collaborative operation. • Planning and execution of collaborative picking of a single object by multiple robots. A control architecture that enables robots to collaborate to lift and place objects in a safe and controlled manner. • Robots can act independently (alone) when they are not performing collaborative tasks.

[0017] In various embodiments, multiple robotic arms are used to collaboratively pick and place a single package. In some embodiments, a robotic singularity (or other pick / place) system detects that an object should be picked collaboratively by two or more robots based on, for example, the size of the object, its weight, previous failed picking attempts, visual classification, and / or a mismatch in affordances between the object and individual robotic grippers.

[0018] In various embodiments, when both robots stop picking independently, the system determines the best way to pick an object and ensures that the grasping point is within the reach of the robot and on the opposite side of the object. The robot removes any surrounding packages that could prevent the robot from picking the desired package. The robot plans a path to independently reach pick positions on both sides of the object. When both robots are positioned at a predetermined location, the leader robot starts to retreat, and the follower robot uses force control to maintain contact with the box while maintaining its relative position / orientation with respect to the leader robot, thereby enabling the heavy or oversized object to be lifted and moved collaboratively.

[0019] Further techniques implemented in various embodiments include, but are not limited to, one or more of the following. · Tactile manipulation to reinforce the unseen part of an object. Sometimes, one of the robots has to pick on the side of the object that is not visible to the robot to ensure that multiple robots can find a collision-free pick for the same object. In some embodiments, tactile perception from the gripper is used to explore without looking at the back side of the object for the purpose of finding a stable and collision-free pick position. · Push to improve the visibility of an object and enhance the stability of grasping. Sometimes, not all sides of an object are visible to the robots to achieve a collaborative pick, and the robots need to change the position / orientation of the object to make the pickable positions of multiple robots visible. This change may be made by one or more robots pushing to identify a more stable grasping point. · Collision avoidance between multiple robots. It is preferable to coordinate the movements of multiple robots with the corresponding environment in a non-colliding way.

[0020] FIG. 1 is a block diagram showing an embodiment of a robot system configured to control a plurality of robots to collaboratively execute tasks. In the example of the figure, the system and environment 100 includes a first robot arm 102 with a suction type end effector 104 and a second robot arm 106 with a suction type end effector 108. In the state shown in the figure, the robot arm 102 and the robot arm 106 are arranged to collaboratively execute a pick-and-place task on a large box 110. A control computer 112 is configured to wirelessly communicate with one or more of the robot arm 102, the robot arm 106, and one or more of one or more cameras or other sensors 114 within the work space. The image data received from the camera 114 may be used by the control computer 112 to generate a three-dimensional view of the work space as needed, for example, for the purpose of facilitating a collaborative pick-and-place task, and to send commands and information to the robot arm 102 and the robot arm 106.

[0021] FIGS. 2A-2C show an example of a collaborative pick-and-place task executed in an embodiment of the robot system disclosed herein. In the example shown in FIG. 2A, a robot arm 202 with a suction type end effector 204 and a robot arm 206 with a suction type end effector 208 are arranged to begin collaboratively executing a pick-and-place task on a large box 210, similar to the starting state shown in FIG. 1. In various embodiments, in the collaborative pick-and-place disclosed herein, the robot arm 202 may be the "leader" and the robot arm 206 may be the "follower". The "leader" may be selected by any suitable method (e.g., by assigning the role of "leader" to the robot that initiated the collaborative task, by randomly assigning the role to one or the other of the participating robots, by "election" or other selection method).

[0022] To initiate the operation, in various embodiments, the robot arm 202, acting as the “leader,” moves its end effector 204 to the position shown in the figure, and then grasps the box by, for example, moving the end effector 204 to a position where it is in contact with or nearly in contact with the side of the box 210 to perform suction. A signal may be sent to the other robot (and / or the process for controlling the other robot) to indicate that the leader has completed its grasp. The follower (e.g., robot arm 206 in this example) then grasps the box 210, for example, on the side opposite to the side where the leader (i.e., robot arm 202) grasped the box 210. The follower records the transformation based on the position and orientation of the leader’s end effector 204 and the relevant dimensions of the box 210. For example, vision systems and / or other sensors may be used to measure dimensions or to recognize box 210 (e.g., specifically and / or by type) and to determine dimensions using item and / or type information (e.g., by lookup).

[0023] As shown in Figure 2B, by this point both robots (202, 206) have grasped the box 210. The leader (in this example, robot arm 202) independently of the follower robot arm 206 performs calculations to move the box along a trajectory determined by robot arm 202 (and / or its associated control process). In various embodiments, the follower robot (in this example, robot arm 206) receives position and orientation information of the leader robot arm 202's end effector 204 (e.g., periodically, continuously, etc.). The follower robot arm 206 (and / or its associated control process) uses the position and orientation information of the leader robots (202, 204) and previously determined and recorded transformations to calculate a new target position and orientation for the follower's end effector 208, and, if necessary, calculates and applies torque to the motors constituting robot arm 206 to minimize the error (difference) between the current position and orientation of the follower's end effector 208 and the (most recently updated) target.

[0024] For example, as shown in Figure 2C, once the object (box 210) is placed in the destination position, the leader robot (robot arm 202) releases its grip and notifies the follower that the pick-and-place task is complete. In response, the follower (robot arm 206) releases its grip, and both robots (202, 206) are able to perform other tasks, such as individually picking and placing smaller / lighter objects (by returning to their original positions), and / or collaboratively performing the next pick-and-place task for another larger or heavier object.

[0025] Figure 3 is a block diagram showing one embodiment of a robot control system. In various embodiments, the robot control system 302 of Figure 3 includes or is included in the control computer 112 of Figure 1. In various embodiments, one or more modules or subsystems constituting the robot control system 302 of Figure 3 may be distributed across multiple computing nodes, such as computers and / or processors constituting one or more of the control computer 112, robot arm 102, and / or robot arm 106 of Figure 1.

[0026] In the example shown in the figure, the robot control system 302 comprises a hierarchical planner, scheduler, and / or control module, which includes a robot collaboration acceleration module 304 configured to facilitate the collaborative execution of tasks by two or more robots disclosed herein, and robot-specific controllers 306 and 308. For example, the controller 306 for robot 1 may relate to the robot arm 102 in Figure 1 and / or the robot arm 202 in Figures 2A to 2C, while the controller 308 for robot 2 may relate to the robot arm 106 in Figure 1 and / or the robot arm 206 in Figures 2A to 2C.

[0027] In various embodiments, each robot associated with the controller 306 for robot 1 and the controller 308 for robot 2 may operate independently, for example, to pick and place objects that the robots can individually manipulate. In various embodiments, a collaborative task using two or more robots may be initiated and / or executed by one or more communications transmitted between the controller 306 for robot 1 and the controller 308 for robot 2, bidirectional communications between the robot collaboration facilitator module 304 and the respective controllers 306 for robot 1 and 308 for robot 2, and / or communications between all three (or four or more) entities.

[0028] In the example shown, the robot control system 302 further includes a computer vision subsystem 310 configured to receive image and depth data from one or more 3D cameras and / or other sensors (such as camera 114 in Figure 1), and to use the received data to generate and / or update a three-dimensional view of the workspace. The output of the computer vision subsystem 310 may be provided to one or more of the robot collaboration acceleration module 304, the controller for robot 1 306, and the controller for robot 2 308, to enable the robots to collaboratively initiate and perform tasks for picking and placing items. For example, image data may be used to determine if a box or other object is too large and / or heavy to pick and place with a single robot. The three-dimensional view of the workspace and the objects within it may be used, for example, to determine the respective grasping strategies and / or positions for each robot, to determine multiple collision-free trajectories for moving each robot's end effector to its corresponding pick position, and to determine collision-free trajectories for collaboratively moving the object to the destination position where the object is placed.

[0029] Figure 4 is a state diagram showing one embodiment of a robotic system configured to control multiple robots to collaboratively perform a task. In various embodiments, the state diagram 400 in Figure 4 may be performed by and / or with respect to robots configured to collaboratively perform a task using two or more robots. In some embodiments, the state diagram 400 in Figure 4 may be performed by the control computer 112 in Figure 1, and / or by one or more of the robot collaboration acceleration module 304, the controller for robot 1 306, and the controller for robot 2 308 in Figure 3.

[0030] In the example shown in the figure, in state 402, the robots work individually to perform the task. For example, a robot may pick and place items individually, such as filling boxes or other containers in a kitting operation, placing items on a conveyor belt or other transport means in a sorting operation, or stacking items on a pallet. When the robots receive a suggestion that they need assistance to perform the task (404), such as a suggestion that an item that needs to be recognized and picked and placed is too large for one robot to grasp and move, the robots and / or controllers transition to state 406, in which collaborative execution of the task begins. For example, a communication may be sent to another robot (e.g., from the controller 306 for robot 1 to the controller 308 for robot 2 in Figure 3), or to a higher-level planner / scheduler (e.g., to the robot collaboration acceleration module 304 in Figure 3), or a higher-level planner / scheduler may recognize the need for collaborative execution of the task and initiate the transition to state 406.

[0031] In the example shown in the figure, the robot and / or controller may return to their individual work in state 402 via the "cancel assistance" transition 408. For example, the robot / controller and / or a higher-level planner / scheduler may determine that the task has already been performed by one or more other robots and / or assigned to other robots.

[0032] In some embodiments, in the “start collaborating” state 406, the robot / controller that has started the collaborative execution of a task communicates directly or indirectly with the helper robot, for example, by requesting assistance. Another robot may be assigned to assist and / or agree to assist. The robot may be assigned to assist and / or agree to assist at some future point in time or when future conditions occur (such as the completion of a task already started by the helper robot and / or a task with a higher priority). For example, a task to remove other objects from around a large or heavy object to facilitate a collaborative task may have a higher priority and therefore be completed first. When the helper robot is ready to perform the collaborative task, the helper robot notifies the task initiator directly or indirectly (for example, via a higher-level planner / scheduler (such as the robot collaboration facilitator module 304 in Figure 3)) that the helper robot is ready and prompts a transition to the “start collaborating” state 412 410. In the example shown in the diagram, the helper may directly transition from the state of working individually (402) to the state of "starting to collaborate" (412) through the transition 414 of "providing support".

[0033] When all participating robots are ready in the "start collaborating" state 412, a "leader" is determined as needed, and the leader transitions to the "acting as leader" state 418 (416), while the followers transition to the "acting as followers" state 422 (420). In the "acting as leader" state 418 and the "acting as followers" state 422, the leader and followers collaborate as disclosed herein to collaboratively perform a task, such as picking and placing large or heavy objects, as shown in the examples in Figures 2A and 2C. Once the task is completed, the leader and followers return to the "working individually" state 402 (424, 426) and resume their individual work.

[0034] Figure 5A is a flowchart illustrating one embodiment of a process for collaboratively performing a task as a “leader” robot in one embodiment of a robot system disclosed herein. In various embodiments, the process 500 in Figure 5A may be performed by a robot controller associated with a robot participating as a “leader” in the collaborative performance of a task by two or more robots disclosed herein.

[0035] In the example shown in the figure, in step 502, an instruction is received to start a collaborative task (together with one or more other robots) in the role of “Leader”. For example, an instruction may be received to collaboratively perform a pick-and-place task. In step 504, the Leader determines the position to grasp the object and plans a trajectory to safely move its end effector to the grasping position, and in step 506, the Leader moves the end effector to the grasping position along the trajectory. In step 508, the Leader determines a trajectory to move the object to an associated destination (independently of any other robots). For example, a model of the robot and its kinematics, as well as images and / or other information about the workspace (e.g., configuration data, CAD files, etc.), one or more attributes of the object (e.g., dimensions, stiffness, etc.), and image / sensor data may be used to plan the trajectory. In step 510, the robot receives an indication from one or more “follower” robots that they are ready to begin collaborative execution of the task. Accordingly, in step 512, the “leader” robot moves its end effector (and the object jointly grasped by the leader and followers) to the destination along a trajectory determined by the leader. In step 514, once the object is placed at the destination, the leader robot releases its grip and notifies the follower robots that the task is complete. In various embodiments, the leader then resumes individual operation.

[0036] Figure 5B is a flowchart illustrating one embodiment of a process for collaboratively performing a task as a “follower” robot in one embodiment of a robot system disclosed herein. In various embodiments, the process 520 in Figure 5B may be performed by a robot controller associated with a robot participating as a “follower” in the collaborative execution of a task by two or more robots disclosed herein.

[0037] In the example shown in the figure, in step 522, the Follower receives instructions to perform a task in collaboration with one or more other robots in the role of “Follower,” as disclosed herein. In step 524, the Follower determines a gripping point (for example, a point on the side of the object opposite to the side indicated by the “Leader” to be gripped) and plans a trajectory to move to a position to grip the object at that point. In step 526, the Follower moves its end effector to the determined gripping position and grips the object, for example, in response to receiving an indication that the Leader has completed its grip. In step 528, information on the position and orientation of the Leader’s end effector is received, and the Follower uses this information, along with information about the object (for example, the size of the object in the dimension separating the Leader’s end effector and the Follower’s end effector), to calculate a transformation. In various embodiments, the transformation includes a matrix or other mathematical construct applicable to the position and orientation of the leader's end effector (typically expressed in the leader's reference frame) to provide a position and orientation corresponding to the follower's end effector that maintains the relative position and orientation of the follower's end effector with respect to the leader's end effector as the end effector and the object grasped between them are moved through the workspace to the destination where the object is to be placed. In step 530, the follower robot notifies the leader that the follower is “ready” (e.g., the follower is ready to grasp an object, calculate the transformation, and maintain the position of its end effector with respect to the leader's end effector (e.g., opposite side)).

[0038] In step 532, when the leader robot begins moving along a trajectory independently determined by the leader, the follower utilizes the transformation it has calculated and the position and orientation information of the leader's end effector that is received sequentially as the leader's end effector moves through the workspace. For example, for each of at least a portion of the received positions and / or orientations of the leader's end effector, the follower calculates a new target position and / or orientation for that end effector and applies a torque to its motor that is determined to be necessary to minimize the error (e.g., difference) between the current position and / or orientation of that end effector and the current target.

[0039] In step 534, the follower receives an indication (e.g., from the leader) that the collaborative task is "completed," and accordingly, the follower releases the grip, and process 520 ends.

[0040] In various embodiments, the techniques disclosed herein are used, for example, to collaboratively perform pick-and-place tasks in connection with singulation / sorting, kitting, palletization or depalletization, and / or loading or unloading onto trucks or other containers.

[0041] Figure 6A shows an embodiment of a robotic system configured to collaboratively pick and place objects using two or more robots. In the example shown, the system and environment 600A demonstrate the use of the techniques disclosed herein in the context of singulation / sorting operations. Items of varying sizes, shapes, and other attributes arrive via the retrieval and transport means 602, such as gravity feed chutes or ramps and / or retrieval conveyor belts or similar structures. In the example shown, robots 202 and 206 of Figures 2A–2C are used in independent operation mode to pick items from the retrieval and transport means 602 and place the items one by one on the conveyor 604. Image data from camera 606 may be used to generate a three-dimensional view of the workspace, enabling robots 202 and 206 to identify and prioritize target objects, plan and strategize for grasping objects, and pick and place objects.

[0042] In various embodiments, robots 202 and 206 operate independently but cooperatively in the operating mode shown in Figure 6A. For example, robots 202 and 206 may alternately pick up from the intake and transport means 602 and place items on the conveyor 604. While one (for example, 202) is picking from the intake and transport means 602, the other is placing items on the conveyor 604, and vice versa.

[0043] Figure 6B shows an embodiment of a robotic system configured to collaboratively pick and place objects using two or more robots. In the example and configuration shown in Figure 6B, robots 202 and 206 are used to collaboratively pick and place a large box that has arrived in the pick area of ​​the intake and transport means 602. Image data from camera 606 may be used to detect the large box and / or to determine the weight and / or other attributes of the box (e.g., by lookup) that indicate the need to collaboratively pick and place the box using two robots 202 and 206.

[0044] In various embodiments, robots 202 and 206 cooperate to pick and place large boxes as disclosed herein. For example, robot 202 may act as a “leader” robot to perform process 500 in Figure 5A, while robot 206 may act as a “follower” robot to perform process 520 in Figure 5B, or vice versa.

[0045] Once the large box shown in Figure 6B has been placed onto the conveyor 604, robots 202 and 206 may resume their individual operations as described above.

[0046] Figure 7 is a flowchart illustrating one embodiment of a process for collaboratively picking and placing an object using two or more robots. In various embodiments, the process 700 in Figure 7 may be performed by one or more computers (such as the control computer 112 in Figure 1) and / or by one or more other computers and / or processors constituting the robotic system disclosed herein. In the example shown, step 702 determines the need for two or more robots to collaboratively perform a pick-and-place task. For example, a computer (such as the control computer 112 in Figure 1), and / or a controller or other control module associated with an individual robot, and / or a higher-level controller of a hierarchical controller may make that decision. In step 704, the two or more robots are assigned (e.g., by themselves, by a coordinator, etc.) to collaboratively perform the task, and for each, the corresponding pick position (on the object) and position (e.g., end effector and / or robot arm) are determined (or attempted to be determined). If, in step 706, it is determined that a sufficiently clear view is not available for one or more of the robots to determine the pick position, then in step 708, one or more robots may be used to move other objects out of the way for the purpose of providing a clearer view. If the pick position is visible (step 706), or if the object is moved to provide a clear view (step 708), then in step 710, it is determined whether all participating robots have a clear path or trajectory to move to their respective pick positions without collision. If any robot does not have a clear path (step 712), then in step 714, one or more robots may be used, as necessary, to move an object for the purpose of providing a clear path. For example, the target object may be pushed or pulled out of a cluttered pile of items.Alternatively, objects adjacent to the target object, or objects in the path between the end effector and its pick position, may be moved out of the way by a robot that needs to remove obstacles in the path to the pick position, or by another robot.

[0047] In some cases, a robot may not be able to clearly see the position of the object it is picking up. In some embodiments, such a robot may use force sensors or other tactile feedback to grope its way to the position for grasping the object.

[0048] Once all participating robots have a clear path to their pick locations (steps 712, 714), the robots work collaboratively to perform the pick-and-place task as disclosed herein. For example, one robot may act as a “leader” and perform process 500 in Figure 5A, while another robot acts as a “follower” and perform process 520 in Figure 5B.

[0049] In various embodiments, the techniques disclosed herein may be used to collaboratively pick and place objects, such as objects that are too heavy, soft, or bulky for a single robot to pick and place, using two or more robots.

[0050] Although the embodiments described above have been explained in some detail for the sake of clarity, the present invention is not limited to the details provided. Many alternative methods exist for carrying out the present invention. The disclosed embodiments are illustrative and not intended to be limiting.

Claims

1. It is a robotic system, A first robot arm having a first end effector, A second robot arm having a second end effector, A control computer configured to pick and place one or more objects using the first robot arm and the second robot arm, which includes working collaboratively to pick and place one or more of a plurality of objects using the first robot arm and the second robot arm, A system that includes these features.

2. A system according to claim 1, further comprising a camera positioned to generate image data associated with the workspace in which the first robot arm and the second robot arm are located.

3. A system according to claim 2, wherein the control computer is configured to utilize the image data to determine whether to work collaboratively with the first robot arm and the second robot arm to pick and place a given object.

4. The system according to claim 3, wherein the determination is at least in part based on the attributes of the object determined directly or indirectly using the image data.

5. The system according to claim 1, wherein the control computer includes two or more processors distributed across two or more nodes.

6. A system according to claim 1, wherein the control computer runs a hierarchical planner comprising separate robot controllers for each of the first robot arm and the second robot arm, and a higher-level controller configured to coordinate the movement of the first robot arm in cooperation with the second robot arm to collaboratively pick and place one or more of the objects.

7. The system according to claim 1, wherein the first robotic arm operates in leader mode and the second robotic arm operates in follower mode to perform a given task of collaboratively picking and placing a given item.

8. A system according to claim 7, wherein the first robot arm, in leader mode, is configured to independently plan a trajectory for moving a given object to be collaboratively picked and placed, grasp the given object, and move the given object along the planned trajectory.

9. The system according to claim 8, wherein the second robot arm, in the follower mode, is configured to grasp the given object, calculate a transformation based at least partially on the position and orientation of the first end effector, and notify the first robot arm that the second robot arm is ready to move the given object.

10. A system according to claim 1, wherein the first robot arm and the second robot arm are configured to pick and place objects individually when they are not working collaboratively to pick and place one or more of the objects.

11. A system according to claim 1, wherein the control computer is configured to move one or more of the plurality of objects out of line of sight to the given object from a camera or other sensor, at least in part on the determination that a more unobstructed view of the given object to be collaboratively picked and placed is necessary for collaboratively performing a pick and place task relating to the given object.

12. A system according to claim 1, wherein the control computer is configured to use one or both of the first and second robotic arms to pull or push the given object into the field of view of a camera or other sensor in order to facilitate a task of collaboratively picking and placing the given object using the first and second robotic arms.

13. A system according to claim 1, wherein the control computer is configured to use one or both of the first and second robotic arms to move one or more of the plurality of objects out of the way, so that one or both of the first and second robotic arms can be used to grasp a given object.

14. A system according to claim 1, wherein when the control computer is not working collaboratively with the first robot arm and the second robot arm to pick and place a given object, the first robot arm and the second robot arm are configured to alternately pick and place objects included in the plurality of objects.

15. A system according to claim 1, wherein the control computer is configured to utilize force sensors or other tactile feedback to grasp an object located in a position invisible to the control computer by touching it using the first robot arm or the second robot arm.

16. A system according to claim 1, wherein the control computer is configured to ensure that the first robot arm and the second robot arm do not collide with each other or with obstacles in the workspace when performing a pick-and-place task.

17. It is a method, A method comprising using a first robotic arm and a second robotic arm to pick and place one or more of a plurality of objects, wherein the robotic arms work collaboratively to pick and place one or more of a plurality of objects.

18. A method according to claim 17, further comprising using image data from a camera to determine whether to collaboratively work with the first robotic arm and the second robotic arm to pick and place a given object.

19. A method according to claim 17, wherein the first robotic arm and the second robotic arm work collaboratively to pick and place a given object, the first robotic arm being used in leader mode and the second robotic arm in follower mode.

20. A computer program product, which is embodied in a non-temporary computer-readable medium, A computer program product comprising computer instructions for picking and placing one or more objects using a first robotic arm and a second robotic arm, including the collaborative work of using the first robotic arm and the second robotic arm to pick and place one or more of a plurality of objects.

Images