Robotic transport system and method thereof

Abstract

A robot is provided. In some embodiments, a robot includes a first imaging system configured to detect a presence of an object / component in a workspace of the robot and a second imaging system configured to determine a position of the object / component relative to an end effector of the robot.

Description

Technical Field

[0001] The present disclosure relates generally to material handling and, more particularly, to stereoscopic object recognition of objects and the transport of those recognized objects. Background Art

[0002] Robots are often used to perform various tasks on objects in a work environment. Typically, the robot must be manually taught the location of the object by a human collaborator. The robot can then perform tasks on the object, such as moving the object between different locations. Summary of the invention

[0003] In some embodiments, a robot is disclosed, the robot comprising: an articulated robot arm, the robot arm being configured to be backdriven by a force applied by a person or cooperative operation in a workspace of the robot arm; an end effector disposed on the robot arm, the end effector comprising a gripper configured to grip an object; a first imaging system, the first imaging system comprising a first imager configured to capture images in a first direction; a second imaging system, the second imaging system comprising a second imager configured to capture images in a second direction; and a controller, the controller being configured to: control the first imager or the second imager to obtain a set of stereo images of a target disposed in the workspace; determine a position of the end effector relative to the target based on the images of the target, wherein the images include only a set of stereo images obtained by the first imager or the second imager; determine a position of the end effector relative to the object based at least in part on the determined position of the end effector relative to the target and the position of the target relative to the object; and control robot movement based on the determined position of the end effector relative to the object using images obtained only by the imager that obtained the set of stereo images.

[0004] In some embodiments, a method of controlling a robot is disclosed, the robot comprising an articulated robotic arm and a gripper disposed on an end effector of the robotic arm, the method comprising: controlling a first imager or a second imager to obtain a set of stereo images of a target disposed in a workspace, the first imager pointing in a different direction from the second imager; determining a position of the end effector relative to the target based on the images of the target, the images comprising only the set of stereo images obtained by the first imager or the second imager; determining a position of the end effector relative to the object based at least in part on the determined position of the end effector relative to the target and the position of the target relative to the object; and controlling the movement of the robot based on the determined position of the end effector relative to the object using images obtained only by the imager that obtained the set of stereo images; wherein the robotic arm is configured to be back-driven in response to a force applied by a person or cooperative operation in the workspace.

[0005] It should be understood that the aforementioned concepts and the additional concepts discussed below can be arranged in any suitable combination, as the present disclosure is not limited in this respect. In addition, other advantages and novel features of the present disclosure will become apparent from the following detailed description of various non-limiting embodiments when considered in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical object illustrated in various figures may be represented by the same reference numeral. For purposes of clarity, not every object may be labeled in every drawing. In the drawings:

[0007] Figure 1 shows a perspective view of a robot according to an embodiment;

[0008] Figure 2 Shows Figure 1 A top view of the robot;

[0009] Figure 3 shows a top view of a plurality of robots arranged in a flexible working environment according to an embodiment;

[0010] Figure 4 shows a lower perspective view of a robotic arm according to an embodiment;

[0011] Figure 5 Shows Figure 4 An upper perspective view of a robot arm;

[0012] Figure 6 shows a simplified representation of a method of determining a three-dimensional position of an object according to an embodiment; and

[0013] Figure 7 A method of operating a robot according to an embodiment is shown. DETAILED DESCRIPTION

[0014] It should be understood that the features of the present invention are described herein with reference to the accompanying drawings showing illustrative embodiments. The illustrative embodiments described herein are not necessarily intended to illustrate all embodiments including all inventive features, but are intended to describe selected combinations of several illustrative embodiments and inventive features. Therefore, in view of the illustrative embodiments, the inventive features are not intended to be narrowly interpreted. In addition, it should be understood that the inventive features can be used alone or in any suitable combination with other inventive features.

[0015] Robots are often used to perform tasks in collaboration with humans in a work environment. In such a work environment, the robot and the human may each perform various tasks on objects to produce a desired deliverable (e.g., a product, test results, etc.). In use, such robots are typically taught the pick-up and drop locations of an object, as well as any other movements required to perform a task with that object (e.g., soldering a component on a PCB, mixing ingredients in a test tube, etc.), and the robot is then configured to repeat such movements on subsequent objects.

[0016] The inventors have recognized the advantages of a robot that is configured to automatically adapt to changes in the work environment. For example, a collaborator may wish to frequently physically reposition the robot in the workspace based on the number of robots or human collaborators available, without having to manually re-teach the robot pick and place locations. Additionally, human collaborators may place objects imprecisely, and it may be desirable for the robot to be configured to accommodate the imprecise placement of the human collaborator.

[0017] The inventors have also recognized the advantages of a robot configured to automatically perform tasks on an object without requiring additional input from a collaborator. Such a configuration can improve efficiency and avoid dead time (e.g., the robot can begin working on an object as soon as it is placed in the workspace, rather than waiting for input commands from a collaborator).

[0018] Thus, the inventors have recognized the advantages of a robot that can be configured to sense the presence of an object and / or other component (e.g., a holding station, a processing station, a workbench, etc.) and determine the position of the object / other component relative to the robot's end effector. The robot can then be configured to perform a task on the object (e.g., move the object from a holding station to a processing station, or from a processing station to a holding station, perform work on the object at the processing station, etc.). This configuration can allow a human collaborator to quickly reconfigure a work environment without manually teaching the robot the location of objects and / or other components. This configuration can also allow a collaborator to work in collaboration with the robot without requiring the collaborator's high precision, and / or allow the robot to adapt to changing conditions, such as variable format object holders and objects. This configuration can also allow tasks to be performed automatically with minimal input from the collaborator.

[0019] In some embodiments, the robot may include a first imaging system and a second imaging system. In some embodiments, each imaging system is configured to be used by a controller to independently control the movement of the robot. The first imaging system may include a first imager (e.g., a camera and an image processing module, such as image analysis software and associated hardware), and the second imaging system may include a second imager. The controller may be configured to control the first imager or the second imager to image the workspace and determine the presence of objects and / or other components in the workspace. When an object and / or other component is detected, the controller may then control the first imager or the second imager to obtain one or more images (e.g., stereo images) of a target, which is set at a predetermined position relative to the object / other component (e.g., set on the object / component, or set adjacent to the object / component). The controller may then use these stereo images to determine the three-dimensional position of the target relative to the end effector and / or other parts of the robot. The controller may then determine the position of the object / other component relative to the end effector based on the determined position of the end effector relative to the target and the predetermined position of the object / component relative to the target. In some embodiments, one imaging system may be used to detect the presence of an object / component, while another imaging system may be used to determine the three-dimensional position of the target set on the object.

[0020] One potential use case is as follows. A collaborator may position a processing station in the robot's workspace. The robot may detect the presence of the processing station via the first imager or the second imager and learn the location of the processing station via the first imager or the second imager. A collaborator may position a first holding station including one or more objects of the first group anywhere in the robot's workspace. The robot may detect the presence of the first holding station and the objects of the first group associated with the holding station via the first imager or the second imager. The robot may determine the position of the first holding station and the objects of the first group relative to the robot's end effector via the first imager or the second imager. The robot may move the object to the processing station, perform work on the object, and move the object back to the first holding station. The robot may repeat the process for each object in the first holding station. While the robot is performing a task on the objects of the first group, a collaborator may position a second holding station in the robot's workspace. The robot may detect the presence / position of the second holding station / objects of the second group via the first imager and / or the second imager and, once the task on the objects of the first group is completed, begin automatically performing the task on the objects of the second group. The collaborator may then move the first holding station and the first group of objects to a different location for further processing, packaging, etc.

[0021] As described above, the inventors also recognize that it may be desirable for the robot to work in collaboration with humans. Such collaboration may require a human collaborator to enter the robot's workspace to perform tasks. The inventors have recognized the advantages of configuring the robot to be back-drivable so that an external force applied by a person can stop or move the robot arm from its controlled motion trajectory. This back-drive capability can allow a human collaborator to push the arm away without damaging the arm. If the arm contacts a person or other component in the workspace, this back-drive capability can also allow the arm to stop, thereby limiting damage to the person, robot, and / or other components. In some embodiments, this back-drive capability can also be used to ensure the correct positioning of an object. For example, during a mating operation, contact between an object and an object receiving portion (e.g., a slot in a holding station for receiving an object) can be used to guide the object into the object receiving portion in the holding station.

[0022] The inventors have also recognized the advantages of configuring a robot to perform tasks in a life science environment. Thus, in some embodiments, the object may be a sample holder configured to hold materials for laboratory analysis. Such a sample holder may be a life science sample tray, a life science microplate, a single life science sample tube held in a tray or microplate; or any piece of life science laboratory ware for holding any type of sample, such as samples including liquid and / or solid materials.

[0023] Figure 1 and Figure 2 A perspective view and a top view of an illustrative robot 100 including one or more inventive features are shown, respectively. The robot may include a base 102, an articulated robot arm 104, and an end effector 108 disposed at the end of the articulated robot arm. The robot 100 may include a controller 120 configured to control a drive motor 132 in the robot arm 104 to move the end effector 108 within a workspace 110. The robot may be configured to move objects 200 between various stations and / or perform operations on the objects 200. Such stations may include a holding station 202 configured to hold a plurality of objects 200 in a receiving portion 208, and a processing station 204 that performs operations on the objects 200. In some embodiments, the robot 100 may be configured to perform various operations on the objects 200 when the objects 200 are placed at the processing station 204. Alternatively, or in addition, a human collaborator or other robot / machine may perform operations on the objects at the processing station 204.

[0024] The robot arm 104 can be any suitable type of robot arm, such as a gantry robot, a SCARA robot, a telescopic or sliding arm robot, etc. In some embodiments, as Figure 2As seen in FIG. 1 , the processing station 204 and the robotic arm 104 can be mounted on the same base 102 such that the processing station 204 is always at a predetermined position relative to the base 102. In other embodiments, the processing station 204 is mounted on a separate base.

[0025] As described above, it may be desirable for the robot to be configured to automatically detect the presence of the object 200 and / or other components (e.g., the holding station 202, the processing station 204, etc.), and automatically determine the position of the end effector 108 of the robot 100 relative to the object / other components. Therefore, the robot 100 may include a first imaging system 112 having a first imager 114 and a second imaging system 116 having a second imager 118. The robot 100 may also include a controller 120 configured to control the first imaging system 112 and the second imaging system 116, and use information from the first imaging system 112 or the second imaging system 116 to control the movement of the robot 100.

[0026] The controller 120 may be configured to control the imaging systems 112, 116 to detect the presence of objects and / or other components in the robot's workspace 110 by imaging the workspace 110 with the imagers 114, 118. The imagers 114, 118 may be disposed in any location that allows imaging of the workspace 110. More details regarding the location of the imagers 114, 118 will be discussed further below.

[0027] Once the presence of an object / part is detected, the controller 120 can be configured to determine the relative positioning of the end effector 108 relative to the object / other part using the imaging systems 112, 116. Each object / part can be associated with at least one target 206 disposed at a predetermined position relative to the object / part. For example, in some embodiments, each object 200 can include a target 206 disposed on the object 200. In some embodiments, the target 206 can be disposed adjacent to the object / part (e.g., on a receiving portion 208). In some embodiments, the target 206 can be disposed on the station 202 and / or the station 204, and the object 200 can be positioned in the station 202 / station 204 at a predetermined position relative to the target 206. For example, in some embodiments, the holding station 202 includes a target 206 disposed on a portion of the holding station 202, and the holding station 202 includes a receiving portion 208 configured to hold the object 200 at a predetermined position relative to the target 206. The controller 120 can control the imaging systems 112, 116 to use the imagers 114, 118 to obtain stereoscopic images of the target 206 using any suitable stereoscopic imaging process, as discussed further below. The controller 120 can use these stereoscopic images to determine the three-dimensional position of the target 206 relative to the end effector 108. The controller 120 can then determine the three-dimensional position of the object / component relative to the end effector 108 based on the determined position of the end effector relative to the target and based on the predetermined position of the target relative to the object / component. In some embodiments, a single target 206 is used to determine the position of the end effector 108 relative to the object 200 / other component. In some embodiments, multiple targets 206 or multiple groups of targets 206 are used to determine the position of the end effector 108 relative to the object / other component.

[0028] In some embodiments, the controller 120 can independently use the imaging systems 112, 116 to control the movement of the robot 100. For example, the imaging systems 112, 116 can have non-overlapping fields of view, and each imaging system 112, 116 can be used separately to determine the position of an object relative to the robot and control the subsequent movement of the robot relative to the object. For example, the imaging system 112 can be used to determine the position of the end effector 108 relative to the object within its field of view (e.g., using one or more sets of stereo images captured by the first imager 114), and the imaging system 116 can be used to determine the position of the end effector 108 relative to the object within its field of view (e.g., using one or more sets of stereo images captured by the second imager 118). This arrangement can provide advantages in some robot and imager configurations such as where the first imager 114 and the second imager 118 are mounted on the end effector 108 and the end effector 108 is limited in its degrees of freedom and therefore limited in how the first imager 114 and the second imager 118 can be oriented for imaging. For example, in some configurations, due to restrictions on the motion of the end effector, the first imager 114 may have only a horizontal field of view, while the second imager 118 may have only a vertically oriented (e.g., downward) field of view, also due to restrictions on the motion of the end effector and / or its degrees of freedom. However, since the first imager 114 and the second imager 118 can be operated to jointly determine the position of all objects in the robot's workspace, the robot can be controlled to move in a wide range of positions relative to the objects. This arrangement can also allow the controller 120 and one of the imaging systems 112, 116 to capture stereo images while the robot 100 is being moved for other tasks, thereby avoiding the end effector from being moved exclusively for the purpose of determining the position of an object. For example, when the robot is being moved based on position detection and control using information from the second imaging system 116, the first imaging system 112 can be used to capture a stereo image of another object in the workspace, so that the controller 120 can determine the position of the object relative to the robot (e.g., the end effector 108) and use this information to control the robot movement later after the current task is completed. This can allow for more efficient operation of the robot, as the robot's motion during a task can be used to capture stereo images of the object for planning future motion operations of the robot, rather than requiring motion of the end effector solely for the purpose of object detection and position determination relative to the robot.

[0029] In some embodiments, the imaging systems 112 and 116 can be integrated into the robot, attached to the end effector 108 or other robot part. In some embodiments, the imaging systems 112 and 116 can be selectively attachable / detachable so that the imaging systems 112 and 116 can be used with other robots.

[0030] As described above, it may be desirable for the robot 100 to perform tasks on an object 200 placed in the robot's workspace 110 without requiring additional input from a human collaborator in order to reduce robot downtime and improve overall efficiency. Thus, once the position of the object / other component relative to the end effector 108 is determined, the controller 120 may automatically control the drive motor 132 of the robot 100 to perform various tasks on the object 200. For example, in some embodiments, the end effector may include a gripper 122 configured to grip the object 200 and move the object 200 between a holding station 202 and a processing station 204. In some embodiments, the gripper 122 may also perform operations on the object 200 at the processing station 204. In some embodiments, the robot 100 can be equipped with a tool changer 124, and the robot 100 can be configured to move the object 200 to the processing station 204 using the gripper 122, then move to the tool changer 124, and change the gripper 122 to a different tool (e.g., a stirring tool, a welding tool, etc.) to perform an operation on the object 200. In some embodiments, the processing station 204 itself can be configured to perform various operations on the object 200. For example, the processing station 204 can include a centrifuge configured to spin the object 200.

[0031] In some embodiments, the robot 100 can be configured to perform operations only on objects 200 placed at the processing station 204. For example, a human collaborator 400 or other robot can place an object 200 at the processing station, and the robot can then detect the position and determine the three-dimensional position of the object 200 relative to the end effector and perform operations on the object 200. The human collaborator 400 or other robot can then remove the object 200 from the processing station 204.

[0032] As described above, a person may wish to work in collaboration with a robot. Such collaborative work may require that the person enter the workspace 110. Therefore, as described above, if a collision occurs between a human collaborator and the robot 100, it may be desirable that the robot 100 be able to be back-driven to limit any injury to the person and to limit any damage to the robot 100.

[0033] As used herein, the term "backdriven" or "backdrive" refers to the ability of an external force (such as an external force applied by a person or any other object / obstacle physically engaging / contacting the arm) applied to the robotic arm 104 to overcome the drive motor 132 in the robotic arm 104. The external force may prevent the robotic arm 104 from moving along its controlled trajectory, and / or may change the trajectory of the robotic arm 104. This backdrive capability may allow the robotic arm 104 to stop moving if the robotic arm 104 contacts a person or other obstacle along the robotic arm's controlled trajectory. This backdrive capability may also allow a person to manually change the trajectory of the robotic arm 104, if necessary, to avoid injury and / or damage.

[0034] In some embodiments, the inventors have also recognized the advantages of utilizing the back-drive capability of the robotic arm to ensure a proper “fit” between the object 200 and the object receptacle 208 in the holding station 202 during mating operations. Contact between the object 200 and the side of the object receptacle 208 can back-drive the robotic arm 104 to guide the object 200 into and out of the object receptacle 208. This configuration can allow the movement of the robotic arm to be corrected when necessary due to errors (e.g., object or object receptacle out of tolerance, inaccurate target placement, errors in relative position determination, movement of the station during operation, etc.).

[0035] In some embodiments, the robot arm 104 may include any suitable feedback sensor 134. For example, the robot arm 104 may include a position sensor to determine and / or verify the position of each arm link relative to the other arm links. The robot arm may also include a force sensor configured to detect an external force applied to the robot arm. In some embodiments, the controller 120 may be configured to reduce / remove the power of the drive motor 132 in response to an external force applied to the robot arm (e.g., in the case of a person pushing the arm). In some embodiments, the object 200 may be a sample holder that holds materials for laboratory analysis. Such a sample holder may be a life science sample tray, a life science microplate, a single life science sample tube held in a tray or microplate; or any piece of life science laboratory ware. The robot may be configured to perform any suitable task or combination of tasks on the sample holder, such as mixing materials in the sample holder, transporting the sample holder between processing stations for a person or other worker to perform work, measuring parameters of the material in the sample holder, and / or any other task or combination of tasks.

[0036] Although the above embodiments disclose that the robot arm performs tasks on sample holders in a life science environment, it is contemplated that the robot 100 can be configured to operate any tasks on any suitable object in any suitable working environment. For example, the robot can be used in a warehouse environment, a manufacturing plant (e.g., automotive, aerospace, electronics, semiconductors, etc.), or any other suitable environment. The object 200 can be a printed circuit board, warehouse cargo / container, automotive part, aviation part, electronic component, semiconductor wafer, or any other suitable object. The robot can be configured to perform any suitable task on such an object, such as milling, welding, brazing, etc.

[0037] Figure 3 A top view of a flexible work environment including one or more inventive features is shown. As described above, in some embodiments, it may be desirable for the robot 100 to be able to work in a flexible work environment. For example, more human collaborators 400 may be available to perform specific tasks on a certain day than on another day. Alternatively, it may be desirable for the robot to switch between different sets of tasks according to current needs, and such different tasks may require different positioning of the robot 100 and various objects / other components. Therefore, in some embodiments, the base 102 of the robot 100 can be configured as a movable base (e.g., with wheels) to allow the robot 100 to move on a floor or other supporting surface. Various holding stations 202, processing stations 204, and workbenches 210 may also be movable. The human collaborator 400 can position the robot 100 and various stations in any desired layout and command the robot to perform a defined set of tasks. The robot can then use an imaging system to detect the presence and position of each station 202, 204, and / or 210 and any object 200 disposed in the station, and then automatically perform the task for each of the objects 200. The layout of the various robots and stations can then be changed as needed to suit the relevant tasks. Figure 3 As seen in FIG. 1 , the first robot 100A may be configured to perform task A, and the second robot 100B may be configured to perform task B.

[0038] To perform Task A, the robot 100A can be configured to move an object from a holding station 202A to a processing station 204A, perform work on the object (with or without a tool changer), and move the object back to the holding station 202A. The robot 100A can be configured to image its workspace 110A to detect the presence of a holding station 202A including an unprocessed object 200A, determine the position of the object 200A relative to the end effector, and perform a defined operation on each object 200A. The human collaborator 400 can then simply push or otherwise move the holding station 202A into the workspace 110A, and the robot 100A can detect the object in the holding station 202A, determine the object position, and perform the task on the object. The human collaborator 400 can then remove the holding station 202A with the processed object 200A and place a new holding station 202A with the unprocessed object. In some embodiments, the human collaborator 400 can place the holding station 202A anywhere within the robot's workspace 110A.

[0039] To perform Task B, the robot B may be configured to move the object 200B from the holding station 202B to a pick / place location 212B disposed on the workbench 210B, and the human collaborator 400 may then remove the object 200B from the pick / place location, perform an operation on the object 200B, and place them back in the pick / place location 212B. The robot 100B may be configured to move the processed object 200B back into the holding station 202B. In some embodiments, the human collaborator may signal that the object 200B is ready to be placed back into the holding station 202B (e.g., with a button). In some embodiments, the robot 100B may be configured to automatically detect when the object 200B has been placed back into the pick / place location 212B, and when the placement is detected, move the object back into the holding station 202B. In some embodiments, the object 200B may include a target configured to be imaged by the first imaging system and / or the second imaging system of the robot 100B so as to allow the robot 100B to determine the position of the object 200B relative to the end effector 108B (e.g., in the event that the human collaborator 400 imprecisely places the object 200B in the pick / place location 212B).

[0040] If there is no / reduced need for Task B, the robot 100B can be repositioned so that at least one processing station 204A and at least one holding station 206A are disposed about the robot 100B, and the robot 100B can be commanded to perform Task A. Robot B can then detect the presence and location of the stations / other components associated with Task A in its workspace, and perform Task A on those objects. Although the above embodiment shows only two robots performing two different tasks, it is contemplated that any number of robots perform any number of tasks. Thus, each robot 100 can be configured to "position and go," wherein each robot 100 can be repositioned and controlled to perform different tasks as needed without the need to manually re-teach the robot the different locations of the objects / stations.

[0041] Figure 4 and Figure 5 An upper perspective view and a lower perspective view of an illustrative robot 100 including one or more inventive features are shown, respectively. As described above, the imagers of the first imaging system 112 and the imagers of the second imaging system 116 can be disposed in any suitable position that allows the imaging systems 112 and 116 to operate properly. For example, in some embodiments, the first imager is separated from the robot arm and disposed outside the workspace (e.g., above the workspace, to the side of the workspace, etc.). In some embodiments, as Figure 4 and Figure 5 As seen in FIG. 1 , the first imager 114 is disposed on the end effector 108, for example, the first imager 114 having a horizontally oriented imaging field of view is disposed on the end effector 108. Even when the object 200 is grasped by the grasper 122, as shown in FIG. Figure 5 As seen in FIG. 1 , such a forward-facing first imager 114 may also allow for presence detection and / or three-dimensional position determination. For example, such a configuration may be desirable if a collaborator were to move the holding station 202 before the robot 100 placed the object 200 in the receptacle 208. In such a case, the controller 102 may control the first imaging system 112 to detect the lack of a holding station presence and perform remedial action (e.g., suspending operation until the holding station 202 is replaced, alerting the human collaborator, etc.) rather than simply placing the object 200 where the receptacle 208 "should be" and causing the object 200 to fall to the floor.

[0042] When the first imager 114 is disposed on the end effector 108, the field of view of the first imager 114 may not be sufficient to view the entire workspace 110 at one time. Therefore, when imaging the workspace 110 to detect the presence of an object / other component, the controller 120 may control the robotic arm 104 to move the end effector 108 to a different position within the workspace 110 to allow the first imager 114 to view the entire workspace 110. For example, the end effector may be rotated 360° about the end effector vertical axis V to allow the forward-facing first imager to view the entire workspace.

[0043] In some embodiments, Figure 4 , the second imager 118 can be disposed at the bottom of the end effector 108 so that the second imager 118 can capture images in a direction that is substantially orthogonal to the direction in which the first imager 114 is configured to capture images. This configuration can allow the second imager 114 to capture stereoscopic images of a target 206H disposed on a horizontal surface, such as on the top surface of the object 200, as shown. Figure 5 Seen in.

[0044] As described above, in some embodiments, the controller 120 can be configured to detect the presence of an object / part in the workspace 110 and use the first imaging system 112 or the second imaging system 116 to determine the three-dimensional position of the object / part relative to the end effector 108. For example, this configuration may be desirable when the imagers 114 and 118 are mounted to the end effector of a SCARA robot arm because the end effector of the SCARA robot arm can only rotate about the vertical axis V and therefore cannot be tilted about the horizontal axis. In this configuration, the imager 114 of the first imaging system 112 can be disposed at the front of the end effector and configured to capture images in the horizontal direction, and the imager 118 of the second imaging system can be disposed at the bottom of the end effector and configured to capture images in the vertical direction, so that the imager of the first imaging system and the imager of the second imaging system have non-overlapping fields of view. The controller 120 can control the first imaging system to utilize a target disposed on a vertical surface (e.g., Figure 4 and Figure 5 The controller may control the second imaging system to detect and determine the three-dimensional position of the object / component using a target 206V disposed on a horizontal surface (e.g., Figure 5 Target 206H) in the image is used to detect and determine the three-dimensional position of the object / part.

[0045] This configuration may allow for presence detection and relative position determination even when an object is being manipulated by the end effector 108. For example, as described above, positioning the first imager 114 in front of the end effector 108 may allow the first imager to image the workspace to perform presence detection even when the robot 100 is currently grasping an object. If the first imaging system 112 is also configured for relative position determination, the controller 120 may be configured to sense whether the object holding station 202 has been moved (e.g., jostled by a collaborator) and to obtain a stereoscopic image of the vertical target 206V using the first imager 114 in order to determine the new position of the holding station 202 and / or the receptacle 208 relative to the end effector, and to control the robot arm 104 to place the object in the repositioned object receptacle 208.

[0046] The controller may use any suitable technique to obtain a stereoscopic image of the target 200 and determine the position of the target 200 relative to the end effector. For example, in some embodiments, Figure 6 As seen in FIG. 1 , the controller can control the position of the imagers 114 / 118 to obtain stereoscopic images. The imagers 114 / 118 can be moved to a first position 126 and controlled to obtain a first image of the target 200, and then moved to a second position 128 to obtain a second image of the target 200. The controller can use any suitable technique to use the stereoscopic images to determine the position of the target 206 relative to the end effector 108. For example, Figure 6 As seen in FIG. 1 , the controller 120 may use any suitable image analysis technique to identify two-dimensional pixel values ​​CX1, CY1 of a portion 214 of the target 200 (e.g., a corner of the target 200) in the first image and two-dimensional pixel values ​​CX2, CY2 of the same portion 214 of the target 200 in the second image. The controller may then employ a stereo imager model to estimate a three-dimensional position of the target 200 relative to the imager 114 / 118 based on the difference between CX1 and CX2 and the difference between CY1 and CY2. The controller may then determine a three-dimensional position of the end effector relative to the object / part based on the estimated position of the imager 114 / 118 relative to the target 200, the predetermined position of the target relative to the object / part, and the predetermined position of the end effector relative to the imager 116 / 118.

[0047] Although the above embodiments disclose the first imaging system and the second imaging system having a single imager, in some embodiments, the first imaging system 112 and / or the second imaging system 114 may each include a pair of imagers. One of the imagers in the pair of imagers may be disposed at the first position 126, and the other imager in the pair of imagers may be disposed at the second position 128. In some embodiments, the second imaging system 116 includes a second imager disposed at the first position 126 and a third imager disposed at the second position 128 adjacent to the second imager on the end effector. The second imager and the third imager may be controlled to simultaneously image the target from the positions 126 and 128 to obtain stereo images for relative position determination, thereby obtaining a set of stereo images 127 and 129.

[0048] Although the above embodiments disclose a robot with multiple imaging systems, in some embodiments, the robot may include a single imaging system, and the controller may be configured to control the single imaging system to detect the presence of an object / component in the workspace and determine the position of the end effector relative to the object / component using the above techniques. Such a single imaging system may be used, for example, on a gantry robot.

[0049] Although the above embodiments disclose that the robot arm performs tasks on sample holders in a life science environment, it is contemplated that the robot 100 can be configured to operate any tasks on any suitable object in any suitable working environment. For example, the robot can be used in a warehouse environment, a manufacturing plant (e.g., automotive, aerospace, electronics, semiconductors, etc.), or any other suitable environment. The object 200 can be a printed circuit board, warehouse cargo / container, automotive part, aviation part, electronic component, semiconductor wafer, or any other suitable object. The robot can be configured to perform any suitable task on such an object, such as milling, welding, brazing, etc.

[0050] Figure 7A method 300 of operating a robot including one or more inventive features is shown. At 302, the robot may detect the presence of an object in the robot's workspace. Such presence detection may be performed using any suitable technique, such as via the imaging system discussed above. At 304, the robot may obtain a stereoscopic image of a target in the workspace. Such a stereoscopic image may be obtained via the same imaging system as the imaging system used for presence detection, may be obtained via a separate imaging system as described above, or may be obtained via any other suitable technique. If an imaging system is used to obtain a stereoscopic image, the imaging system may obtain the stereoscopic image by imaging the target from multiple positions with a single imager, or may obtain the stereoscopic image with multiple imagers simultaneously. The target may be associated with the object (e.g., disposed on the object, disposed on a receiving portion that holds the object, disposed on a holding station and / or a processing station, etc.). At 306, the robot may determine a position of an end effector of the robot relative to the target based on the stereoscopic image. At 308, the robot may determine a position of the end effector relative to the object. At 310, the robot may perform a task on the object. These tasks may include grasping an object with a gripper disposed on an end effector, manipulating the position of an object, performing an operation on an object (e.g., brazing, heating, stirring, performing a test, etc.), or any other suitable task. At 312, the robot arm of the robot may be configured to be back-driven. This back-driving capability may be used for any suitable purpose, such as allowing a human collaborator to change the controlled trajectory of the robot arm, or ensuring a proper "fit" between an object and a receiving portion for holding the object.

[0051] Although the present teaching has been described in conjunction with various embodiments and examples, it is not intended to limit the present teaching to these embodiments or examples. On the contrary, as will be appreciated by those skilled in the art, the present teaching includes various alternatives, modifications and equivalents. Therefore, the foregoing description and accompanying drawings are only intended as examples.

Claims

1. A robot comprising: an articulated robotic arm configured to be backdriven by forces applied by a person or cooperating operator in a workspace of the robotic arm; an end effector disposed on the robotic arm, the end effector comprising a gripper configured to grip an object; a first imaging system comprising a first imager configured to capture an image in a first direction; a second imaging system comprising a second imager configured to capture an image in a second direction; as well as A controller, wherein the controller is configured to: controlling the first imager or the second imager to obtain a set of stereoscopic images of a target disposed in the workspace; determining a position of the end effector relative to the target based on an image of the target, wherein the image includes only the set of stereo images acquired by the first imager or the second imager; determining a position of the end effector relative to the object based at least in part on the determined position of the end effector relative to the target and the position of the target relative to the object; and Robotic motion is controlled based on the determined position of the end effector relative to the object using images acquired only by the imager that acquires the set of stereo images.

2. The robot according to claim 1, wherein: The object is one of: a sample holder, a life science sample tray, a life science microplate, a single life science sample tube held in a tray or microplate, a piece of life science labware, or a printed circuit board assembly.

3. The robot according to claim 1, wherein: One of the first imager and the second imager is configured to capture an image in a horizontal direction, and the other of the first imager and the second imager is configured to capture an image in a vertical direction.

4. The robot according to claim 1, wherein: The first imager and the second imager are disposed on the end effector.

5. The robot according to claim 1, wherein: The workspace includes a holding station and a processing station, and the robot is configured to move the object between the holding station and the processing station.

6. The robot of claim 5, further comprising a movable base configured for movement along a floor or other surface, wherein: The robot arm is arranged on the base.

7. The robot according to claim 6, wherein: At least one of the holding station and the processing station is configured to be movable relative to the base.

8. The robot according to claim 5, further comprising a receiving portion disposed on the holding station and configured to receive the object, wherein: The robot is configured to remove the object from the receiving portion and place the object in the receiving portion, wherein the robot arm is configured to be back-driven by a force generated due to contact between the object and the receiving portion when the robot removes or places the object in the receiving portion.

9. The robot according to claim 8, wherein: The receiving portion is a first receiving portion among a plurality of receiving portions provided on the holding station, and wherein each receiving portion is configured to slidably receive an object.

10. The robot according to claim 5, wherein: The robot is configured to perform at least one task on the object at the processing station.

11. The robot according to claim 1, wherein: The first imaging system and the second imaging system are selectively attachable to the robot.

12. The robot according to claim 1, wherein: The first imager and / or the second imager each comprises a single camera.

13. The robot according to claim 12, wherein: The first imager and / or the second imager is configured to obtain a stereoscopic image of the target by imaging the target from a plurality of positions.

14. The robot according to claim 1, wherein: The first imager and the second imager have non-overlapping fields of view.

15. The robot according to claim 1, wherein: The target is disposed on the object.

16. A method of controlling a robot comprising an articulated robot arm and a gripper disposed on an end effector of the robot arm, the method comprising: controlling a first imager or a second imager to obtain a set of stereoscopic images of an object disposed in a workspace, the first imager pointing in a different direction than the second imager; determining a position of the end effector relative to the target based on an image of the target, the image including only the set of stereo images acquired by the first imager or the second imager; determining a position of the end effector relative to an object based at least in part on the determined position of the end effector relative to the target and the position of the target relative to the object; as well as controlling the movement of the robot based on the determined position of the end effector relative to the object using images obtained only by the imager that obtains the set of stereo images; Wherein, the robotic arm is configured to be backdriven in response to forces applied by a person or cooperative operation in the workspace.

17. The method according to claim 16, wherein: The first imager and the second imager are disposed on the end effector, and wherein controlling the first imager or the second imager to obtain the set of stereoscopic images of the target in the workspace comprises: moving the end effector to a first position, capturing a first image of the target, moving the end effector to a second position, and capturing a second image of the target.

18. The method of claim 16, further comprising gripping the object with the gripper and moving the object between a holding station and a processing station.

19. The method of claim 18, further comprising backdriving the robotic arm in response to a force generated by contact between the object and a receiving portion disposed on the holding station.

20. The method of claim 16, further comprising moving a base on which the robotic arm is mounted across a floor or other surface.

Images