Robotic transport system

Abstract

Robots, exhibiting: an articulated robot arm configured to operate within a work area of ​​the robot arm; an end effector arranged on the robot arm, wherein the end effector has a gripper configured to grasp an object; a first imaging system comprising a first imager positioned at the end effector and configured to capture images in a first direction; a second imaging system comprising a second image sensor located at the end effector and configured to acquire images in a second direction orthogonal to the first direction; and a control unit configured to: Control either the first image sensor or the second image sensor to obtain a set of stereo images of a target positioned in the workspace; Determining the position of the end effector relative to the target based on images of the target; Determining a position of the end effector relative to the object based on the determined position of the end effector relative to the target and a position of the target relative to the object; and Controlling robot movement based on the determined position of the end effector relative to the object.

Description

AREA

[0001] The present disclosure relates generally to material handling and in particular to the stereoscopic target identification of objects and the transport of these identified objects. BACKGROUND

[0002] Robots can generally be used to perform a variety of tasks on objects in a work environment. Typically, these robots need to be manually taught the positions of these objects by a human operator. Such robots can then perform tasks on these objects, such as moving them between different positions. SUMMARY

[0003] In some embodiments, a robot is disclosed, wherein the robot comprises an articulated robot arm configured to retract by a force exerted by a human or a coupling process in a work area of ​​the robot arm, an end effector arranged on the robot arm, the end effector having a gripper configured to grasp an object, a first imaging system comprising a first imager configured to acquire images in a first direction, a second imaging system comprising a second imager configured to acquire images in a second direction, and a control unit configured to control either the first imager or the second imager to obtain a set of stereo images of a target arranged in the work area.a position of the end effector relative to the target is determined based on images of the target, wherein the images comprise only the set of stereo images received from the first imager or the second imager, in order to determine a position of the end effector relative to the object, at least partially, based on the determined position of the end effector relative to the target and a position of the target relative to the object, and controls the robot movement based on the determined position of the end effector relative to the object using images captured only by the imager that received the set of stereo images.

[0004] In some embodiments, a method for controlling a robot is disclosed, the robot having an articulated robot arm and a gripper arranged on an end effector of the robot arm, the method comprising controlling a first imager or a second imager to obtain a set of stereo images of a target arranged in the workspace, wherein the first imager points in a different direction than the second imager, determining a position of the end effector relative to the target based on images of the target, wherein the images comprise only the set of stereo images obtained from the first imager or second imager, and determining a position of the end effector relative to an object at least partially based on the determined position of the end effector relative to the target and a position of the target relative to the object.and to control the movement of the robot based on the determined position of the end effector relative to the object using images obtained only from the imager that received the set of stereo images, with the robot arm configured to retract in response to a force exerted by a human or a coupling process in the workspace.

[0005] It should be noted that the foregoing concepts and the additional concepts explained below can be arranged in any suitable combination, since the present disclosure is not limited in this respect. Furthermore, additional advantages and novel features of the present disclosure will become apparent from the following detailed description of various non-limiting embodiments in conjunction with the accompanying figures. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings are not to scale. In the drawings, any identical or nearly identical object depicted in different figures may be represented by the same number. For clarity, not every object may be labeled in every drawing. In the drawings: Fig. Figure 1 shows a perspective view of a robot according to one embodiment; Fig. 2 shows a top view of the robot. Fig. 1; Fig. Figure 3 shows a top view of a plurality of robots according to an embodiment, arranged in a flexible working environment; Fig. Figure 4 shows a lower perspective view of a robot arm according to one embodiment; Fig. Figure 5 shows an upper perspective view of the robot arm. Fig. 4; Fig. Figure 6 shows a simplified representation of a method for determining a three-dimensional position of a target according to one embodiment; and Fig. Figure 7 shows a method for operating a robot according to one embodiment. DETAILED DESCRIPTION

[0007] It is understood that the features of the invention are described here with reference to the figures, which show illustrative embodiments. The illustrative embodiments described here are not necessarily intended to show all embodiments containing all features of the invention, but rather serve to describe some illustrative embodiments and selected combinations of features of the invention. Therefore, the features of the invention should not be interpreted narrowly in light of the illustrative embodiments. Furthermore, it should be clear that features of the invention can be used alone or in any suitable combination with other features of the invention.

[0008] Robots are frequently used to perform tasks in collaboration with humans in a work environment. In such an environment, both robots and humans can perform various tasks on objects to achieve desired results (e.g., producing goods, obtaining test results, etc.). Typically, these robots are taught by human operators the picking and dropping positions of an object, as well as any other movements required to perform tasks with that object (e.g., soldering components onto a circuit board, mixing ingredients in a test tube, etc.), and then configured to repeat these movements with subsequent objects.

[0009] The inventors recognized an advantage of a robot configured to automatically adapt to changes in the work environment. For example, employees might frequently want to physically reposition the robot within a work area, depending on the number of available robots or human workers, without having to manually retrain the robot's pick-up and drop-off positions. Furthermore, human workers might place objects inaccurately, and it may be desirable for the robot to be configured to compensate for this inaccuracy.

[0010] The inventors also recognized an advantage of a robot configured to automatically perform tasks on objects without requiring additional input from employees. Such a configuration can improve efficiency and avoid downtime (e.g., the robot can begin working on objects as soon as they are placed in the workspace, instead of waiting for input commands from employees).

[0011] The inventors therefore recognized an advantage of a robot that can be configured to detect the presence of objects and / or other components (e.g., holding stations, processing stations, workbenches, etc.) and determine the position of the objects / other components relative to an end effector of the robot. The robot can then be configured to perform tasks with the objects (e.g., moving objects from holding stations to processing stations, or vice versa, performing work on the objects at the processing stations, etc.). Such a configuration can allow human employees to quickly reconfigure the work environment without having to manually teach the robot the positions of objects and / or other components.Such a configuration can also allow employees to collaborate with the robot without requiring a high degree of precision from them, and / or enable the robot to adapt to different conditions, such as object holders and objects with variable formats. Such a configuration can also enable the automated execution of tasks with minimal effort from employees.

[0012] In some embodiments, the robot may include a first imaging system and a second imaging system. In some embodiments, each imaging system is configured to be used by a control unit to independently control the robot's movement. The first imaging system may include a first imager (e.g., a camera and an image processing module such as image analysis software and associated hardware), and the second imaging system may include a second imager. The control unit may be configured to control either the first or the second imager to map the workspace and determine the presence of objects and / or other components within the workspace. When an object and / or other component is detected, the control unit may then control either the first or the second imager to generate one or more images (e.g.,The system obtains stereoscopic images of a target located in a predetermined position relative to the object / other component (e.g., positioned on or next to the object / component). The control unit can then use these stereoscopic images to determine the three-dimensional position of the target relative to the end effector and / or other parts of the robot. Based on the determined position of the end effector relative to the target and the predetermined position of the object / component relative to the target, the control unit can then determine the position of the objects / other components relative to the end effector. In some embodiments, the presence of an object / component can be detected using one imaging system, while the three-dimensional position of a target positioned on the object can be determined using the other imaging system.

[0013] One possible use case is as follows: Operators can position a workstation within the robot's workspace. The robot can detect the presence of the workstation using its first or second image sensor and determine its position using the same sensor. Operators can then position a first holding station, containing an initial set of one or more objects, at any location within the robot's workspace. The robot can detect the presence of the first holding station and the initial set of objects belonging to it using its first or second image sensor. The robot can then determine the position of the first holding station and the initial set of objects relative to its end effector using the same sensor.The robot can move an object to the processing station, perform tasks on the object, and then move the object back to the first holding station. The robot can repeat this process for each object in the first holding station. While the robot is performing tasks on the first set of objects, employees can position a second holding station within the robot's workspace. The robot can detect the presence / position of the second holding station / second set of objects using the first and / or second image sensor and automatically begin performing tasks on the second set of objects as soon as the tasks on the first set are completed. Employees can then move the first holding station and the first set of objects to another location for further processing, packaging, etc.

[0014] As mentioned above, the inventors also recognized the potential benefits of the robot collaborating with humans. Such collaboration might require human workers to enter the robot's workspace to perform tasks. The inventors identified an advantage in configuring the robot to retract, allowing an external force applied by a person to stop or move the robot arm out of its controlled path. This retractability enables human workers to move the arm out of the way without damaging it. It also allows the arm to stop if it comes into contact with a person or other component within the workspace, thus limiting potential harm to people, the robot, and / or other components.In some embodiments, this retractability can also be used to ensure the correct positioning of objects. For example, contact between an object and an object receptacle (e.g., a slot in a holding station for receiving an object) can be used to guide the object into an object receptacle in the holding station during a coupling process.

[0015] The inventors also recognized an advantage in configuring the robot to perform tasks in the life sciences. Therefore, in some embodiments, the objects can be sample holders configured to contain material for laboratory analysis. Such a sample holder can be a life science sample tray, a life science microplate, a single life science sample tube held in a tray or microplate, or any part of a life science laboratory device used to hold any type of sample, including liquid and / or solid materials.

[0016] The Fig. 1 and Fig. Figure 2 shows a perspective view or a top view of an illustrative robot 100, which has one or more features according to the invention. The robot can comprise a base 102, an articulated robot arm 104, and an end effector 108 arranged at one end of the articulated robot arm. The robot 100 can include a control unit 120 configured to control one or more drive motors 132 in the robot arm 104 to move the end effector 108 within the working area 110. The robot can be configured to move objects 200 between different stations and / or to perform operations on objects 200. Such stations can include holding stations 202 configured to hold a plurality of objects 200 in containers 208, and processing stations 204 where operations are performed on the objects 200.In some embodiments, the robot 100 can be configured to perform various operations on the objects 200 when they are placed at the processing station 204. Alternatively or additionally, a human operator or another robot / machine can perform operations on the objects at the processing station 204.

[0017] The robot arm 104 can be any suitable type of robot arm, such as a gantry robot, a SCARA robot, a telescopic or sliding arm robot, etc. In some embodiments, as in Fig. As shown in Figure 2, the machining station 204 and the robot arm 104 can be mounted on the same base 102, so that the machining station 204 is always in a predetermined position relative to the base 102. In other embodiments, the machining station 204 is mounted on a separate base.

[0018] As explained above, it may be desirable for the robot to be configured to automatically detect the presence of objects 200 and / or other components (e.g., holding stations 202, processing stations 204, etc.) and to automatically determine the position of the end effector 108 of the robot 100 relative to the objects / other components. The robot 100 can therefore include a first imaging system 112 with a first imager 114 and a second imaging system 116 with a second imager 118. The robot 100 can also include a control unit 120 configured to control the first and second imaging systems 112 and 116, respectively, and to use information from either the first or second imaging system 112 or 116 to control the movement of the robot 100.

[0019] The control unit 120 can be configured to control the imaging systems 112 and 116 to detect the presence of objects and / or other components in the robot's workspace 110 by mapping the workspace 110 with the imagers 114 and 118. The imagers 114 and 118 can be positioned anywhere that allows for mapping the workspace 110. Further details regarding the location of the imagers 114 and 118 are explained below.

[0020] Once the presence of an object / component is detected, the control unit 120 can be configured to use the imaging systems 112, 116 to determine the relative positioning of the end effector 108 with respect to the objects / other components. Each object / component can be associated with at least one target 206, which is located at a predetermined position relative to the object / component. For example, in some embodiments, each object 200 can include a target 206 located on the object 200. In some embodiments, a target 206 can be located adjacent to the object / component (e.g., on a recording 208). In some embodiments, a target 206 can be located at stations 202 and / or 204, and objects 200 can be positioned at predetermined positions in stations 202 / 204 relative to the target 206.For example, in some embodiments, holding stations 202 include targets 206 arranged on a section of the holding stations 202 and include containers 208 configured to hold objects 200 at predetermined positions relative to the targets 206. The control unit 120 can control the imaging systems 112, 116 to use the imagers 114, 118 to obtain stereo images of the target 206 using a suitable stereo imaging process, as explained in more detail below. The control unit 120 can use these stereo images to determine the three-dimensional position of the target 206 relative to the end effector 108. The control unit 120 can then determine the three-dimensional position of the object / component relative to the end effector 108 based on the determined position of the end effector relative to the target and based on the predetermined position of the target relative to the object / component.In some embodiments, a single target 206 is used to determine the position of the end effector 108 relative to the objects 200 / other components. In some embodiments, multiple targets 206 or sets of targets 206 are used to determine the position of the end effector 108 relative to the objects / other components.

[0021] In some embodiments, the imaging systems 112, 116 can be used independently of each other by the control unit 120 to control the movement of the robot 100. For example, the imaging systems 112, 116 can have non-overlapping fields of view, and each imaging system 112, 116 can be used alone to determine the position of an object relative to the robot and to control the subsequent movement of the robot relative to the object. For example, imaging system 112 can be used to determine the position of the end effector 108 relative to objects within its field of view (e.g., using one or more sets of stereo images acquired by the first imager 114), and imaging system 116 can be used to determine the position of the end effector 108 relative to objects within its field of view (e.g., using one or more sets of stereo images acquired by the first imager 114).(using one or more sets of stereo images acquired by the second image transmitter 118). Such arrangements can offer advantages in some robot and imaging configurations, for example, when the first and second image transmitters 114, 118 are mounted on an end effector 108 and the end effector 108 has limited degrees of freedom, thus also limiting how the first and second image transmitters 114, 118 can be oriented. For example, in some configurations, a first image transmitter 114 may only have a horizontal field of view due to movement restrictions of the end effector, and a second image transmitter 118 may also only have a vertically oriented (e.g., downward-facing) field of view due to limitations in the movement of the end effector and / or its degrees of freedom.However, since the first and second image sensors 114, 118 can be operated in such a way that they jointly determine the position of all objects in the robot's workspace, the robot can be controlled to move relative to objects in a wide range of positions. Such arrangements can also allow the control unit 120 and one of the image systems 112, 116 to acquire stereo images while the robot 100 is being moved for other tasks, thus avoiding a dedicated movement of the end effector solely for the purpose of determining the position of an object. For example, while the robot is being moved based on position detection and control using information from the second image system 116, the first image system 112 can be used to acquire stereo images of another object in the workspace, so that the control unit 120 can determine the object's position relative to the robot (e.g.,the end effector 108) can determine and use this information to control the robot's movement later, once the current task is completed. This can enable more efficient robot operation, as the robot's movement during a task can be used to acquire stereo images of an object in order to plan future robot movements, instead of requiring movement of the end effector 108 solely for the purpose of object detection and position determination relative to the robot.

[0022] In some embodiments, the imaging systems 112 and 116 can be integrated into the robot, attached to an end effector 108, or to another robot section. In some embodiments, the imaging systems 112 and 116 can be selectively attached / removable, so that the imaging systems 112 and 116 can be used with other robots.

[0023] As explained above, it may be desirable for the robot 100 to perform tasks on objects 200 located within its working area 110 without requiring additional input from human operators, thereby reducing robot downtime and improving overall efficiency. Once the position of the objects / other components relative to the end effector 108 is determined, the control unit 120 can therefore automatically control the robot 100's drive motor(s) 132 to perform various tasks on the objects 200. In some embodiments, for example, the end effector may include a gripper 122 configured to grasp the objects 200 and move them between the holding stations 202 and the processing stations 204. In some embodiments, the gripper 122 may also perform operations on the objects 200 at the processing stations 204.In some embodiments, the robot 100 can be equipped with a tool changer 124, and the robot 100 can be configured to move an object 200 with the gripper 122 to the processing station 204, then move to the tool changer 124 and exchange the gripper 122 for another tool (e.g., a stirring tool, a soldering tool, etc.) to perform operations on the object 200. In some embodiments, the processing station 204 itself can be configured to perform various operations on the object 200. For example, the processing station 204 can include a centrifuge configured to rotate the object 200.

[0024] In some embodiments, the robot 100 can be configured to perform operations only on an object 200 that is placed at a processing station 204. For example, a human operator 400 or another robot can place the object 200 at the processing station, whereupon the robot can detect the position and determine the three-dimensional position of the object 200 relative to the end effector and perform operations on the object 200. A human operator 400 or another robot can then remove the object 200 from the processing station 204.

[0025] As explained above, it may be desirable for humans to collaborate with the robot. Such collaboration may require humans to enter work area 110. Therefore, as explained above, it may be desirable for the robot 100 to be able to be retracted to limit injuries to humans and damage to the robot 100 in the event of collisions between human employees and the robot 100.

[0026] As used here, the term "reversed" or "return" refers to the ability for an external force (e.g., from a person or other object / obstacle physically attacking / touching the arm) exerted on the robot arm 104 to overcome the drive motor(s) 132 in the robot arm 104. This external force can prevent the robot arm 104 from moving along its controlled path and / or alter the path of the robot arm 104. Such a reversing capability can enable the robot arm 104 to stop its movement if it touches a person or other obstacle along the controlled path of the robot arm. It can also enable people to manually alter the path of the robot arm 104 if necessary to avoid injury and / or damage.

[0027] In some embodiments, the inventors have also recognized an advantage in using the robot arm's retraction capability to ensure a proper fit between the objects 200 and the object containers 208 in the holding stations 202 during a coupling process. Contact between the object 200 and the sides of the object containers 208 can cause the robot arm 104 to retract, guiding the object 200 into and out of the object container 208. Such a configuration can allow for correction of the robot arm's movement if necessary due to errors (e.g., objects or object containers outside tolerance, inaccurately placed targets, errors in relative position determination, station displacement during operation, etc.).

[0028] In some embodiments, the robot arm 104 can include any suitable feedback sensors 134. For example, the robot arm 104 can include position sensors to determine and / or verify the positions of each arm segment relative to other arm segments. The robot arm can also include force sensors configured to detect the application of an external force to the robot arm. In some embodiments, the control unit 120 can be configured to reduce / remove the power of the drive motor(s) 132 in response to the application of an external force to the robot arm (e.g., when a person pushes on the arm). In some embodiments, the objects 200 can be sample holders containing materials for laboratory analysis.Such sample holders can be life science sample trays, life science microplates, individual life science sample tubes held in a tray or microplate; or any part of a life science laboratory device. The robot can be configured to perform any suitable tasks or combinations of tasks on the sample holders, such as mixing materials within the sample holders, transporting sample holders between processing stations so that humans or other personnel can work on them, measuring parameters of the material within the sample holders, and / or any other tasks or combinations of tasks.

[0029] While the embodiment described above discloses a robot arm performing tasks on a sample holder in a life sciences environment, it is envisioned that the robot 100 can be configured to operate in any suitable work environment and perform tasks on any suitable object. For example, the robot can be used in a warehouse environment, a manufacturing plant (e.g., automotive, aerospace, electronics, semiconductor, etc.), or any other suitable environment. The objects 200 can be printed circuit boards, goods / containers in warehouses, automotive parts, aerospace parts, electronic components, semiconductor wafers, or other suitable objects. The robot can be configured to perform any suitable tasks on such objects, such as milling, welding, soldering, etc.

[0030] Fig. Figure 3 shows a top view of a flexible work environment comprising one or more features of the invention. As explained above, in some embodiments it may be desirable for the robot 100 to operate in a flexible work environment. For example, more human employees 400 may be available on one day to perform a particular task than on another day. Alternatively, it may be desirable for the robot to switch between different task groups depending on current needs, with such different tasks requiring different positioning of the robot 100 and various objects / other components. Therefore, in some embodiments, the base 102 of the robot 100 may be configured as a movable base (e.g., with wheels) so that the robot 100 can be moved across a floor or other surface.Various holding stations 202, processing stations 204, and workbenches 210 can also be moved. Human employees 400 can position the robot 100 and various stations in any layout and instruct the robot to perform a series of defined tasks. The robot can then use the imaging systems to detect the presence and position of the various stations 202, 204, and / or 210 and all objects 200 arranged in the stations, and can then automatically perform the tasks at each of the objects 200. The layouts of the various robots and stations can then be changed as needed to adapt them to the specific task. As in . Fig. As can be seen in Figure 3, for example, a first robot 100A can be configured to perform task A, and a second robot 100B can be configured to perform task B.

[0031] To perform task A, robot 100A can be configured to move objects from holding stations 202A to a processing station 204A, perform operations on the objects (with or without using a tool changer), and move the object back to holding station 202A. Robot 100A can be configured to map its workspace 110A to detect the presence of holding stations 202A with unprocessed objects 200A, determine the position of the objects 200A relative to the end effector, and perform the defined operations on each object 200A. Human workers 400 can then simply roll or otherwise move the holding stations 202A into workspace 110A, and robot 100A can detect the objects in holding station 202A, determine the position(s) of one or more objects, and perform the tasks on those one or more objects.Human employees 400 can then remove the holding stations 202A containing the processed objects 200A and place new holding stations 202A containing unprocessed objects. In some embodiments, the human employees 400 can place the holding stations 202A at any point within the robot's work area 110A.

[0032] To perform task B, robot B can be configured to move objects 200B from holding stations 202B to pick-up / placement locations 212B, which are arranged on workbenches 210B. Human workers 400 can then pick up the objects 200B from the pick-up / placement locations, perform operations on the objects 200B, and place them back at the pick-up / placement locations 212B. Robot 100B can be configured to move the processed object 200B back to the holding stations 202B. In some embodiments, the human worker can signal that the object 200B is ready to be returned to the holding station 202B (e.g., by pressing a button).In some embodiments, the robot 100B can be configured to automatically detect when an object 200B has been returned to the pick-up / placement location 212B and return the object to the holding station 202B when the return is detected. In some embodiments, the objects 200B can include targets configured to be imaged by the first and / or second imaging system of the robot 100B so that the robot 100B can determine the position of the object 200B relative to the end effector 108B (e.g., if human operators inaccurately place 400 objects 200B at the pick-up / placement locations 212B).

[0033] If there is no or only a low demand for task B, robot 100B can be repositioned so that at least one processing station 204A and at least one holding station 206A are arranged around robot 100B, and robot 100B can be instructed to perform task A. Robot B can then detect the presence and positions of stations / other components related to task A within its workspace and perform task A on those objects. While the above illustrates only two robots performing two different tasks, any number of robots performing any number of tasks is conceivable.Each robot 100 can therefore be configured to “position and execute”, whereby each robot 100 can be repositioned and controlled to perform different tasks as needed, without having to manually reteach the robot different positions of objects / stations.

[0034] The Fig. 4 and Fig. Figure 5 shows an upper perspective view and a lower perspective view, respectively, of a representative robot 100 that has one or more features according to the invention. As mentioned above, the image sensors of the first and second imaging systems 112 and 116 can be arranged at any suitable location that allows for the proper operation of the imaging systems 112 and 116. In some embodiments, for example, the first image sensor is separate from the robot arm and arranged outside the work area (e.g., above the work area, on one side of the work area, etc.). In some embodiments, as shown in Fig. 4 and Fig. As can be seen in Figure 5, the first image sensor 114 is arranged at the end effector 108, e.g., with a horizontally oriented field of view for imaging. Such a forward-facing first image sensor 114 can enable presence detection and / or three-dimensional position determination, even when an object 200 is grasped by the grippers 122, as shown in Figure 5. Fig. Figure 5 shows that such a configuration might be desirable, for example, if employees were to move a holding station 202 before the robot 100 has placed the object 200 into the receptacle 208. In such a scenario, the control unit 102 could control the first imaging system 112 to detect the absence of the holding station and take corrective action (e.g., interrupt operation until the holding station 202 is reattached, alert human employees, etc.), instead of simply placing the object 200 where the receptacle 208 "should" be, thereby causing the object 200 to fall to the floor.

[0035] If the first image sensor 114 is located on the end effector 108, its field of view may not be sufficient to view the entire work area 110 at once. Therefore, when imaging the work area 110 to detect the presence of objects / other components, the control unit 120 can control the robot arm 104 to move the end effector 108 to different positions within the work area 110 so that the first image sensor 114 can view the entire work area 110. For example, the end effector can be rotated 360° about a vertical axis V of the end effector so that a forward-facing first image sensor can view the entire work area.

[0036] In some embodiments, such as in Fig. As shown in Figure 4, the second image sensor 118 can be arranged on a bottom side of the end effector 108, so that the second image sensor 118 can acquire images in a direction approximately orthogonal to the direction in which the first image sensor 114 is configured to acquire images. Such a configuration can enable the second image sensor 114 to acquire stereo images of targets 206H arranged on horizontal surfaces, for example, on the tops of the object 200, as shown in Figure 4. Fig. 5 shown.

[0037] As explained above, in some embodiments the control unit 120 can be configured to detect the presence of objects / components in the work area 110 and determine the three-dimensional position of the objects / components relative to the end effector 108 using either the first imaging system 112 or the second imaging system 116. Such a configuration may be desirable, for example, when the image sensors 114 and 118 are mounted on the end effector of a SCARA robot arm, since the end effector of a SCARA robot arm can only rotate about the vertical axis V and therefore cannot be tilted about a horizontal axis.In such a configuration, the image sensor 114 of the first imaging system 112 can be located on the front of the end effector and configured to acquire images in a horizontal direction, and the image sensor 118 of the second imaging system can be located on the underside of the end effector and configured to acquire images in a vertical direction, so that the image sensors of the first and second imaging systems have non-overlapping fields of view. The control unit 120 can control the first imaging system to acquire three-dimensional positions of objects / components with targets arranged on vertical surfaces (e.g., target 206V in ). Fig. 4 and Fig. 5) is detected and determined, and the control unit can control the second imaging system to determine three-dimensional positions of objects / components with targets arranged on horizontal surfaces (e.g., target 206H in Fig. 5) recorded and determined.

[0038] Such a configuration can enable presence detection and relative position determination even when an object is being manipulated by the end effector 108. As explained above, for example, placing the first imager 114 on the front of the end effector 108 allows the first imager to image the work area to perform presence detection, even when the robot 100 is grasping an object. If the first imaging system 112 is also configured for relative position determination, the control unit 120 can be configured to detect whether the object holding station 202 has been moved (e.g.,(initiated by an employee), and receives stereo images of vertical targets 206V from the first image transmitter 114 to determine the new position of the holding station 202 and / or the recordings 208 relative to the end effector, and controls the robot arm 104 to place the object into the newly positioned object recording 208.

[0039] The control unit can use any suitable technique to obtain stereo images of target 200 and to determine the position of target 200 relative to the end effector. For example, in some embodiments, such as in Fig. 6. The position of the image transmitter 114 / 118 can be controlled to obtain the stereo images. The image transmitter 114 / 118 can be moved to a first position 126 and controlled to obtain a first image of the target 200, and then moved to a second position 128 to obtain a second image of the target 200. The control unit can use the stereo images to determine the position of the target 206 relative to the end effector 108 using any suitable technique. As shown in Fig. As shown in Figure 6, the control unit 120 can, for example, identify a two-dimensional pixel value CX1, CY1 of a section 214 of the target 200 (e.g., a corner of the target 200) in the first image and a two-dimensional pixel value CX2, CY2 of the same section 214 of the target 200 in the second image using a suitable image analysis technique. The control unit can then use a stereoscopic imager model to estimate the three-dimensional position of the target 200 relative to the imager 114 / 118 based on the differences between CX1 and CX2, as well as the differences between CY1 and CY2. The control unit can then determine the three-dimensional position of the end effector relative to the object / component based on the estimated position of the imager 114 / 118 relative to the target 200, a predetermined position of the target relative to the object / component, and a predetermined position of the end effector relative to the imager 116 / 118.

[0040] While the above embodiments disclose a first and a second imaging system with a single image sensor, in some embodiments the first imaging system 112 and / or the second imaging system 114 may each comprise a pair of image sensors. One image sensor of the pair of image sensors may be located at a first position 126, and the other image sensor of the pair of image sensors may be located at a second position 128. In some embodiments, the second imaging system 116 comprises the second image sensor located at the first position 126 and a third image sensor located adjacent to the second image sensor on the end effector at the second position 128. The second and third image sensors may be controlled to image the target simultaneously from positions 126 and 128 to obtain the stereo images for relative position determination, in order to obtain the set of stereo images 127 and 129.

[0041] While the embodiments described above reveal a robot with multiple imaging systems, in some embodiments the robot may include a single imaging system, and the control unit may be configured to control this single imaging system to detect the presence of objects / components in the workspace and determine the position of the end effector relative to the objects / components using the techniques described above. Such a single imaging system may be used, for example, in a gantry robot.

[0042] While the embodiment described above discloses a robot arm performing tasks on a sample holder in a life sciences environment, it is envisaged that the robot 100 can be configured to operate in any suitable work environment and perform tasks on any suitable object. For example, the robot can be used in a warehouse environment, a manufacturing plant (e.g., automotive, aerospace, electronics, semiconductor, etc.), or any other suitable environment. The objects 200 can be printed circuit boards, goods / containers in a warehouse, automotive parts, aerospace parts, electronic components, semiconductor wafers, or other suitable objects. The robot can be configured to perform suitable tasks on such objects, such as milling, welding, soldering, etc.

[0043] Fig.Figure 7 shows a method 300 for operating a robot, which includes one or more features according to the invention. In Figure 302, the robot can detect the presence of an object in the robot's workspace. Such presence detection can be performed using any suitable technique, for example, via the imaging system(s) described above. In Figure 304, the robot can acquire stereo images of a target in the workspace. Such stereo images can be obtained using the same imaging system as the imaging system used for presence detection, a separate imaging system as described above, or any other suitable technique.When an imaging system is used to obtain stereo images, the imaging system can obtain the stereo images by imaging the target from multiple positions with a single imager, or it can acquire the stereo images simultaneously with multiple imagers. The target can be related to the object (e.g., located on the object, located on a container holding the object, located on a holding station and / or processing station, etc.). In Figure 306, the robot can use the stereo images to determine the position of the robot's end effector relative to the target. In Figure 308, the robot can determine the position of the end effector relative to the object. In Figure 310, the robot can perform tasks on the object. Such tasks can include grasping the object with a gripper located on the end effector, manipulating the object's position, and performing operations on the object (e.g.,Soldering, heating, stirring, performing tests, etc.) or any other suitable task. In the 312, the robot's arm can be configured to retract. This retractability can be used for any suitable purpose, such as allowing human operators to change the controlled path of the robot arm or ensuring a proper fit between the object and a receptacle for holding the object.

[0044] Although the present teachings have been described in connection with various embodiments and examples, they are not intended to be limited to such embodiments or examples. On the contrary, the present teachings include various alternatives, modifications, and equivalents as are known to those skilled in the art in this field. Accordingly, the foregoing description and the drawings serve only as examples.

Claims

[1] Robot, exhibiting: an articulated robot arm configured to operate within a work area of ​​the robot arm; an end effector arranged on the robot arm, wherein the end effector has a gripper configured to grasp an object; a first imaging system comprising a first imager positioned at the end effector and configured to capture images in a first direction; a second imaging system comprising a second image sensor located at the end effector and configured to acquire images in a second direction orthogonal to the first direction; and a control unit configured to: Control either the first image sensor or the second image sensor to obtain a set of stereo images of a target positioned in the workspace; Determining the position of the end effector relative to the target based on images of the target; Determining a position of the end effector relative to the object based on the determined position of the end effector relative to the target and a position of the target relative to the object; and Controlling robot movement based on the determined position of the end effector relative to the object. [2] Robot according to claim 1, wherein the object is one of the following: a sample holder, a life science sample tray, a life science microplate, a single life science sample tube held in a tray or microplate, part of a life science laboratory device or a printed circuit board assembly. [3] Robot according to claim 1, wherein the first direction is a horizontal direction and the second direction is a vertical direction. [4] Robot according to claim 1, wherein the first image sensor or the second image sensor is arranged between gripper components of the end effector. [5] Robot according to claim 1, wherein the work area has a holding station and a processing station and the robot is configured to move the object between the holding station and the processing station. [6] Robot according to claim 5, further comprising a movable base configured for movement along a floor or other surface, wherein the robot arm is arranged at the base. [7] Robot according to claim 6, wherein the holding station and / or the processing station are configured to be movable relative to the base. [8] Robot according to claim 5, further comprising a container arranged at the holding station and configured to receive the object, wherein the robot is configured to remove the object from the container and place the object into the container, wherein the robot arm is configured to retract by a force resulting from the contact between the object and the container when the robot removes the object from the container or places it into the container. [9] Robot according to claim 8, wherein the container is a first container from a plurality of containers arranged at the holding station, and wherein each container is configured to receive an object in a displaceable manner. [10] Robot according to claim 1, wherein the images used to determine the position of the end effector relative to the target comprise only the set of stereo images obtained from the first image sensor or the second image sensor, and the control of the robot movement is carried out using images obtained only from the image sensor that has received the set of stereo images. [11] Robot according to claim 1, wherein the first image sensor has only a horizontal field of view due to movement limitations of the end effector and the second image sensor has only a vertical field of view due to movement limitations of the end effector. [12] Robot according to claim 1, wherein the first image sensor and / or the second image sensor each comprise a single camera. [13] Robot according to claim 12, wherein the first image sensor and / or the second image sensor are configured to obtain stereo images of the target by imaging the target from multiple positions. [14] Robot according to claim 1, wherein the first and second image sensors have non-overlapping fields of view. [15] Robot according to claim 1, wherein the target is arranged on the object. [16] Robot according to claim 1, wherein the robot is configured to retract by a force applied to the robot arm by a human. [17] Robot according to claim 1, wherein the robot arm is configured such that the end effector is not tiltable about a horizontal axis, and wherein the first image sensor is arranged on a front side of the end effector to capture images in a horizontal direction, and the second image sensor is arranged on a bottom side of the end effector to capture images in a vertical direction. [18] Robot according to claim 1, wherein the first image sensor and the second image sensor are configured to acquire images for relative position determination while the gripper holds an object. [19] Robot according to claim 1, wherein the robot is a SCARA robot, wherein the robot arm comprises a first arm which is pivotable about a first vertical axis relative to a base, and a second arm which is coupled to the first arm and is pivotable about a second vertical axis relative to the first arm. [20] Robot according to claim 19, wherein the end effector is coupled to the second arm, is pivotable about a vertical axis and is not tiltable about a horizontal axis. [21] Robots, exhibiting: an articulated robot arm configured to operate within a work area of ​​the robot arm; an end effector arranged on the robot arm, wherein the end effector has a gripper configured to grasp an object; a first imaging system comprising a first imager positioned at the end effector and configured to capture images in a first direction; a second imaging system comprising a second image sensor located at the end effector and configured to acquire images in a vertical direction orthogonal to the horizontal direction; and a control unit configured to control robot movement based on the position of the end effector relative to an object, determined using images received from the first and / or second image sensor. [22] Robot according to claim 21, wherein the object is one of the following: a sample holder, a life science sample tray, a life science microplate, a single life science sample tube held in a tray or microplate, part of a life science laboratory device or a printed circuit board assembly. [23] Robot according to claim 21, wherein the first or second image sensor is arranged between gripper components of the end effector. [24] Robot according to claim 21, wherein the robot arm is configured to retract by a force applied to the robot arm and / or an object held by the gripper. [25] Robot according to claim 21, wherein the first image sensor has only a horizontal field of view due to movement limitations of the end effector and the second image sensor has only a vertical field of view due to movement limitations of the end effector. [26] Robot according to claim 21, wherein the first image sensor and / or the second image sensor each comprise a single camera. [27] Robot according to claim 21, wherein the first image sensor and / or the second image sensor are configured to obtain stereo images of a target by imaging the target from multiple positions, and wherein the control unit is configured to determine the position of the end effector relative to the object based on the stereo images of the target and a position of the target relative to the object. [28] Robot according to claim 27, wherein the target is arranged on the object. [29] Robot according to claim 21, wherein the first and second image sensors have non-overlapping fields of view. [30] Robot according to claim 21, wherein the robot arm is configured such that the end effector is not tiltable about a horizontal axis, and wherein the first image sensor is arranged on a front side of the end effector to capture images in a horizontal direction, and the second image sensor is arranged on a bottom side of the end effector to capture images in a vertical direction. [31] Robot according to claim 21, wherein the first image sensor and the second image sensor are configured to acquire images for determining the position of the end effector while the gripper holds an object. [32] Robot according to claim 21, wherein the robot is a SCARA robot, wherein the robot arm has a first arm which is pivotable about a first vertical axis relative to a base, and a second arm which is coupled to the first arm and is pivotable about a second vertical axis relative to the first arm. [33] Robot according to claim 32, wherein the end effector is coupled to the second arm, is pivotable about a vertical axis and is not tiltable about a horizontal axis. [34] Robot according to claim 21, wherein the control unit is configured to control the robot movement on the basis of a position of the end effector which is determined relative to an object only by using images taken by the first or second image sensor.

Images