Aligning a robotic arm to an object

Abstract

An example robotic system includes a robotic arm configured to move in multiple degrees of freedom and a control system including one or more processing devices. The one or more processing devices are programmed to perform operations including: identifying an object in the environment accessible to the robotic arm based on sensor data indicative of the environment; determining that a component associated with the robotic arm is within a predefined distance of the object; and controlling the robotic arm to move the component toward or into alignment with the object in response to the component being within the predefined distance of the object.

Description

[0001]ALIGNING A ROBOTIC ARM TO AN OBJECT TECHNICAL FIELD This specification describes examples of systems and processesfor aligning a robotic arm to an object.BACKGROUND A robot, such as a robotic arm, is configured to control an end effector to interact with the environment. An example end effector is anaccessory or tool that the robot uses to perform an operation.An example robotic arm is a computer-controlled robot that iscapable of moving in multiple degrees of freedom. The robotic arm may besupported by a base and may include one or more links interconnected by joints.The joints may be configured to support rotational motion and / or translationaldisplacement relative to the base. A tool flange may be on the opposite end ofthe robotic arm from the base. The tool flange contains an end effector interface.The end effector interface enables an accessory to connect to the robotic arm.In an example operation, the joints are controlled to position the robotic arm toenable the accessory to implement a predefined operation. For instance, if theaccessory is a welding tool, the joints may be controlled to position, andthereafter to reposition, the robotic arm so that the welding tool is at successivelocations where welding is to be performed on a workpiece.SUMMARY An example robotic system includes a robotic arm configured tomove in multiple degrees of freedom and a control system including one or moreprocessing devices. The one or more processing devices are programmed toperform operations including: identifying an object in an environmentaccessible to the robotic arm based on sensor data indicative of the environment;determining that a component associated with the robotic arm is within apredefined distance of the object; and controlling the robotic arm to move thecomponent toward or into alignment with the object in response to thecomponent being within the predefined distance of the object. The environmentaccessible to the robotic arm can be any space, area or region within the reachof the robotic arm. It is to be understood that the sensor data indicative of oneof more properties of the environment can indicate properties of theenvironment in a space, area or region within the reach of the robotic arm andalso comprise properties of the environment in a space, area or region outside the reach of the robotic arm. The example robot system may include one or more of the following features, either alone or in combination. Determining that the component associated with the robotic arm is within the predefined distance of the object may include determining that the component of the robotic arm is within a first distance from the object (e.g., apredefined vicinity of the object). Controlling the robotic arm to move thecomponent toward alignment with the object may be performed in response to the component of the robotic arm being within the first distance from the object. Determining that the component associated with the robotic arm is within the predefined distance of the object may include determining that the component of the robotic arm is within a second distance from the object (e.g., a predefined threshold distance). The second distance may be less than the first distance. Controlling the robotic arm to move the component into alignment with the object may be performed in response to the component of the robotic arm beingwithin the second distance from the object and with greater force than movingthe component toward alignment. Identifying the object may include identifying an axis of the object.The predefined distance may be measured relative to the axis of the object.Controlling the robotic arm to move the component into alignment may includecontrolling the robotic arm to move the component into alignment with the axis.The axis may be along a center of the object. The axis may be along a part ofthe object. The operations may include, following alignment of the component with the axis, constraining movement of at least part of the robotic arm to bealong the axis. The operations may include, following alignment of thecomponent with the axis, constraining movement of at least part of the robotic arm relative to the axis. The operations may include, following controlling the robotic arm to move the component into alignment with the object, enabling manual movement of at least part of robotic along the axis to allow the robotic arm to interact with the object; recording movements of the robotic arm interacting with the object; translating the movements into robot code; and storing the robot code in memory on the control system. The operations may include recording operational parameters of the robotic system; translating theoperational parameters into robot code; and storing the robot code in memoryon the control system. The operational parameters may relate to one or moreof the following: input / output ports in the robotic system or an end effector ortool connected to the robotic arm. The operations may include, prior to controlling the robotic arm to move the component into alignment, controlling the robotic arm to enable manual movement of the robotic arm in multiple degrees of freedom. Controlling the robotic arm to move the component into alignment may be performed automatically absent manual intervention. Controlling the roboticarm to move the component into alignment may be performed in combinationwith manual movement of the robotic arm. The component may include a tool connected to the robotic arm.The component may include a part of the robotic arm.The robotic system may include a vision system associated withthe robotic arm to capture the sensor data. The vision system may include oneor more cameras and / or other sensors mounted to the robotic arm. Theoperations may include receiving the sensor data electronically.The operations may include enabling the robotic arm to be movedout of the predefined distance during alignment in response to a predeterminedamount of manual force. The environment may contain multiple objects. Each of themultiple objects may be a candidate for alignment with the component. Eachof the multiple objects may be at a different distance from the component. Theobject to which the component is configured to align may be a closest one ofthe multiple objects to the component. An example method of controlling a robotic arm includes obtainingsensor data indicative of an environment accessible to the robotic arm;identifying an object in the environment based on the sensor data; determiningthat a component associated with the arm is within a predefined distanceof the object; and controlling the robotic arm to move the component toward orinto alignment with the object in response to the component being within thepredefined distance of the object. The environment accessible to the robotic armcan be any space, area or region surrounding the robotic arm within the reach of the robotic arm. It is to be understood that the obtained sensor data indicate one of more properties of the environment in a space, area or region within thereach of the robotic arm and that the sensor date also may indicate propertiesof the environment in a space, area or region outside the reach of the robotic arm. The example method may include one or more of the following features, either alone or in combination. Determining that the component associated with the robotic arm is within the predefined distance of the object may include determining that the component of the robotic arm is within a first distance from the object (e.g., a predefined vicinity of the object). Controlling the robotic arm to move the component toward alignment with the object may be performed in response to the component of the robotic arm being within the first distance from the object. Determining that the component associated with the robotic arm is within the predefined distance of the object may include determining that the component of the robotic arm is within a second distance from the object (e.g., a predefined threshold distance). The second distance may be less than the first distance. Controlling the robotic arm to move the component into alignment with the object may be performed in response to the component of the robotic arm being within the second distance from the object and with greater force than moving the component toward alignment. Identifying the object may include identifying an axis of the object.The predefined distance may be measured relative to the axis of the object.Controlling the robotic arm to move the component into alignment may include controlling the robotic arm to move the component into alignment with the axis.The axis may be along a center of the object. The axis may be along a part ofthe object. The method may include, following alignment of the component with the axis, constraining movement of at least part of the robotic arm to be along the axis. The method may following alignment of the component with the axis; constraining movement of at least part of the robotic arm relative to the axis. The method may include, following controlling the robotic arm to move the component into alignment with the object; enabling manualmovement of at least part of the robotic arm along the axis to allow the roboticarm to interact with the object; recording movements of the robotic arm interacting with the object; translating the movements into robot code; and storing the robot code in memory on the control system. The method may include, recording operational parameters of the robotic system; translating theoperational parameters into robot code; and storing the robot code in memoryon the control system. The operational parameters may relate to one or more of the following: input / output ports in the robotic system, or an end effector or tool connected to the robotic arm. The method may include, prior to controlling the robotic arm to move the component into alignment, controlling the robotic arm to enable manual movement of the robotic arm in multiple degrees of freedom. Controlling the robotic arm to move the component into alignment may be performed automatically absent manual intervention. Controlling the roboticarm to move the component into alignment may be performed in combinationwith manual movement of the robotic arm. The component may include a tool connected to the robotic arm.The component may include a part of the robotic arm.The sensor data may be obtained electronically. The sensor datamay be obtained from a vision system, such as one or more cameras, connectedto the robotic arm. The method may include enabling the robotic arm to be moved outof the predefined distance during alignment in response to a predeterminedamount of manual force. The environment may contain multiple objects. Each of themultiple objects may be a candidate for alignment with the component. Eachof the multiple objects may be at a different distance from the component. Theobject to which the component is configured to align may be a closest one ofthe multiple objects to the component. In an example, one or more non-transitory machine-readablestorage devices store instructions that are executable by one or more processingdevices to control a robotic arm. The instructions are executable to performexample operations that include: obtaining sensor data indicative of anenvironment accessible to the robotic arm; identifying an object in theenvironment based on the sensor data; determining that a componentassociated with the robotic arm is within a predefined distance of the object;and controlling the robotic arm to move the component toward or into alignmentwith the object in response to the component being within the predefineddistance of the object.As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having," "contains," "containing," and any variations thereof, are intended to cover a non-exclusive inclusion, such that robots, systems,techniques, apparatus, structures, or other subject matter described or claimedherein that includes, has, or contains an element or list of elements does notinclude only those elements but can include other elements not expressly listedor inherent to such robots, systems, techniques, apparatus, structures, or othersubject matter described or claimed herein.Any two or more of the features described in this specification, including in this summary section, may be combined to form implementations not specifically described in this specification. At least part of the robots, systems, techniques, apparatus, and / orstructures described in this specification may be configured or controlled byexecuting, on one or more processing devices, machine-executable instructions that are stored on one or more non-transitory machine-readable storage media. Examples of non-transitory machine-readable storage media include read-only memory, an optical disk drive, memory disk drive, and random access memory.The robots, systems, techniques, apparatus, and / or structures described in thisspecification may be configured, for example, through design, construction, composition, arrangement, placement, programming, operation, activation, deactivation, and / or control. The details of one or more implementations are set forth in the accompanying drawings and the description. Other features and advantages will be apparent from the description and drawings, and from the claims. DESCRIPTION OF THE DRAWINGS Fig. 1 is a perspective view of an example system containing an example robot, specifically a robotic arm. Fig. 2 is a flowchart showing example operations included in an example process for aligning a component associated with a robotic arm to an object. Figs. 3, 4, 5, and 6 are block diagrams showing, graphically, operations performed for aligning a component associated with a robotic arm to an example cylindrical object. Fig. 7 is a block diagram showing an example of another object towhich a component associated with a robotic arm may be aligned. Figs. 8, 9, 10, and 11 are block diagrams showing, graphically, operations performed for aligning a component associated with a robotic arm to an example intersection of two surface. Figs. 12, 13, and 14 show examples of other objects to which a component associated with a robotic arm may be aligned. Fig. 15 is a block diagram showing, graphically, operations performed for aligning a component associated with a robotic arm to one ofmultiple example cylindrical objects.Like reference numerals in different figures indicate like elements DETAILED DESCRIPTION Described herein are examples of systems and processes foraligning a component associated with a robotic arm to an object. Thecomponent associated with the robotic arm may be part of the robotic arm itself or an accessory or other device connected or attached to the robotic arm.Example implementations are described in the context of a robotic arm system;however, the systems and processes, and their variants described herein, arenot limited to this context and be used with appropriately movablecomponents associated with of any type of robotic system.Fig. 1 show an example robotic system (“system”) 100 with whichthe systems and processes described herein may be implemented. System 100includes robotic arm (“arm”) 101. Arm 101 includes robot joints (“joints”) 102a,102b, 102c, 102d, 102e, and 102f connecting a robot base (“base”) 103 and arobot tool flange (“tool flange”) 104.In this example, example arm 101 includes seven joints that aremovable or rotatable; however, other implementations of arm 101 may includefewer than seven joints that are movable or rotatable or more than seven jointsthat are movable or rotatable. Arm 101 is thus a seven-axis robot arm havingseven degrees of freedom enabled by the seven joints. The joints in thisexample include the following: base joint 102a configured to rotate around axis105a; shoulder joint 102b configured to rotate around axis 105b; elbow joint 102c configured to rotate around elbow axis 105c; first wrist joint 102dconfigured to rotate around first wrist axis 105d; and second wrist joint 102econfigured to rotate around second wrist axis 105e. As noted, the joints in thisexample also include joint 102f. Joint 102f is a tool joint containing tool flange104 and is configured to rotate around axis 105f. Tool flange 104 is joint thatis configured to rotate around axis 105g. In some implementations one or moreof the above-described axes of rotation can be omitted. For example, rotationaround axis 105d can be omitted, making arm 101 a six-axis robot in thisexample. Arm 101 also includes links 110 and 111. Link 110 is a cylindricaldevice that connects joint 102b to 102c. Link 111 is a cylindrical device thatconnects joint 102c to 102d. Other implementations may include more than, orfewer than, two links and / or links having non-cylindrical shapes.In this example, tool flange 104 is on an opposite end of arm 101 from base 103; however, that need not be the case in all robots. Tool flange104 contains an end effector interface. The end effector interface enables anend effector to connect to arm 101 mechanically and / or electrically. To this end,the end effector interface includes a configuration of mechanical and / orelectrical contacts and / or connection points to which an end effector may mateand thereby attach to arm 101. An example end effector includes a tool or anaccessory, such as those described below, configured to interact with theenvironment. Examples of accessories – for example, end effectors – that maybe connected to the tool flange via the end effector interface include, but arenot limited to, mechanical grippers, vacuum grippers, magnetic grippers, screwing machines, reverse screwing machines, welding equipment, gluing equipment, liquid or solid dispensing systems, painting equipment, visual systems, cameras, scanners, wire holders, tubing holders, belt feeders,polishing equipment, laser-based tools, and / or others not listed here.Arm 101 includes one or more motors and / or actuators (not shown)associated with the tool flange and each joint. The one or more motors oractuators are responsive to control signals that control the amount of torqueprovided to the joints by the motors and / or actuators to cause movement, suchas rotation, of the tool flange and joints, and thus of arm 101. For example,the motors and / or actuators may be configured and controlled to apply torqueto one or more of the joints to control movement of the joints and / or links inorder to move the robot tool flange 104 to a particular pose or location in theenvironment. In some implementations, the motors and / or actuators areconnected to the joints and / or the tool flange via one or more gears and thetorque applied is based on the gear ratio. Arm 101 also includes a vision system 90. Arm 101 is not limitedto use with this type of vision system or to using these specific types of sensors.Vision system may include one or more visual sensors of the same or differenttypes(s), such as one or more three-dimensional (3D) cameras, one or moretwo-dimensional (2D) cameras, and / or one or more scanners, such as one ormore light detection and ranging (LIDAR) scanner(s). In this regard, a 3Dcamera is also referred to as an RGBD camera, where R is for red, G is for green,B is for blue, and D is for depth. The 2D or 3D camera may be configured tocapture information such as video, still images, or both video and still images.In some implementations, the image can be in form of visual information, depthinformation and / or a combination thereof, where visual information is indicativeof visual properties of the environment such a as color information and grayscaleinformation, and the depth is indicative of the 3D depth of theenvironment in the form of point clouds, depth maps, heat maps indicative ofdepth, or combinations thereof. The information obtained by vision system 90may be referred to as sensor data and includes, but is not limited, to the images,visual information, depth information, and other information captured by thevision system described herein.Components of vision system 90 are configured – for example,arranged and / or controllable – to capture sensor data for and / or to detect thepresence of objects in the vision system’s field-of-view (FOV). This FOV maybe based, at least in part, on the orientation of the component(s) of the roboticarm on which the vision system is mounted. In the example of Fig. 1, visionsystem 90 is mounted on joint 102f and configured to have a FOV having acenter at arrow 91, which is parallel to axis 105g. The FOV of the vision systemmay extend 10°, 20°, 30°, 40°, 50°, 60°, 70°, 80°, 90°, or more equally onboth sides of arrow 91 and may increase with distance.In some implementations, the vision system is static in that its components, such as cameras or sensors, move along with movement of the robotic arm but do not move independently of the robotic arm. For example,the components of the vision system are fixedly mounted to point in onedirection, which direction will change based on the position of the component ofrobotic arm 101 on which those components are mounted. In someimplementations, the vision system is dynamic in that its components, such ascameras or sensors, move along with movement of the robotic arm and alsomove independently of the robotic arm. For example, one or more cameras invision system 90 may be controlled to move so that its / their field of view centersaround arrows 91, 92, 93, and / or others (not shown). To do this, one or moreactuators may be controllable to point lenses of corresponding cameras inresponse control signals from the robot controller described below. In someimplementations, the vision system is fixed in the environment of the roboticarm, meaning that the vision system is not on the robotic arm and that its fieldof view is fixed in relation to the environment and does not move along withmovements of the robotic arm. For instance, the vision system can be fixed tomonitor a specified area of the environment around the robot base.In the example of Fig. 1, as previously noted, vision system 90 ismounted on joint 102f, which is a component of robotic arm 101. However, allor part of vision system 90 may be mounted on one or more other componentsof robotic arm 101. For example, all or part of the vision system may bemounted on tool flange 104. All or part of the vision system may be mountedon a link or other joint in the robotic arm, such as joint 102e or link 111. All or part of the vision system may be distributed across multiple links and / or joints.For example, individual cameras and / or scanners may be mounted to two ormore different joints 102f, 102e, and link 111, which differently-mountedcameras and / or scanners together may constitute all or part of the vision system.All or part of the vision system may be external to the robotic arm. For example,individual cameras and / or scanners may be mounted at or on locations in aspace or environment containing the robotic arm but not on the robotic armitself, which cameras and / or scanners together may constitute all or part of thevision system. In some implementations, part of the vision system may be mounted on the robotic arm and part of the vision system may be mounted off of the robotic arm. Sensor data, including data for images captured by the visionsystem, is provided to the robot controller described below. The robot controlleris configured – for example programmed – to use all or some of this data, suchas representing image(s), in the techniques described herein for aligning acomponent associated with the robotic arm to an object.As also shown in Fig. 1, system 100 includes robot controller(“controller”) 110 to control operation of arm 101. Controller 110 may beconfigured to output the control signals described herein to control movement,or restrain movement, of arm 101. Controller 110 may include, for example,one or more microcontrollers, one or more microprocessors, programmablelogic such as a field programmable gate array (FPGA), one or more application-specific integrated circuits (ASICs), solid state circuitry, or any appropriatecombination of two or more of these types of processing devices. In someimplementations, controller 110 may include local components integrated into,or at a same site as, arm 101. In some implementations, controller 110 mayinclude remote components that are in the sense that they are notlocated on, or at a same site as, arm 101. In some implementations, controller110 may include computing resources distributed across a centralized or cloudcomputing service, at least a portion of which is remote from robotic arm 101and / or at least part of which is local. The local components may receiveinstructions to control arm 101 from the remote or distributed components andcontrol the motors and / or actuators accordingly.Controller 110 may be configured to control motion of arm 101 bysending control signals to the motors and / or actuators to control the amount oftorque provided by the motors and / or actuators to the joints. The controlsignals may be based on a dynamic model of arm 101, a direction of gravity,signals from sensors (not shown) connected to or associated with each or someof the joints and / or links in the robotic arm, user-applied force, and / or acomputer program stored in a memory 118 of controller 110. In this regard,the torque output of a motor is the amount of rotational force that the motordevelops. The dynamic model may be stored in memory 118 of controller 110or remotely and may define a relationship between forces acting on arm 101and the velocity, acceleration, or other movement, or lack of movement of arm101 that result(s) from those forces.The dynamic model may include a kinematic model of arm 101,knowledge about inertia of arm 101 and other operational parametersinfluencing the movements of arm 101. The kinematic model may define arelationship between the different parts / components of arm 101 and mayinclude information about arm 101 such as the lengths and / or sizes of the jointsand links. The kinematic model may be described by Denavit-Hartenbergparameters or like. The dynamic model may make it possible for controller 110to determine which torques and / or forces that the motors and / or actuatorsshould provide in order to move joints or other parts of the robotic arm, e.g., ata specified velocity, at a specified, acceleration, or to hold the robot arm in astatic pose in the presence or absence of force(s).Controller 110 may also include, or connect to, an interface device111. Interface device 111 is configured to enable a user to control and / or toprogram operations of arm 101 via controller 110. Interface device 111 maybe a dedicated device, such as a teach pendent, which is configured tocommunicate with controller 110 via wired and / or wireless communicationprotocols. Such an interface device 111 may include a display 112 and a oneor more types of input devices 113 such as buttons, sliders, touchpads, joysticks,track balls, gesture recognition devices, keyboards, microphones, and the like.Display 112 may be or include a touch screen acting both as display and inputdevice or user interface. Interface device 111 device may be or include ageneric computing device (not shown), such as a smartphone, a tablet, or apersonal computer including a laptop computer, configured with appropriate programming to communicate with controller 110. Arm 101 is controllable by controller 110 to operate in differentmodes, including a teaching mode. For example, a user may provideinstructions to the controller via interface device 111 to cause arm 101 to enterthe teaching mode. In the teaching mode, controller 110 controls the amountof torque provided to the joints by the motors and / or actuators to enable arm101 to maintain a pose in the presence of gravitational force, but also to allowone or more components associated with arm 101, such as one or more links,or more joints, the tool flange, or an end effector, to be moved in response toan applied force. Such movement(s) change(s) the pose of the robotic arm.During or after such movement, controller 110 controls the amount of torqueprovided to the joints by the motors and / or actuators to enable arm 101 tomaintain the changed pose and / or to allow continued movement in response toadditional applied force.In some implementations, the applied force may be manual. Forexample, a user may grab onto one or more components associated with arm101, such as one or more links, or more joints, the tool flange, or an end effector,and physically move the component(s) to reposition arm to the changed pose.In some implementations, the applied force may be programmatic. For example,controller 110 may instruct the amount of torque to be provided to the joints bythe motors and / or actuators to reposition one or more component(s) into thechanged pose. In some implementations, the applied force may be a combination of manual and programmatic. In the teaching mode, arm 101 is taught various movements,which it may reproduce during automated operation. For example,in the teaching mode, arm 101, which includes an accessory such as a grippermounted to tool flange 104, is moved to positions in its environment. Arm 101is moved into a position that causes the gripper to interact with an object, also referred to as a “primitive”, in the robot’s environment. For example, a usermay physically / manually grasp part of arm 101 and move that part of arm 101into a different pose in which the gripper is capable of gripping the object. Asnoted above, the controller 110 controls the amount of torque provided to thejoints by the motors and / or actuators to enable the robotic arm to maintain thedifferent pose and / or to allow continued movement in response to this appliedforce. The gripper may be controlled by the controller to grasp the object and,thereafter, arm 101, with the gripper holding the object, may be moved into a new pose and position at which the gripper installs the object or deposits theobject. The user may physically / manually move the robotic arm into the newpose and position. Controller 110 records, and stores data representing operationalparameters such as an angular position of the output flange, an angular positionof a motor shaft of each joint motor, a motor current of each joint motor duringmovement of the robotic arm, and / or others listed below. This data may berecorded and stored in memory 118 continuously or at small intervals, such asevery 0.1 seconds (s), 0.5s, 1s, and so forth. Taken together, this data definesthe movement of the robotic arm that is taught to the robotic arm during theteaching mode. These movements can later be replicated automatically byexecuting code on controller 110, thereby enabling the robot to perform thesame task automatically without manual intervention.During user-applied physical movements in particular, it can bechallenging to align component(s) associated with the robotic arm with anintended target, such as an object in the environment. Misalignment can adversely affect future operation of the robot. In the example of a gripper, ifthe gripper is misaligned by as little as single-digital millimeters, the grippermay not be able to grasp the object during automatic operation. Due to theprecision required, alignment can be time-consuming for a user to implement.And, even then, the alignment may be prone to error. The processes described herein may address the foregoing issuesby identifying an object in the environment and by controlling at least part ofthe robotic arm to move into alignment with that object during the teachingmode. By automating at least part of the alignment with the object, the amountof time required during teaching may be reduced, since painstaking manualalignments to objects may no longer be required. Also, automating at least partof the alignment with the object may reduce the occurrence of misalignments.Fig. 2 is a flowchart showing example operations included in anexample process 120 of the foregoing type. Process 120 is described withrespect to arm 101 and may be performed by controller 110 either alone or incombination with one or more local and / or remote computing systems.During at least part of process 120, prior to controlling arm 101 tomove a component into alignment, controller 110 controls arm 101 to enablemanual movement of arm 101 in multiple degrees of freedom. This is done bycontrolling the amount of torque provided to the joints by the motors and / oractuators. For example, sufficient torque may be applied to overcome gravity,while enabling manual movement of components of arm 101 in multiple – e.g.,two, three, four, five, or six – degrees of freedom. In some implementations,this mode of operation is called free-drive mode. Accordingly, a component associated with arm 101 may be movedmanually by a user. The component may be or include any or all of the jointsand / or links of Fig. 1, such as joints 102f, 102e, 102d, and / or link 111 for example, and / or an end effector / accessory mounted on arm 101. An exampleof this movement is depicted graphically in Fig. 3. More specifically, Fig. 3 showscomponent 125 of arm 101 containing joints 102e, 102f, vision system 90, toolflange 104, and gripper 126 attached to the end effector interface on tool flange 103. The remainder of arm 101 is present, but not shown in Fig. 3 (or Fig. 4,5, 6, 8, 9, 10, or 11). During the teaching mode, a user 128 manually movescomponent 125 in the direction of arrow 130 towards object 131, which is to bepicked-up by gripper 126. This may be done in the free-drive mode in someimplementations. In an example, the object may be a workpiece, a container,a tool, or any other item. Vision system 90 has a FOV 132 depicted graphicallyby lines 132a, 132b. In Fig. 3 object 131 is outside of FOV 132 of vision system90 and, therefore, is not detected.Referring also to Fig. 4, during manual movement in the directionof arrow 130, at least part of object 131 comes within the FOV 132 of visionsystem 90. When enough of the object is in the FOV, process 120 is able toidentify (120a) object 131. For example, process 120 may be able to identifythe object if at least 20%, 30%, 40%, 50%, 60%, or more of the object is visibleto the vision system. In some implementations, identifying the object mayinclude capturing sensor data, such as one or more images, of an environmentusing vision system 90 and comparing those image(s) to images of variousobjects previously stored in memory 118. Image processing techniques may beused to identify the size and shape of the object in the image(s) and to comparethose to sizes and shapes of objects stored in memory. When there is sufficientsimilarity between features of the object in the image(s) and those stored in memory, the object is identified. For example, if an object in the image(s) has at least 60%, 70%, 80%, or more features in common with an object stored in memory, then the object in the image(s) may be deemed to be an instance of the object stored in memory. Similar processing may be performed using sensor data other than images. Still referring to Fig. 4, identifying (120a) the object may alsoinclude identifying an axis 134 along a part of object 131, such as a designatedcenter of object 131. The axis of the object that is used may be based on whatthe object is. For example, axes for different types of objects may be stored inmemory 118 and may be accessed by controller 110 to determine the axis ofan identified object. For example, if the object is determined to be a cylinderlike object 131 in Figs. 3 to 6, then controller 110 may read information frommemory 118 indicating that the axis is along a longitudinal dimension of theobject and through a center of the circular top of the cylinder. Controller 110may determine the dimensions of the object based on the image(s) of the object,and may calculate the location of the axis of the object based on the readinformation. In this example, controller 110 identifies the location of axis 134of object 131 in this manner.Referring to Fig. 7, in another example, an example object isdetermined to be a right-angle intersection 136 of two planar surfaces 137, 138(e.g., an intersection to be welded). Controller 110 may read information frommemory 118 indicating where the axis for such an object is located. In thisexample, the axis 139 is determined to be at 45° relative to each of surfaces137 and 138. Controller 110 may determine the dimensions of the object basedon image(s) of the object, and may calculate the location of axis 139 based onthe images(s) and the information obtained from memory 118. In some robotic systems, sensor data, such as one or more images,of the environment may be received electronically, rather than being capturedby vision system 90. In an example like this, the object may be identified usingthe sensor data in the same manner as described above. In addition, thelocation of the object in the environment may be identified. For example,controller 110 may store a map of the environment and compare image(s) tothe map in order to identify the location of the object within the environment.The axis of the object may be identified as described previously. As described below with respect to Fig. 15, if more than oneinstance of the object is identified in the environment, each instance is apotential candidate for alignment during teaching. In this example, process 120determines a distance between a component associated with arm 101 andidentified instances 131a, 131b of the object. The object that is determined tobe closest to the component associated with arm 101 is selected as the one foralignment. Example techniques for calculating the distance between arm 101and different instances of an object are described below with respect tooperation 120b. Referring back to Figs. 2 and 3, process 120 includes determining(120b) if a component 125 associated with arm 101, such as joint 102f orgripper 126, is within a predefined vicinity (e.g., distance) of object 131. Themagnitude of the predefined vicinity may be set by a user on a teach pendantor by a computer program, may be stored in memory 118, and may beaccessible to controller 110. The predefined vicinity may be based on the axisof the object. For example, the predefined vicinity may be defined as a distancebetween the axis of the object and a axis of component 125 of arm101 that is moved relative to the object. In some implementations, thepredefined vicinity may be 50.8 millimeters (mm) (2 inches) or less, 40mm or less, 30mm or less, or any other appropriate value. As explained with respect to Fig. 3, in that example, a user 128manually moves component 125 of arm 101 in the direction of arrow 130towards object 131 so that object 131 is within the FOV 132 of vision system90. The object within the FOV 132 of vision system 90 is shown in Fig. 4.Process 120 identifies object 131 in the manner described above and determineswhether component 125 of arm 101 is within a predefined vicinity of object 131.In this example, the predefined vicinity 140 is the distance between axis 134 ofobject 131 and a predefined axis 142 associated with arm 101. For example,the predefined axis 142 may be the center of tool flange 104 (as in this example)or the center of gripper 126. The predefined axis may be defined to be along asurface of component 125, or along or through any other component, surface,or part of arm 101. To determine if component 125 of arm 101 is within the predefinedvicinity of object 131, process 120 measures the distance between axes 134and 142 continually, periodically, or sporadically. The distance may bemeasured based on sensor data, such as image(s), captured by vision system90 as shown in Fig. 4. For example, controller 110 may know the scale of theimages and the FOV 132 of vision system 90. Knowing this information,controller 110 may calculate the real-world distance (as opposed to the distancein the image(s)) between axes 134 and 142. In another example, controller110 may know the location of the object in the environment based on a map ofthe environment and determine the location in the environment of component125 of arm 101 based, for example, on movements of joints in arm 101. Usingthis information, controller 110 may calculate the real-world distance betweenaxes 134 and 142.To determine if component 125 of arm 101 is within the predefined vicinity of object 131, controller 110 compares the calculated distance between axes 134 and 142 to the distance that defines the predefined vicinity. If thecalculated distance is greater than the distance that defines the predefinedvicinity, then component 125 is not to be within the predefined vicinity of object 131 (120c). In this case, new values of the calculated distanceare determined and compared to the distance that defines the predefinedvicinity. During this time, the user can manipulate the arm freely; no extraforce will be applied from the arm. This continues during operation of arm 101,e.g., until component 125 is determined to be within the predefined vicinity ofobject 131. If the calculated distance is less than the distance that defines the predefined vicinity, then component 125 is determined to be within thepredefined vicinity of object 131 (120c).After it is determined (120c) that component 125 of arm 101 iswithin the predefined vicinity of object 131, processing proceeds to operation120d. In operation 120d, controller 110 controls arm 101 to move component125 towards or into alignment with the object. To control arm 101 to movecomponent 125 towards or into alignment with the object, controller 110 controls the amount of torque provided to the joints by the motors and / oractuators. The movement is automatic and does not require manual intervention.Effectively, the torque is provided to the joints by the motors and / or actuatorsto draw, pull, or move component 125 of robotic arm towards or into alignmentusing minimal or no additional manual force. For example, drawing, pulling, ormoving component 125 towards or into alignment may be implemented absent manual force or with the assistance of manual force. As shown in Fig. 4, the torque is provided to the joints by themotors and / or actuators to draw, pull, or move component 125 in the directionof arrow 144 to arrive at, or close to, the alignment of Fig. 5. In Fig. 4, arrow144 is from component 125 to indicate that the drawing, pulling, or movementoccurs through operation of the motors and / or actuators and not manually (incontrast to Fig. 3 where the movement is manual).In some implementations, torque is provided to the joints by themotors and / or actuators to generate force to draw, pull, or move component125 of robotic arm 101 towards alignment with, but not into final alignment with,object 131. In some implementations, the amount of force applied to draw, pull,or move component 125 of robotic arm 101 towards alignment with, but not into final alignment with, object 131, may be set or configured by a user in software that control operation of the arm. For example, a user interface may be generated by the software and output on a display device associatedwith the robotic arm (e.g., interface device 111), into which a user may providethe requisite amount of force. In an example, the amount of force applied todraw, pull, or move component 125 of robotic arm 101 towards alignment with,but not into final alignment with, object 131 may be 3 Newtons (N), 4N, 5N ormore. The amount of force that a user may apply manually to overcome thedrawing, pulling, or moving may thus be an amount of force that exceeds theamount of force drawing, pulling, or moving component 125 towards alignment with the object. In some examples, a six degree of freedom force and torque maybe applied at the end of the robotic arm. In some implementations, the amountof force is proportional to the distance to the object. For example, as component125 gets closer to object 131, the amount of force automatically applied to draw, pull, or move component 125 of robotic arm 101 towards alignment with, but not into final alignment with, object 131 may increase proportionally as the distance to the object decreases. During the time when component 125 of robotic arm 101 is withinthe predefined vicinity of the object, controller 110 continues to calculate thedistance between axes 134 and 142. Upon reaching a predefined thresholddistance, which is less than the predefined vicinity, a final alignment process isimplemented. For example, the threshold distance may be 10mm, 5mm, 4mm,3mm or less, 2mm or less, 1mm or less, or any other appropriate distancebetween axes 134 and 142. The final alignment process may include controllingcomponent 125 to snap component 125 into final alignment with the object.This final alignment may be performed by controlling the motors and / oractuators to provide greater, and more abrupt, torque to the joints than wasapplied while drawing, pulling or moving component 125 prior to reaching thethreshold distance. At the final stage of alignment, the robotic arm is given amove command to the final destination. In some implementations, the amount of force applied to snapcomponent 124 into final alignment with the object may be set or configured bya user in software that control operation of the robotic arm. For example, a user interface may be generated by and output on a display deviceassociated with the robotic arm (e.g., interface device 111), into which a usermay provide the requisite amount of force. In an example, the amount of forceapplied to snap component 124 into final alignment with the object may be 4N,5N, 6N, 7N, 8N, 9N, 10N, 11N, 12N, 13N, 14N, 15N, or more. The amount offorce that a user may apply manually to overcome the snapping action may thusbe an amount of force that exceeds the amount of force snapping component125 into alignment with the object. In some implementations, the snappingaction may occur so quickly as to effectively prevent manual intervention to prevent it. In some implementations, the vision system may confirm the finalalignment by capturing sensor data, such as an image, of arm 101 aligned withthe object and confirming that the alignment is correct based on positions of the axes of component 125 and object 131. Following alignment (120d), controller 110 controls the amount oftorque provided to the joints by the motors and / or actuators to constrainmovement of component 125 relative to the 134 of object 131. For example,the movement of component 125 of arm 101 may be constrained to move inone dimension relative to, or along, axis 134. This is shown in Fig. 6, whichdepicts component 125 constrained to move vertically along up and down(depicted by arrow 145) along axis 134. In some implementations, the one-dimensional movement may be horizontal or at an oblique angle relative to anobject. This movement may be implemented manually to cause gripper 126 tocontact object 131 during the teaching mode. The automatic alignment andconstrained movement thus reduces the chances of misalignment whencomponent 125 is brought into contact with the object. In some implementations, the amount of torque that is provided tothe joints is sufficient to counteract manual / physical attempts to movecomponent 125 of arm 101 out of alignment with the object or to preventalignment with the object. For example, in some implementations, an amount of manual force exceeding 4N, 5N, 6N, 7N, 8N, 9N, 10N, 11N, 12N, 13N, 14N,15N, or more may be used to move component 125 of arm 101 out of alignmentwith the object.Referring to Fig 15, which is a variant of Fig. 4, in someimplementations, there may be more than one object 131a, 131b within theFOV 132 of vision system 90. In cases such as this, the distance betweenpredefined axis 142 associated with arm 101 and each of axis 134a of object131a and axis 134b of object 131b is measured. The distance 140a for object131a and the distance 140b for object 131b are compared. Whichever distance140a, 140b is less is identified and the corresponding object is selected as theobject to which arm 124 is drawn, pulled, or moved into alignment with. Thealignment process then proceeds as described above.Figs. 8 to 11 show, graphically, another example of aligningcomponent 125 of a robotic arm to a different object 150. In the example ofFigs. 8 to 11, component 125 of arm 101 is controlled to align to the right-angleintersection 152 of two planes comprising object 150. As shown in Fig. 8, auser 128 manually moves component 125 of arm 101 toward the object in thedirection of arrow 155. In Fig. 9, vision system 90 detects object 150. Enoughof the object is detected to determine the identity of object 150 based on storedinformation as described above. Information stored about the object includes the location of axis 156 to which component 125 is to align. When component125 of arm 101 is within the predefined vicinity of axis 156 of object 150,controller 110 controls the amount of torque provided to the joints by the motorsand / or actuators to draw, pull, or move component 125 of arm 101 near or intoalignment with axis 156 using minimal or no additional manual force asillustrated by arrow 157. In some implementations, as described herein, when component 125 is within a threshold distance of axis 156 (e.g., on the order ofsingle-digit millimeters), component 125 may snap into alignment with axis 156.The resulting alignment is shown in Fig. 10. Thereafter, component 125 of arm101 is constrained to move relative to axis 156 in the directions of arrows 158.In this example, what this means is that component 125 of arm 101 isconstrained to move at a 45° angle to the left and to the right of axis 156 alongat least the entirety of intersection 152. A use case such as this may beappropriate, e.g., when a welding tool 160 is connected to the end effectorinterface of tool flange 104 to weld the intersection.Figs 12, 13, and 14 show examples of other objects to which arm101 may align according to process 120, although it is noted that process 120may be used to align arm 101 to any appropriate object or part of an object.Fig. 12 shows a cylinder 161 having a flange 162 to which a component of arm101 may align according to process 120. The arm may for instance be alignedto a center axis of the cylinder the outer perimeter of the cylinder, the outer perimeter of the flange, the interception between the cylinder and flange. Fig.13 shows a plane 163 having a hole 164 therethrough to which a component ofarm 101 may align according to process 120, for instance the center axis 165of the hole. Fig. 14 shows a surface 166 having a corner 167 to which acomponent of arm 101 may align according to process 120, for insurance to oneor more of the coordinate axis X, Y, Z of a coordinate system having origo atthe corner 167. Generally speaking, arm 101 may align to any type of objecthaving a regular or irregular shape using process 120. In another example (notshown in the figures), arm 101 can be taught to identify a chuck – which isdevice that securely holds a workpiece in its position during a machining process– of a computer numerical control (CNC) lathe machine and to align acomponent of arm 101 (such as a gripper) holding the workpiece to the chuckso that the robot can be taught to place the workpiece in the chuck. Referring back to Fig. 2, during the teaching mode, process 120records (120f) operational parameters associated with arm 101 based onmovements made during teaching, including manual movements andautomated movements. The operational parameter may be or include anyparameters, values and / or states relating to the robot system such as sensorparameters obtained via various sensors on or associated with the robot system.Examples of the sensor parameters include, but are not limited to, angle,position, speed, and / or acceleration of the robot joints; values of force / torquesensors of or on the robot system; images / depth maps obtained by the visionsystem; environmental sensor parameters such as temperature, humidity or thelike; distances measured by distance sensors; and / or positions of devicesexternal to arm 101 such as conveyer positions, speed, and / or acceleration.The operational parameters can also include status parameters of devicesassociated with, or connected to, system such as status of endeffectors, status of devices external to arm 101, status of safety devices, or thelike. The status parameters may also relate to an end effector interface of therobotic system or a tool connected to end effector interface. A force / torquesensor, for example, may be included on the tool flange to measure forces and / or torques applied by the robotic arm. The forces and / or torques may beprovide to the robot control system and used to affect – for example, change –operation of the robotic arm. The forces and / or torques many be recorded (120f) as operational parameters. Additionally, the operational parameters can include parametersgenerated by a robot program during a recording process such as target torque,positions, speed, and / or acceleration of the robot joints; force / torques thatparts of arm 101 or other parts of the of that robotic system experience; and / orvalues of logic operators such as counters and / or logic values.The operational parameters can also include external informationprovided by external systems or central services or other systems; for instancein form of information sent to and from central servers over a network. Suchparameters can be obtained via any type of communication ports of the robotsystem including, but not limited to, digital input / output ports, Ethernet ports,and / or analog ports.Process 120 translates (120g) all or part of the operationalparameters into robot code. Example robot code includes executableinstructions that, when executed by controller 110, cause the robot system to perform robot operations, such as imitating and / or replicating the movements performed during teaching that produced the operational parameters, includingactivating / deactivation end effectors, e.g., opening and / or closing grippers asdemonstrated during teaching. The robot code is stored (120h) in memory 118,from which it can be accessed by controller 110. Accordingly, when the robotis no longer in teaching mode, and is instructed to perform a task, the robotcode corresponding to that task is retrieved from memory and executed by the robot controller to control operation of the robot to perform the task automatically, e.g., without manual intervention. Controlling operation of the robot may include, for example, controlling torques and / or forces that the motors and / or actuators provide to joints or other parts of arm 101, e.g.,at a specified velocity and / or acceleration, or to hold arm 101 in a particularstatic pose, among other things in order to perform the task.Process 120 is described with respect to arm 101 shown in Fig. 1;however, process 120 is not limited to use with robotic arms like those shownin Fig. 1 or even to robotic arms in general. Process 120 is may be used withany part of a robot that is movable in multiple – for example, two, three, four,five or six – degrees of freedom to perform an operation. For example, anautomated vehicle, such as a rover, may include an appendage that iscontrollable according to process 120. In an example, process 120 may be usedwith an appendage connected to an autonomous vehicle robot of the type thatis the subject of U.S. Patent No. 11,287,824 (issued March 29, 2022), and which is described with respect to Figs. 1, 2, and 3 thereof. The contents of U.S. Patent No. 11,287,824 relating to the description of the autonomous vehicle areincorporated herein by reference. In another example, process 120 may beused with an appendage connected to an autonomous vehicle robot of the typethat is the subject of U.S. Patent Publication No. 2021 / 0349468 (published November 11, 2021), and which is described with respect to Figs. 1, 2, and 3 thereof. The contents of U.S. Patent Publication No. 2021 / 0349468 relating to the description of the autonomous vehicle are incorporated herein by reference. The example robots, systems, and components thereof, describedherein can be controlled, at least in part, using one or more computer program products, e.g., one or more computer program tangibly embodied in one or more information carriers, such as one or more non-transitory machine- readable media, for execution by, or to control the operation of, one or more data processing apparatus, e.g., a programmable processor, a computer, multiple computers, and / or programmable logic components. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment. Acomputer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a network. Actions associated with implementing at least part of the robots,systems, and components thereof can be performed by one or moreprogrammable processors executing one or more computer programs to perform the functions described herein. At least part of the robots, systems,and components thereof can be implemented using special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) and / or an ASIC (application-specific integrated circuit). Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only storage area or a random access storage area or both. Elements of a computer include one or more processors for executing instructions and one or more storage area devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from, or transfer data to, or both, one or more machine-readable storage media, such as mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. Machine- readable storage media suitable for embodying computer program instructions and data include all forms of non-volatile storage area, including by way of example, semiconductor storage area devices, e.g., EPROM, EEPROM, and flash storage area devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. In the description and claims provided herein, the adjectives “first”,“second”, “third”, and the like do not designate priority or order unless contextindicates otherwise. Instead, these adjectives may be used solely todifferentiate the nouns that they modify. Any mechanical or electrical connection herein may include a direct physical connection or an indirect connection that includes interveningcomponents unless context indicates otherwise.Elements of different implementations described herein may be combined to form other implementations not specifically set forth above. Elements may be left out of the structures described herein without adversely affecting their operation. various separate elements may be combined into one or more individual elements to perform the functions described herein.

Claims

CLAIMS 1. A robotic system comprising: a robotic arm configured to move in multiple degrees of freedom; and a control system comprising one or more processing devices, the one or more processing devices being programmed to perform operations comprising: identifying an object in the environment accessible to the robotic arm based on sensor data indicative of one of more properties of theenvironment; determining that a component associated with the robotic arm is within a predefined distance of the object; andcontrolling the robotic arm to move the component toward or intoalignment with the object in response to the component being within the predefined distance of the object.

2. The robotic system of claim 1, wherein identifying the object comprises identifying an axis of the object; wherein the predefined distance is measured relative to the axis of theobject; and wherein controlling the robotic arm to move the component toward or into alignment comprises controlling the robotic arm to move the component toward or into alignment with the axis.

3. The robotic system of claim 2, wherein the axis is along a center of the object.

4. The robotic system of claim 2, wherein the axis is along a part of the object.

5. The robotic system of any one of claims 2-4, wherein the operationscomprise: following alignment of the component with the axis, constrainingmovement of at least part of the robotic arm to be along the axis.

6. The robotic system of any one of claims 2-5, wherein the operationscomprise: following alignment of the component with the axis, constraining movement of at least part of the robotic arm relative to the axis.

7. The robotic system of any one of claims 2-6, wherein the operationscomprise: following controlling the robotic arm to move the component toward or into alignment with the object; enabling manual movement of at least part of the robotic arm along the axis to allow the robotic arm to interact with the object; recording movements of the robotic arm interacting with the object; translating the movements into robot code; and storing the robot code in memory on the control system.

8. The robotic system of any one of claims 1-7, wherein the operationscomprise: recording operational parameters of the robotic system; translating the operational parameters into robot code; and storing the robot code in memory on the control system.

9. The robotic system of claim 8, wherein the operational parameters relate to one or more of the following: sensor data of said robotic system,input / output ports in the robotic system, or an end effector or tool of therobotic system.

10. The robotic system of any one of claims 1-9, wherein theoperations comprise, prior to controlling the robotic arm to move thecomponent toward or into alignment, controlling the robotic arm to enablemanual movement of the robotic arm in multiple degrees of freedom; and wherein controlling the robotic arm to move the component toward orinto alignment is performed automatically absent manual intervention.

11. The robotic system of any one of claims 1-10, wherein theoperations comprise, prior to controlling the robotic arm to move thecomponent toward or into alignment, controlling the robotic arm to enablemanual movement of the robotic arm in multiple degrees of freedom; and wherein controlling the robotic arm to move the component toward or into alignment is performed in combination with manual movement of the robotic arm.

12. The robotic system of any one of claims 1-11, wherein thecomponent comprises a tool or end effector connected to the robotic arm.

13. The robotic system of any one of claims 1-12, wherein thecomponent comprises a part of the robotic arm.

14. The robotic system of any one of claims 1-13, further comprising:a vision system associated with the robotic arm to capture at least apart of the sensor data.

15. The robotic system of any one of claims 1-14, wherein the visionsystem comprises one or more cameras mounted to the robotic arm.

16. The robotic system of any one of claims 1-15, wherein theoperations comprise: receiving the sensor data electronically.

17. The robotic system of any one of claims 1-16, wherein theoperations comprise: enabling the robotic arm to be moved out of the predefined distanceduring alignment in response to a predetermined amount of manual force.

18. The robotic system of any one of claims 1-17, wherein theenvironment contains multiple objects, each of the multiple objects being acandidate for alignment with the component, and each of the multiple objects being at a different distance from the component.

19. The robotic system of claim 18, wherein the object to which the component is configured to align is a closest one of the multiple objects to the component.

20. The robotic system of any one of claims 1-19, wherein determining that the component associated with the robotic arm is within the predefined distance of the object comprises determining that the component of the robotic arm is within a first distance from the object; wherein controlling the robotic arm to move the component toward alignment with the object is performed in response to the component of the robotic arm being within the first distance from the object; wherein determining that the component associated with the robotic arm is within the predefined distance of the object comprises determining that the component of the robotic arm is within a second distance from the object, the second distance being less than the first distance; and wherein controlling the robotic arm to move the component into alignment with the object is performed in response to the component of the robotic arm being within the second distance from the object and is performed with greater force than moving the component toward alignment.

21. A method of controlling a robotic arm, the method comprising: obtaining sensor data indicative of one of more properties of anenvironment accessible to the robotic arm; identifying an object in the environment based on the sensor data; determining that a component associated with the robotic arm is withina predefined distance of the object; andcontrolling the robotic arm to move the component toward or intoalignment with the object in response to the component being within thepredefined distance of the object.

22. The method of claim 21, wherein identifying the object comprises identifying an axis of the object; wherein the predefined distance is measured relative to the axis of theobject; and wherein controlling the robotic arm to move the component toward or into alignment comprises controlling the robotic arm to move the component toward or into alignment with the axis.

23. The method of claim 22, wherein the axis is along a center of the object.

24. The method of claim 22, wherein the axis is along a part of the object.

25. The method of any one of claims 22-24, further comprising:following alignment of the component with the axis, constraining movement of at least part of the robotic arm to be along the axis.

26. The method of any one of claims 21-25, further comprising:following alignment of the component with the axis, constrainingmovement of at least part of the robotic arm relative to the axis.

27. The method of any one of claims 21-26, further comprising:following controlling the robotic arm to move the component toward or into alignment with the object; enabling manual movement of at least part of the robotic along the axis to allow the robotic arm to interact with the object; recording movements of the robotic arm interacting with the object; translating the movements into robot code; and storing the robot code in memory on the control system.

28. The method of any one of claims 20-27, further comprising:recording operational parameters of the robotic system; translating the operational into robot code; andstoring the robot code in memory on the control system.

29. The method of claim 28, wherein the operational parameters relateto one or more of the following: input / output ports in the robotic system oran end effector or tool connected to the robotic arm.

30. The method of any one of claims 20-29, further comprising:prior to controlling the robotic arm to move the component toward or into alignment, controlling the robotic arm to enable manual movement of the robotic arm in multiple degrees of freedom; and wherein controlling the robotic arm to move the component toward or into alignment is performed automatically absent manual intervention.

31. The method of any one of claim 20-30, further comprising: prior to controlling the robotic arm to move the component toward or into alignment, controlling the robotic arm to enable manual movement of the robotic arm in multiple degrees of freedom; and wherein controlling the robotic arm to move the component toward or into alignment is performed in combination with manual movement of the robotic arm.

32. The method of any one of claims 20-31, wherein the componentcomprises a tool connected to the robotic arm.

33. The method of any one of claims 20--32, wherein the componentcomprises a part of the robotic arm.

34. The method of any one of claims 20-33, wherein the one or moreimages are obtained electronically.

35. The method of any one of claims 20-33, wherein the one or moreimages are obtained from one or more cameras connected to the robotic arm.

36. The method of any one of claims 20-35, further comprising:enabling the robotic arm to be moved out of the predefined distanceduring alignment in response to a predetermined amount of manual force.

37. The method of any one of claims 20-36, wherein the environmentcontains multiple objects, each of the multiple objects being a candidate for alignment with the component, and each of the multiple objects being at a different distance from the component.

38. The method of claim 37, wherein the object to which the component is configured to align is a closest one of the multiple objects to the component.

39. The method of any one of claims 21-38, wherein determining that the component associated with the robotic arm is within the predefined distance of the object comprises determining that the component of the robotic arm is within a first distance from the object; wherein controlling the robotic arm to move the component toward alignment with the object is performed in response to the component of the robotic arm being within the first distance from the object; wherein determining that the component associated with the robotic arm is within the predefined distance of the object comprises determining that the component of the robotic arm is within a second distance from the object, the second distance being less than the first distance; and wherein controlling the robotic arm to move the component into alignment with the object is performed in response to the component of the robotic arm being within the second distance from the object and is performed with greater force than moving the component toward alignment.

40. One or more non-transitory machine-readable storage devices storing instructions that are executable by one or more processing devices to control a robotic arm, the instructions being executable to perform operations according to the method of any one of 21-39.

Images