Method and device for robot control

Abstract

The present application relates to methods and devices for robot control. Methods, devices, systems and articles for robot control are disclosed. An example device includes a command generator for instructing a robot to move an end effector from a temporary position to an estimated pre-task position to perform a first task based on a first posture of the robot, the first posture is based on a model, when the robot is to move to a second actual pre-task position, adjust the robot to the first actual pre-task position to perform the first task, the first actual pre-task position is close to the estimated pre-task position, and guide the robot to perform the second task based on a correction factor, and determine the correction factor by determining a second posture of the robot, the second posture corresponding to position information associated with the post-task position, and calculating the correction factor based on the first posture and the second posture.

Description

Technical Field

[0001] The present disclosure relates generally to robots, and more particularly to methods and apparatus for robotic control. Background Art

[0002] Object recognition systems are being deployed in a range of applications and corresponding devices. Devices such as collaborative robots can use object recognition systems to perform tasks related to assembly or manufacturing. Object recognition systems can guide robot arms and end effectors to designated locations to perform assembly tasks. However, object recognition systems can increase the cost, complexity, and delays of assembly tasks. Summary of the invention

[0003] Systems, methods, apparatus, and articles of manufacture for robotic control are disclosed.

[0004] An example apparatus disclosed herein includes a command generator for: instructing / commanding a robot to move an end effector from a staging position to an estimated pre-task position to perform a first task based on a first posture of the robot, the first posture being based on a model; when the robot is to move to a second actual pre-task position, adjusting the robot to a first actual pre-task position to perform the first task, the first actual pre-task position being close to the estimated pre-task position; and guiding the robot to perform the second task based on a correction factor, the correction factor being determined by: determining a second posture of the robot, the second posture corresponding to position information associated with the robot at a post-task position, the post-task position corresponding to position information associated with the robot after performing the first task and calculating the correction factor based on the first posture and the second posture.

[0005] A non-transitory computer-readable storage medium containing instructions, which, when executed, causes a machine to at least: instruct a robot to move an end effector from a temporary position to an estimated pre-task position to perform a first task based on a first posture of the robot, the first posture being based on a model; when the robot is to move to a second actual pre-task position, adjust the robot to a first actual pre-task position to perform the first task, the first actual pre-task position being close to the estimated pre-task position; after performing the first task, determine a second posture of the robot, the second posture corresponding to position information associated with the robot at a post-task position, the post-task position corresponding to the position of the robot after performing the first task; calculate a correction factor based on the first posture and the second posture; and guide the robot to perform a second task based on the correction factor.

[0006] An example method disclosed herein includes: instructing a robot to move an end effector from a first position to a second position to perform a first task based on a first posture of the robot, the first posture being based on a model; in response to the robot moving to a third position, adjusting the robot to a fourth position to perform the first task, the fourth position being close to the second position; in response to performing the first task, determining a second posture of the robot, the second posture corresponding to position information associated with the robot at a fifth position, the fifth position corresponding to the position of the robot after performing the first task; calculating a correction factor based on the first posture and the second posture; and guiding the robot to perform a second task based on the correction factor. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1A and Figure 1B An example robot manager that controls an example robot to perform an assembly task on an example component is depicted.

[0008] Figure 2 yes Figure 1A and Figure 1B A block diagram of an example implementation of an example robot manager.

[0009] Figure 3 It means that it can be executed to implement Figure 1A-Figure 2 The example robot manager can be used by using Figure 1A-1B A flow chart of example machine readable instructions for an example robot to perform an assembly task.

[0010] Figure 4 It means that Figure 1A-Figure 2 A flow chart of example machine readable instructions for using an example robot manager to update a model based on a correction factor.

[0011] Figure 5 It means that it can be executed to implement Figure 1A-Figure 2 The example robot manager can be used by using Figure 1A-1B Another flow chart of example machine readable instructions for an example robot to perform an assembly task.

[0012] Figure 6 is constructed to execute Figure 3-Figure 5 Example machine readable instructions to implement Figure 1A-Figure 2 Block diagram of an example processing platform for an example robot manager.

[0013] These drawings are not to scale. Generally, the same reference numbers will be used throughout the drawings and accompanying written description to refer to the same or like parts. DETAILED DESCRIPTION

[0014] Typical robots used for industrial assembly or manufacturing operations, such as collaborative robots, utilize and / or rely on external sensors or additional sensors to the main sensors associated with the robot's end effector for positioning. Collaborative robots (also known as cobots) are machines used to physically interact with humans in a shared workspace. In some cases, collaborative robots are programmed and / or otherwise configured to operate autonomously or under limited guidance. These robots can be used to perform workflows that include one or more assembly operations or tasks at one or more assembly locations. For example, a robot can be used to perform assembly tasks that connect and / or otherwise install fasteners on an object surface at a location or orientation on the object surface.

[0015] Localization refers to the robot determining the robot's position in a computer-based model environment that corresponds to the robot's physical environment. For example, the robot can use feedback from a primary sensor such as an encoder to determine the robot's position, changes in position, etc. in the physical environment by reference to a computer-based model. In some cases, the robot includes one or more auxiliary sensor systems to assist in localization determination. For example, the robot can use another encoder (e.g., an optical encoder), a camera (e.g., a camera-based sensor system), a laser, etc., in coordination with the primary sensor to determine the robot's position.

[0016] Some known robots perform sequential assembly tasks using pre-generated trajectories, where each trajectory includes a starting position and an ending position. In some cases, the known robot may fail to successfully complete the assembly task because the uncertainty in the robot positioning meets an uncertainty threshold. The known robot may determine that the assembly task cannot be successfully completed because the difference between the actual robot pose (e.g., the actual robot position) and the expected robot pose (e.g., the expected robot position) is greater than the uncertainty threshold. For example, because the known robot cannot reliably determine the position information of the robot relative to the physical environment, the known robot may cause a collision between the robot's end effector and the component.

[0017] As used herein, the term "pose" refers to position information including at least one of the orientation or position of an object (e.g., an end effector, one or more joints of a robot, etc.) relative to a (defined) coordinate system. In some cases, a pose is described with the aid of a rotational transformation and / or a translational transformation that can bring an object from a reference pose or an estimated pose to an actual pose or an observed pose. In some cases, the rotational and / or translational transformation information can be represented by one or more rotation matrices, translation matrices, etc. In some cases, the rotational and / or translational transformation information can be represented by one or more rotation quaternions, translation quaternions, etc. In some cases, a robot can move from a first pose to a second pose by following a trajectory that links and / or otherwise connects the first pose to the second pose, moving along the trajectory to one or more waypoints, etc. For example, a trajectory can refer to a motion path of an end effector of a robot to include motions, angles, velocities, etc., of various robot arm parts, and / or combinations thereof.

[0018] In some cases, a known robot may use an auxiliary sensor system to adjust the actual robot pose to an expected robot pose in order to complete an assembly task. For example, a known robot may be intended to move to a first position in a physical environment, where the first position is based on a model, but moves to a second position in the physical environment due to size differences, tolerance stacking, etc. between the robot in the physical environment and the robot in the model environment. In such an example, the first pose at the first position is an estimated robot pose, and the second pose at the second position is an actual robot pose. The known robot may use an auxiliary sensor system to move the robot from the second position to the first position to perform an assembly task by calculating a correction factor or positioning offset. The known robot may calculate a correction factor and perform such an adjustment after performing each assembly task, where the calculated correction factor is different from one assembly task to the next.

[0019] However, auxiliary sensor systems add additional material costs and maintenance costs to these known robots. Auxiliary sensor systems also increase computational costs and development costs because of the added complexity in programming these known robots to incorporate sensor-associated systems. Known robots that use and / or rely on auxiliary sensor systems to perform operations are adversely affected by the increased execution time required to obtain sensor data, process the sensor data, and generate positioning information based on the processed sensor data.

[0020] Examples disclosed herein include an example robot manager for robot control by determining and applying positioning offsets to robot trajectories when the robot performs a task. In some disclosed examples, the robot manager generates a computer-based model including an estimated pose, trajectory, etc. associated with the robot, which sequentially executes a workflow including multiple assembly tasks (e.g., 10 assembly tasks, 100 assembly tasks, 1000 assembly tasks, etc.) without user or third-party intervention. In some disclosed examples, before the robot starts and / or otherwise executes the workflow, the user guides the robot to a first assembly position (e.g., a first pre-task position, a first pre-manipulation position, etc.) associated with a first assembly task included in the workflow. After guiding the robot to the first assembly position, the user verifies that the robot is in an accurate position to perform the first assembly task. In some disclosed examples, the robot manager calculates a first actual pre-task pose of the robot at the first pre-task position. The first actual pre-task pose may refer to the position of the robot before performing an assembly task such as twisting a collar onto a bolt.

[0021] When the user verifies the position, the robot manager instructs the robot to perform a first assembly task. When the robot performs the first assembly task, the robot moves to a post-task position (e.g., a post-manipulation position), and the robot manager calculates a first actual post-task pose of the robot. The first actual post-task pose may refer to the position of the robot after performing an assembly task, such as torqueing a collar onto a bolt. The robot manager calculates a first actual pre-task pose and a first actual post-task pose of the robot by obtaining measurements from a sensor monitoring an end effector of the robot. The robot manager calculates a positioning offset by comparing at least one of (1) the first actual pre-task pose to a first estimated pre-task pose or (2) the first actual post-task pose to a first estimated post-task pose, wherein the first estimated pre-task pose and the first estimated post-task pose are determined by a computer-based model. For example, the positioning offset may be defined by rotation and / or translation transformation information represented by one or more rotation quaternions, translation quaternions, and the like. In some examples, the positioning offset corresponds to a positioning offset pose, wherein the robot manager 100 determines a trajectory from a current estimated pose to the positioning offset pose.

[0022] In some disclosed examples, the robot manager performs multiple assembly tasks based on the positioning offset without further user intervention. For example, the robot manager can instruct the robot to perform a second assembly task, a third assembly task, etc. based on the positioning offset. The robot manager can instruct the robot to move to a second estimated pre-task posture associated with performing the second assembly task. In response to the instruction to move to the second estimated pre-task posture, the robot can move along a first trajectory to move from the original posture to a second actual pre-task posture, wherein the first trajectory is determined by a computer-based model. When the robot completes the first trajectory, the robot manager applies the positioning offset to the first trajectory to adjust the position of the robot. For example, the robot manager can instruct the robot to move along a second trajectory to move from a second actual pre-task posture to a third actual pre-task posture, wherein the second trajectory is determined by applying the positioning offset to the first trajectory. And the third actual pre-task posture is close to the second estimated pre-task posture. In such an example, the robot manager calculates additional positioning offsets by using positioning offsets without user intervention or auxiliary sensor systems, thereby facilitating the execution of the second assembly task and subsequent assembly tasks.

[0023] Figure 1A An example robot manager 100 is depicted controlling an example robot 102 to perform an assembly task on an example component 104 in an example environment 106. Figure 1A In the embodiment, the robot manager 100 is external to the robot 102. Additionally or alternatively, the robot manager 100 may be included in the robot 102. Figure 1A , the robot 102 is an impedance-controlled robot (eg, an impedance-controlled collaborative robot) including an exemplary end effector 108 . Figure 1A The robot 102 includes one or more actuators 110 (e.g., electric motors, hydraulic pumps, pneumatic pistons, solenoids, valves, etc.), which may be actuated and / or otherwise controlled using one or more processors. The one or more processors may be included in the robot 102 and / or (communicatively) coupled to the robot 102. Alternatively, the robot 102 may be any other type of machine capable of performing assembly tasks with or without assistance or input from an operator (e.g., a maintenance person, technician, user, etc.). Figure 1A , environment 106 is a physical environment that includes robot 102, components 104, etc. For example, environment 106 may correspond to an assembly plant, a manufacturing facility, etc.

[0024] exist Figure 1AIn the embodiment, the end effector 108 is a tightening wrench tool (e.g., an electric tightening wrench, a pneumatic wrench, etc.) or a torque wrench. For example, the tightening wrench tool can be used to install parts, assembly items, etc. to each other and / or otherwise connect parts, assembly items, etc. Alternatively, the end effector 108 can be any other type of object manipulator, such as a clamp (e.g., an impact clamp, an intrusion clamp, a shrink clamp, a continuous clamp, etc.), a paint spray gun, a welding head, etc. Figure 1A The robot 102 includes one or more exemplary sensors 112 associated with the end effector 108. Figure 1A In the embodiment, the sensor 112 is a force-torque sensor. For example, the sensor 112 can measure force and / or torque along one or more axes (e.g., X-axis, Y-axis, Z-axis, etc.) of the end effector 108 and / or more generally, associated with assembling an item. Additionally or alternatively, the sensor 112 can be an encoder. Alternatively, the robot 102 can include sensors such as cameras, encoders, etc. For example, Figure 1A The robot 102 may include a camera-based system (e.g., a system including one or more cameras, video cameras, etc.) communicatively coupled to the robot manager 100 to assist the robot 102 in moving, orienting, etc. when performing tasks. For example, the robot 102 may include one or more sensors 112, such as cameras, encoders, force-torque sensors, etc., and / or combinations thereof.

[0025] exist Figure 1A In the embodiment of the present invention, the robot 102 is programmed (eg, one or more processors executing machine-readable instructions) and / or otherwise configured to perform one or more assembly tasks associated with the component 104. Figure 1A Component 104 is an air intake device of a turbine engine. For example, component 104 may be an aircraft engine inlet. Alternatively, component 104 may be any other type of component, device, structure, etc.

[0026] Figure 1A The robot 102 is configured to perform one or more assembly tasks associated with the component 104. Figure 1A In the embodiment, the robot 102 is configured to install and / or otherwise connect a first assembly item including an exemplary collar 114 to a second assembly item including an exemplary bolt 116. Alternatively, the robot 102 may couple any other type of fastener to the assembly 104 or perform any other manipulation task on the assembly 104. Figure 1A , the robot 102 is in a first position (eg, home position, starting position, etc.).

[0027] exist Figure 1A, the robot 102 may perform an assembly task including traveling to a first position, moving to a second position (e.g., an assembly item collection position, a temporary position / stage position, etc.) to obtain one of the collars 114 to be processed, traveling to a third position (e.g., a pre-manipulation position, a pre-task position, etc.) proximate one of the bolts 116 to be processed, and moving to a fourth position (e.g., a post-manipulation position, a post-task position, etc.) after coupling the collar 114 to the bolt 116. For example, the pre-task position may correspond to a position of the robot 102 proximate to the bolt 116 to be processed but before coupling the bolt 116 to the collar 114 to be processed. The post-task position may correspond to a position of the robot 102 after coupling the bolt 116 to the collar 114. In response to coupling the collar 114 to the bolt 116, the robot 102 may move and / or otherwise return to the first position to perform another assembly task.

[0028] Figure 1B Depicted Figure 1A The robot manager 100 controls Figure 1A The robot 102 performs the above combination Figure 1A For example, the robot manager 100 may guide and / or otherwise instruct the robot 102 to move from a first position (such as Figure 1A ) is moved to the second position, while moving to obtain the collar 114. Figure 1B , the robot 102 is in a third position. For example, the robot 102 is in a pre-task position near the bolt 116 to be processed. The pre-task position may correspond to a pre-manipulation or pre-torque position, or a position to which the robot 102 moves before torqueing the collar / nut 114 onto the bolt 116.

[0029] In operation, the robot manager 100 obtains a workflow including one or more assembly tasks to be performed by the robot 102. For example, the workflow may include a series of assembly tasks (e.g., 10 assembly tasks, 100 assembly tasks, 1000 assembly tasks, etc.) that are associated with one or more exemplary computer-based models 118. The computer-based model 118 may include a computer-aided design (CAD) model of the robot 102, the component 104, the collar / nut 114, the bolt, etc. The CAD model may be a two-dimensional (2-D) model, a three-dimensional (3-D) model, etc. For example, the model 118 may include position information (e.g., pose) of the robot 102, the component 104, etc. in a model environment corresponding to the physical environment 106 of the robot 102.

[0030] In some examples, the robot manager 100 determines that the workflow includes the robot 102 coupling the collar 114 to the bolt 116 at an assembly position corresponding to a position proximate to the bolt 116. Additionally or alternatively, the workflow may include different assembly tasks. The robot manager 100 may determine that the workflow includes a sequence of assembly tasks including a first assembly task associated with coupling a first of the collars 114 to a first of the bolts 116 at a first assembly position, a second assembly task associated with coupling a second of the collars 114 to a second of the bolts 116 at a second assembly position, etc., wherein the first assembly position is proximate to and / or otherwise proximate to the second assembly position, and the first assembly position is different from the second assembly position. Alternatively, the first assembly position may not be proximate to and / or otherwise proximate to the second assembly position.

[0031] In operation, the robot manager 100 determines an assembly position included in one or more models 118 associated with the bolt 116 and / or more generally associated with the assembly 104. For example, the robot manager 100 may determine an estimated pre-task position, an estimated post-task position, etc. The robot manager 100 determines a trajectory (e.g., a robot trajectory) including one or more joint states, waypoints, etc. based on the assembly position. In some examples, the joint state may include at least one of one or more angles, one or more coordinates, etc. associated with a position of the end effector 108, a joint (e.g., a connection point of two or more structural components of the robot 102), and / or more generally the robot 102. For example, a joint state of the robot 102 when the robot 102 is in a home position may correspond to an angle (in degrees, radians, etc.) of the end effector 108 relative to a base of the robot 102.

[0032] In some examples, the robot manager 100 determines a trajectory corresponding to moving the robot 102 from the home position to the temporary position, from the temporary position to an estimated pre-task position, etc. In such examples, the robot manager 100 may determine a first joint state associated with the home position, a second joint state associated with the temporary position, etc. The robot manager 100 may generate and send a command to the robot 102 based on the estimated pre-task pose of the robot 102 to move the robot from the home position to the estimated pre-task position. For example, the estimated pre-task pose may correspond to a pose of the robot 102 defined and / or otherwise represented by the model 118 after moving along the trajectory from the home position to the estimated pre-task position defined by the model 118 and / or otherwise included in the model 118.

[0033] In operation, before the robot manager 100 starts and / or otherwise executes a workflow, a user may guide the robot 102 from a first temporary position to a first pre-task position along a first trajectory to perform a first assembly task. The first temporary position may correspond to the position of the robot 102 when the first of the collars 114 is obtained. The first pre-task position may correspond to a first actual pre-task position of the robot 102. When the user moves the robot 102 to the first pre-task position, the user aligns and verifies the first pre-task position of the robot 102 to perform the first assembly task. When the user verifies the first pre-task position of the robot 102, the robot manager 100 calculates a first actual pre-task pose. After calculating the first actual pre-task pose, the robot manager 100 instructs the robot 102 to perform the first assembly task. When the robot 102 reaches an actual post-task position after performing the first assembly task, the robot manager 100 calculates the first actual post-task pose. The robot manager 100 calculates the positioning offset by comparing at least one of (1) a first actual pre-task pose to a first estimated pre-task pose or (2) a first actual post-task pose to a first estimated post-task pose, wherein the first estimated pre-task pose and the first estimated post-task pose are based on the model 118.

[0034] In operation, the robot 102 uses the positioning offset determined by the robot manager 100 to perform the rest of the workflow without user or other third-party assistance or intervention. For example, the robot 102 can perform a second assembly task, a third assembly task, and the like by using the positioning offset. For example, the robot manager 100 can instruct the robot 102 to move from the home position to the second temporary position to the second estimated pre-task position along the second trajectory, which in turn causes the robot 102 to move to a second actual pre-task position that is different from the second estimated pre-task position. In such an example, the robot manager 100 calculates a third trajectory to be attached to the second trajectory based on the positioning offset. For example, after the robot 102 completes the second trajectory, the robot manager 100 can instruct the robot 102 to move from the second actual pre-task position to the third estimated pre-task position along the third trajectory, which in turn causes the robot 102 to move to the third actual pre-task position, wherein the difference between the second and third actual pre-task positions corresponds to the positioning offset. When the robot 102 moves to the third actual pre-task position, the robot 102 performs the second assembly task. The robot 102 performs additional assembly tasks in the workflow in a similar manner. In some examples, the robot manager 100 updates the model 118 based on the positioning offset. For example, the robot manager 100 can update the model 118 to adjust the poses associated with unprocessed or yet-to-be-processed assembly tasks (e.g., the third assembly task, the fourth assembly task, etc.).

[0035] In some examples, the robot 102 travels and / or otherwise navigates to the assembly location and performs the assembly task. In other examples, the robot 102 cannot perform the assembly task based on the uncertainty of the positioning that meets the threshold (e.g., the positioning uncertainty threshold). The positioning uncertainty may correspond to the output of one or more probability functions that determine whether the robot 102 cannot perform the assembly task based on the position uncertainty at one or more waypoints along the robot trajectory. Position uncertainty may refer to the probabilistic correlation of the estimated posture and the actual posture determined in part or in whole by the measurement of the sensor 112, the verified joint state of the robot 102, etc. For example, the robot 102 may determine that performing the assembly task at the assembly location will produce a collision that may damage the robot 102, the collar 114, the bolt 116, etc. based on the determination of the positioning uncertainty. In other examples, the robot 102 cannot perform the assembly task due to the collision of the end effector 108 with the bolt 116, the component 104, etc. In some examples, in response to the positioning uncertainty meeting the threshold, the robot manager 100 calculates and / or otherwise determines the positioning offset (e.g., positioning adjustment, correction factor, etc.).

[0036] Figure 2 yes Figure 1A and Figure 1B The robot manager 100 is a block diagram of an exemplary embodiment of the robot manager 100. The robot manager 100 facilitates Figure 1A-1B The robot manager 100 generates commands and sends them to the robot 102 to guide the robot 102 to one or more positions based on a trajectory including joint states, waypoints, etc. determined from the model 118. In some examples, the robot manager 100 determines a correction factor when the user guides the robot 102 to a verified actual pre-task position. In other examples, the robot manager 100 determines a correction factor when a positioning uncertainty associated with an assembly task satisfies a threshold. The robot manager 100 updates the model 118 based on the correction factor (as described below in conjunction with the model processor 220) to facilitate operation of subsequent assembly tasks. Figure 2 In the illustrated example, the robot manager 100 includes an example workflow director 210 , an example model processor 220 , an example gesture engine 230 , an example command generator 240 , and an example database 250 .

[0037] exist Figure 2 In the illustrated example, the robot manager 100 includes a workflow director 210 to obtain a workflow including one or more assembly tasks to be performed by the robot 102. In some examples, the workflow director 210 obtains the workflow from an example network 260. Figure 2The network 260 of the illustrated example is the Internet. However, the network 260 can be implemented using any suitable wired and / or wireless network, including, for example, one or more data buses, one or more local area networks (LANs), one or more wireless LANs, one or more process control networks, one or more cellular networks, one or more private networks, one or more public networks, etc. In some examples, the network 260 enables the robot manager 100 to communicate with external computing devices (e.g., databases, servers, etc.) coupled to the network 260.

[0038] In some examples, the workflow director 210 includes a means for importing the models 118 associated with the workflow. For example, the workflow director 210 can obtain a first CAD model associated with the robot 102, a second CAD model associated with the component 104, a third CAD model associated with the environment of the robot 102 and / or the component 104, etc. The workflow director 210 can synthesize and / or otherwise combine two or more of the models 118 into a single model.

[0039] In some examples, the workflow director 210 includes means for determining an order or sequence of operations associated with the workflow. For example, the workflow director 210 may determine an order or sequence of operations associated with coupling the first of the collars 114 to the Figure 1A-1B 116, followed by a sequence of a second assembly task associated with coupling a second one of the collars 114 to the second one of the bolts 116. In other examples, the workflow director 210 may determine that there are no additional assembly tasks to be performed by its robot 102.

[0040] exist Figure 2 In the illustrated example, the robot manager 100 includes a model processor 220 to coordinate and / or otherwise facilitate movement of the robot 102 based on the model 118. In some examples, the model processor 220 determines an assembly position, such as an estimated pre-task position, an estimated post-task position, etc., based on the model 118. For example, the model processor 220 can determine coordinates associated with the estimated pre-task position in a modeling environment (e.g., model space, a CAD environment, etc.). In other examples, the model processor 220 can determine coordinates associated with the original position of the robot 102, one or more temporary positions, one or more of the collars 114, one or more of the bolts 116, etc., or with the model processor 220. Figure 1A-1B The coordinates associated with the position of the robot 102 and / or any other components or objects associated with the assembly 104 .

[0041] In some examples, the model processor 220 includes a device for determining a trajectory (e.g., a robot trajectory) associated with an assembly task. For example, the model processor 220 can identify one or more joint states, waypoints, etc. included in the trajectory, which can be used to move the robot 102 from an original position, a temporary position, etc. to a pre-task position. In some examples, the robot 102 can satisfy the appropriate manipulation of an assembly item (e.g., a collar 114) through more than one trajectory. For example, the model processor 220 can select a first assembly task to be processed, which includes manipulating a first assembly item (e.g., the first of the collars 114). In response to selecting the first assembly task, the model processor 220 can generate a trajectory set that includes one or more trajectories associated with the first assembly task.

[0042] In some examples, the model processor 220 includes a device for identifying a trajectory from the generated trajectory set to perform a first assembly task by determining that the identified trajectory has a target state that is close to the post-task state of the previously processed assembly item. For example, the model processor 220 can identify the trajectory by determining that the posture of the robot 102 at the post-task position immediately following the previous assembly task can produce a more reliable trajectory than using the posture of the robot 102 from several previous tasks. In some examples, the target state refers to the state of the assembly item with desired attributes, such as the desired position, the desired torque value when installed, etc. For example, the target state of one of the collars 114 can correspond to being connected to one of the bolts 116 at the desired position and / or the desired torque. For example, the desired position can be based on the position of the bolt 116 in the model 118. In response to identifying the trajectory, the model processor 220 generates a joint state associated with the trajectory. For example, each joint state can correspond to a waypoint, the properties of the robot 102 at the waypoint, etc.

[0043] In some examples, the model processor 220 includes a device for determining an estimated robot pose based on ground truth, where the ground truth corresponds to the most accurate measurement available to the robot 102. The model processor 220 can use the model 118 as a ground truth for determining an estimated robot pose (e.g., an estimated pre-task pose, an estimated post-task pose, etc.). The model processor 220 can determine the estimated robot pose by calculating coordinates in the model 118 of a simulation of moving the robot 102 using the trajectory. For example, the model processor 220 can use the trajectory to simulate the robot 102 moving from a home position to an estimated pre-task position associated with the first of the bolts 116. When the robot 102 moves along the trajectory in the simulation, the model processor 220 can identify the coordinates of the home position, the estimated pre-task position, etc. by determining the coordinates of the robot 102. As a result, the model processor 220 can determine an estimated pose of the end effector 108 and / or the robot 102 by identifying the coordinates of the end effector 108 and / or the robot 102 when simulating the trajectory using the model 118 as the ground truth.

[0044] In some examples, the model processor 220 includes a device for updating the model 118 based on the correction factor. For example, the model processor 220 can update the model 118 by generating a set of trajectories using the correction factor, which can be attached to the trajectory associated with performing the assembly task in the workflow. For example, the model processor 220 can generate a first set of trajectories (a first set of trajectories) including a first trajectory that the robot 102 will use to move from the original position to the first temporary position, to the first estimated pre-task position, etc. during the first assembly task, which in turn can move the robot 102 to the first actual pre-task position. The model processor 220 can generate a second set of trajectories (a second set of trajectories) including a second trajectory that the robot 102 will use to move from the first actual pre-task position to the second actual pre-task position, wherein the difference in motion from the first to the second actual pre-task position is based on the correction factor. For example, the robot 102 can use the second set of trajectories to attach to the first set of trajectories when performing the assembly task in the workflow. In other examples, the model processor 220 generates a third set of trajectories (a third set of trajectories) that combines or synthesizes the first set of trajectories and the second set of trajectories, wherein the third set of trajectories replaces the first set of trajectories and the second set of trajectories.

[0045] In some examples, the model processor 220 includes means for determining an actual pose of the robot 102 when the robot 102 moves to the estimated pre-task position. For example, the model processor 220 may determine the actual pre-task pose of the robot 102 by applying measurements from the sensor 112 to the home position or the temporary position to determine a change in rotation, translation, etc. of the end effector 108 from the home position or the temporary position. The model processor 220 may determine the actual pre-task pose by applying the measurements to the home pose, the temporary pose, etc. to calculate one or more rotation quaternions, one or more translation quaternions, etc., and / or a combination thereof.

[0046] In some examples, the robot manager 100 determines the actual pose of the robot 102 when the assembly task is (successfully) completed (e.g., the robot 102 is in the post-task position). For example, when the robot 102 has begun coupling the first of the collars 114 to the first of the bolts 116 at the pre-task position and moves to the post-task position after completing the coupling, the robot manager 100 can determine the actual post-task pose of the robot 102. In such an example, when the torque measurement associated with the coupling satisfies the torque threshold (e.g., the first of the collars 114 has bottomed out onto the first of the bolts 116), the robot manager 100 determines the actual post-task pose of the robot 102 at the actual post-task position. The robot manager 100 can determine the actual post-task pose by applying measurements from the sensors 112 to the original pose, the interim pose, etc. to generate one or more rotation quaternions, one or more translation quaternions, etc., and / or combinations thereof.

[0047] exist Figure 2 In the illustrated example, the robot manager 100 includes a pose engine 230 to determine a correction factor or positioning offset to facilitate the robot 102 to perform the assembly task. In some examples, the pose engine 230 determines the correction factor after the user moves the robot to a verified actual pre-task position, a verified actual post-task position, etc. In some examples, the pose engine 230 determines the correction factor 102 based on whether the assembly task has been completed. For example, the pose engine 230 can determine that the uncertainty of the positioning associated with the end effector 108 meets the positioning uncertainty threshold. In other examples, the pose engine 230 can determine that the uncertainty of the positioning associated with the end effector 108 does not meet the positioning uncertainty threshold. In such an example, the pose engine 230 can determine that the assembly task has been successfully performed when the torque value associated with the end effector 108 and the collar 114 meets the manipulation threshold (e.g., the torque threshold). For example, when the torque value is greater than the torque threshold and therefore the torque value meets the torque threshold, the pose engine 230 can determine that the collar 114 has reached the lowest point on the bolt 116.

[0048] In some examples, the posture engine 230 includes a device for determining whether the positioning uncertainty meets the positioning uncertainty threshold. For example, the posture engine 230 can determine a first joint state of the robot 102 corresponding to the first temporary position, and determine a second joint state of the robot 102 at the pre-task position. In some examples, the posture engine 230 determines that the correlation between the uncertainty of the first joint state and the second joint state meets the positioning uncertainty threshold. For example, the correlation between the uncertainties of the two joint states can increase according to the distance between the two joint states in Euclidean space. In such an example, the posture engine 230 can determine that the reliability of the position information associated with the (subsequent) joint state increases with the similarity of the position information associated with the current or given joint state of the robot 102.

[0049] In some examples, gesture engine 230 includes means for determining whether a correction factor has been determined when the assembly task has not been successfully performed. For example, gesture engine 230 may query database 250 for the correction factor. In some examples, gesture engine 230 may determine that the correction factor has been determined based on receiving and / or otherwise obtaining the correction factor from database 250. In other examples, gesture engine 230 may determine that the correction factor has not been determined based on receiving and / or otherwise obtaining a null value, a null index, etc. from database 250 in response to a query.

[0050] In some examples, gesture engine 230 includes means for determining a correction factor. For example, gesture engine 230 can calculate a correction factor corresponding to a positioning adjustment, as described in equation (1) below:

[0051] correction_factor=robot_base_to_actual_pose*robot_base_to_expected_pose.inverse()

[0052] Equation (1)

[0053] In the example of equation (1) above, the term "robot_base_to_actual_pose" corresponds to the pose of the robot 102 when the robot 102 completes the assembly task. Figure 1A-1BThe robot_base_to_actual_pose is determined when a torque value associated with one of the collars 114 satisfies a torque threshold (e.g., the torque value is greater than the torque threshold). In the example of equation (1) above, the term "robot_base_to_expected_pose" corresponds to the pose of the robot 102 determined based on the model 118. For example, the term "robot_base_to_expected_pose" may correspond to the expected pose of the robot 102 when the robot 102 has completed the assembly task based on the model 118.

[0054] In the example of equation (1) above, the term "correction_factor", the term "robot_base_to_actual_pose", and the term "robot_base_to_expected_pose" are matrices. In some examples, the matrix includes at least one of one or more complex numbers or one or more real numbers. For example, one or more terms included in the example of equation (1) above may correspond to a matrix representing a quaternion, the quaternion including at least one of one or more complex numbers or one or more real numbers. In some examples, one or more of the terms are homogeneous matrices (e.g., a homogeneous transformation matrix, a 4×4 homogeneous transformation matrix, etc.). For example, one or more terms included in the example of equation (1) may correspond to a matrix including one or more coordinates associated with the position information of the robot 102. For example, the posture engine 230 may determine the correction factor by multiplying the inverse of a first homogeneous matrix associated with the first posture of the robot 102 corresponding to the term "robot_base_to_actual_pose" and a second homogeneous matrix associated with the second posture of the robot 102 corresponding to "robot_base_to_expected_pose". Alternatively, the entries may be matrices of different sizes and / or types.

[0055] exist Figure 2In the illustrated example, the robot manager 100 includes a command generator 240 to generate and / or otherwise determine commands, instructions, etc., and send the commands, instructions, etc. to the robot 102 to control the robot 102. For example, the command generator 240 generates commands to perform an assembly task. For example, the command generator 240 can generate one or more commands to instruct the robot 102 to couple one of the collars 114 with one of the bolts 116. The command generator 240 can generate a first command to move the robot 102 from the original position to the temporary position, a second command to obtain one of the collars 114 at the temporary position, a third command to move from the temporary position to the pre-task position, etc. In some examples, the command generator 240 generates commands based on an estimated pre-task position generated using the ground truth of the model 118.

[0056] In some examples, the command generator 240 includes a device for controlling the robot 102 to perform the assembly task using the correction factor. For example, the command generator 240 may send a first command to one or more processors controlling the robot 102 to move the robot 102 to a first estimated pre-task position, thereby moving the robot 102 to a first actual pre-task position. In some examples, the command generator 240 may send a second command to the one or more processors controlling the robot 102 to move the robot 102 to a second actual pre-task position, wherein the difference between the first actual pre-task position and the second actual pre-task position is determined by the correction factor. In other examples, the posture engine 230 uses the correction factor based on determining that the uncertainty of the positioning associated with the assembly position satisfies the positioning uncertainty threshold. In response to the positioning uncertainty satisfying the positioning uncertainty threshold, the command generator 240 may generate a second command to perform the assembly task.

[0057] In some examples, the command generator 240 includes means for controlling the robot 102 using an assistance-based system when the uncertainty of the positioning associated with the assembly position satisfies a positioning uncertainty threshold. For example, the command generator 240 can direct the robot 102 to enter and / or otherwise transition to an assistance mode. For example, the robot 102 can enter the assistance mode when the robot 102 moves to a position close to the assembly position based on the trajectory, but the difference between the adjacent position and the assembly position produces a positioning uncertainty that satisfies the positioning uncertainty threshold.

[0058] In some examples, when in the assist mode, the user can adjust the posture of the robot 102 from the proximity position to the assembly position. In other examples, when in the assist mode, the command generator 240 can instruct the robot 102 to move from the proximity position to the assembly position based on output from a sensor such as a camera-based system, an encoder, etc., or any other type of sensor. For example, the camera-based system can identify a difference (e.g., a position difference) between the proximity position and the assembly position. The command generator 240 can generate a command to move the robot 102 to the assembly position based on the difference.

[0059] In some examples, the command generator 240 includes a device for manipulating the assembly item at the assembly location. For example, the command generator 240 can generate a command to direct the robot 102 to use the end effector 108 to couple the collar 114 to the bolt 116 at the assembly location. When the sensor measurement meets the threshold, the command generator 240 can stop manipulating the assembly item at the assembly location. For example, the command generator 240 can obtain a torque measurement from the sensor 112, compare the torque measurement to a torque threshold, and when the torque measurement is greater than the torque threshold, generate a command to stop the end effector 108 from coupling the collar 114 to the bolt 116. When the sensor measurement is greater than the threshold (e.g., the sensor measurement meets the threshold), the command generator 240 can direct the posture engine 230 to determine a posture of the robot 102 that corresponds to the actual robot posture.

[0060] exist Figure 2 In the illustrated example, the robot manager 100 includes a database 250 to record and / or otherwise store data. Figure 2 The database 250 includes Figure 1A-1B The database 250 records and / or otherwise stores one or more of the models 118 associated with the robot 102, a sequence of one or more assembly tasks, one or more trajectories, measurements from the sensors 112, one or more thresholds, one or more correction factors, one or more poses, etc. The database 250 may be implemented by a volatile memory (e.g., synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS dynamic random access memory (RDRAM), etc.) and / or a non-volatile memory (e.g., flash memory). The database 250 may additionally or alternatively be implemented by one or more double data rate (DDR) memories, such as DDR, DDR2, DDR3, mobile DDR (mDDR), etc. The database 250 may additionally or alternatively be implemented by one or more mass storage devices such as hard disk drive (s), optical disk drive (s), digital universal disk drive (s), etc.

[0061] And in Figure 2 In the example shown, the database 250 is shown as a single database, and the database 250 can be implemented by any number and / or (one or more) types of databases. In addition, the data stored in the database 250 can be in any data format, such as binary data, comma-delimited data, model data (e.g., CAD data), tab-delimited data, structured query language (SQL) structures, etc. In some examples, the database 250 can be cloud-based to enable synchronous retrieval and updating by one or more computing devices communicatively coupled to the robot manager 100 via the network 260.

[0062] To achieve Figure 1A-1B An example embodiment of the robot manager 100 is shown in FIG. Figure 2 It is shown in Figure 2 One or more of the elements, processes, and / or devices shown in the example may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other manner. In addition, the example workflow guide 210, the example model processor 220, the example gesture engine 230, the example command generator 240, the example database 250, and / or more generally, Figure 1A-1B The exemplary robot manager 100 may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, the exemplary workflow director 210, the exemplary model processor 220, the exemplary posture engine 230, the exemplary command generator 240, the exemplary database 250, and / or more generally, any of the exemplary robot manager 100 may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, graphics processing units (GPUs), digital signal processors (DSPs), application specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field programmable logic devices (FPLDs). When any apparatus or system claim of this patent is read to cover a purely software and / or firmware implementation, at least one of the exemplary workflow director 210, the exemplary model processor 220, the exemplary posture engine 230, the exemplary command generator 240, and / or the exemplary database 250 is hereby expressly defined as comprising a non-transitory computer-readable storage device or storage disk, such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc., including software and / or firmware. In addition, Figure 1A-1B The example robot manager 100 may include one or more elements, processes and / or devices as Figure 2The present invention may be in addition to or replace those elements, processes and / or devices shown in the drawings, and / or may include any or all of the elements, processes and devices shown in the drawings. As used herein, the phrase "communication", including its variations, includes direct communication and / or indirect communication through one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or continuous communication, but in addition, also includes selective communication at periodic intervals, scheduled intervals, non-periodic intervals and / or one-time events.

[0063] Indicates that it is used to implement Figure 1A-Figure 2 A flowchart of example hardware logic, machine readable instructions, hardware implemented state machines, and / or any combination thereof of the robot manager 100 is provided in Figure 3-Figure 5 The machine-readable instructions may be an executable program or a portion of an executable program for execution by a computer processor, such as the following in conjunction with Figure 6 The processor 612 shown in the example processor platform 600 discussed above. The program may be embodied in software stored on a non-transitory computer-readable storage medium, such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray disk, or memory associated with the processor 612, but the entire program and / or portions thereof may alternatively be executed by a device other than the processor 612 and / or embodied in firmware or dedicated hardware. In addition, although reference is made to Figure 3-Figure 5 The flowchart shown in describes an example program, but many other methods of implementing the example robot manager 100 may be used instead. For example, the order of execution of the boxes may be changed, and / or some of the boxes described may be changed, eliminated, or combined. Additionally or alternatively, any or all of the boxes may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) that are configured to perform the corresponding operations without executing software or firmware.

[0064] As mentioned above, Figure 3-Figure 5 The example processes of can be implemented using executable instructions (e.g., computer and / or machine readable instructions) stored on a non-transitory computer and / or machine readable medium (e.g., a hard drive, flash memory, read-only memory, compact disk, digital versatile disk, cache, random access memory, and / or any other storage device or storage disk in which information is stored for any duration (e.g., an extended period of time, permanently, for a short period of time, temporarily buffered and / or cached information). As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and / or storage disk and to exclude propagating signals and to exclude transmission media.

[0065] "Include" and "comprising" (and all their forms and tenses) are used herein as open-ended terms. Thus, whenever a claim adopts any form of "include" or "comprising" (e.g., includes, contains, has, etc.) as a preamble or in any type of claim, it should be understood that additional elements, items, etc. may be present without falling outside the scope of the corresponding claim or reference. As used herein, when the phrase "at least" is used as a transitional term, such as in the preamble of a claim, it is open-ended in the same manner as the terms "include" and "comprising" are open-ended. The term "and / or" when used, for example, in a form such as A, B, and / or C refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, and (7) A and B and C. As used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" is intended to mean embodiments that include any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" is intended to mean embodiments that include any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the execution or performance of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A and B" is intended to mean embodiments that include any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing the execution or performance of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A or B" is intended to mean embodiments that include any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0066] Figure 3 is a flow chart representing exemplary machine readable instructions 300 that may be executed to implement Figure 1A-Figure 2 The machine-readable instructions 300 begin at block 302, where the robot manager 100 obtains a workflow including an assembly task to be performed by a robot. For example, the workflow director 210 ( Figure 2 ) can be obtained including Figure 1A-1B A workflow for performing one or more assembly tasks on component 104 .

[0067] At block 304, the robot manager 100 imports a model associated with the workflow. For example, the workflow director 210 may import a model associated with the workflow. Figure 1A-1BRobot 102, Figure 1A-1B Component 104 and other associated Figure 1A-Figure 2 One or more of the models 118 .

[0068] At block 306, the robot manager 100 determines the assembly locations included in the model. For example, the model processor 220 ( Figure 2 ) may determine, based on the model 118 , a first estimated pre-mission position associated with a first one of the bolts 116 , a second estimated pre-mission position associated with a second one of the bolts 116 , and so on, which are coupled to the assembly 104 .

[0069] At block 308, the robot manager 100 determines a robot trajectory based on the assembly position. For example, the model processor 220 can determine a trajectory associated with the first estimated pre-task position. The model processor 220 can determine a first joint state associated with the robot 102 at the first temporary position to obtain one of the collars 114, a second joint state associated with the robot 102 at the first estimated pre-task position, and the like, and / or any other joint states associated with one or more other positions or waypoints included in the trajectory.

[0070] At block 310, the robot manager 100 commands the robot to perform an assembly task. For example, the command generator 240 ( Figure 2 ) may instruct the robot 102 to move from the first temporary position to the first estimated pre-task position to Figure 1A-1B One of the collars 114 is coupled to Figure 1A-1B In response to the instruction, the robot 102 moves to a first actual pre-task position that is different from the first estimated pre-task position.

[0071] At block 312, the robot manager 100 determines whether a correction factor has been determined. For example, the posture engine 230 ( Figure 2 ) can query the database 250 ( Figure 2 ) to determine whether the correction factor has been calculated. In such examples, gesture engine 230 can retrieve and / or otherwise obtain the correction factor from database 250 in response to the query. In other examples, gesture engine 230 can determine that the correction factor has not been determined based on retrieving and / or otherwise obtaining a null value from database 250 in response to the query.

[0072] If the robot manager 100 determines at box 312 that the correction factor has been determined, control proceeds to box 316 to command the robot to perform the assembly task using the correction factor. If the robot manager 100 determines at box 312 that the correction factor has not been determined, at box 314, the robot manager 100 determines the correction factor. For example, the posture engine 230 can direct the robot 102 to enter the auxiliary mode. The user can move the robot 102 to the correct or verified pre-task position. When the user verifies that the robot 102 moves to the correct position to perform the first assembly task, the posture engine 230 can determine the first actual pre-task posture of the robot 102. The posture engine 230 can determine the correction factor based on a comparison of the first actual pre-task posture of the robot 102 with the first estimated pre-task posture of the robot 102. In other examples, the posture engine 230 can call the command generator 240 to move the robot 102 to the correct or verified position based on output from a camera system or any other type of auxiliary sensor-based system.

[0073] In response to determining the correction factor at block 314, the robot manager 100 commands the robot to perform the assembly task based on the correction factor at block 316. For example, the command generator 240 may generate and send a command to direct the robot 102 to couple the collar 114 to the bolt 116 at a first pre-task position based on an additional trajectory to the first trajectory determined by the model processor 220 using the correction factor.

[0074] At block 318, the robot manager 100 updates the model based on the correction factor. For example, the model processor 220 may update one or more of the models 118 by generating a set of trajectories to be appended to the trajectories generated at block 308 based on the correction factor. In other examples, the model processor 220 may update one or more of the models 118 by generating a set of trajectories that synthesize and / or otherwise combine the trajectories generated at block 308 with the appended trajectories. Figure 4 An example process that may be used to implement block 318 is described.

[0075] At block 320, the robot manager 100 determines whether to perform another assembly task. For example, the workflow director 210 may determine that there is another assembly task to be processed. In other examples, the workflow director 210 may determine that the workflow has been completed and there are no additional assembly tasks to be processed. If at block 320, the robot manager 100 determines to perform another task, control returns to block 310 to command the robot to perform another assembly task. If at block 320, the robot manager 100 determines not to perform another task, then Figure 3 The machine readable instructions 300 end.

[0076] Figure 4 is a flow chart representing example machine readable instructions 400 that may be executed by Figure 1A-Figure 2 An example of the robot manager 100 uses to update the model based on the correction factor. Figure 4 The example process can be used to implement Figure 4 The operation of box 318.

[0077] The machine-readable instructions 318 begin at block 402 where the robot manager 100 updates a trajectory associated with the assembly task based on the correction factor. For example, the model processor 220 ( Figure 2 ) can update the trajectory associated with the first assembly task based on the correction factor. In such an example, the model processor 220 can generate a new trajectory to append to the current trajectory to move the robot 102 from the first actual pre-task position to the second actual pre-task position.

[0078] At block 404, the robot manager 100 generates a set of trajectories associated with the next assembly task. For example, the model processor 220 may generate a set of trajectories associated with the next assembly task. Figure 1A-1B The second of the collars 114 to Figure 1A-1B A set of one or more trajectories associated with the second one of the bolts 116 in FIG. The set of one or more trajectories may correspond to the command generator 240 ( Figure 2 ) is used to correct a potential additional trajectory to be used to perform a second assembly task, wherein the correction enables the robot 102 to perform the second assembly task without user intervention or input from an auxiliary sensor system.

[0079] At block 406, the robot manager 100 determines a trajectory whose target state (position) is closest to the actual robot pose. For example, when performing the first assembly task, the model processor 220 may select one of the trajectories in the set based on the target state associated with the selected trajectory, which is closest to the actual pre-task pose, the actual post-task pose, etc. of the robot 102.

[0080] At block 408, the robot manager 100 generates a new joint state by applying the correction factor to the target state. For example, the model processor 220 may generate a third joint state associated with the robot 102 moving from the second actual pre-task position to the third actual pre-task position, a fourth joint state associated with the robot 102 moving from the third actual pre-task position to the second actual post-task position, and so on.

[0081] At block 410, the robot manager 100 determines whether to generate a set of trajectories for another assembly task. For example, the model processor 220 may determine to generate a set of trajectories for the third assembly task. In other examples, the model processor 220 may determine to generate a set of trajectories for each assembly task included in the workflow to determine a new joint state based on the correction factor. In other examples, the model processor 220 may determine not to generate a set of trajectories for another assembly task.

[0082] If at block 410, the robot manager 100 determines to generate a set of trajectories for another assembly task, control returns to block 404 to generate a set of trajectories associated with the next or subsequent assembly task. If at block 410, the robot manager 100 determines not to generate a set of trajectories for another assembly task, then Figure 4 The machine readable instructions 318 return to Figure 4 Block 320 of the example of FIG. 5 is used to determine whether to perform another assembly task.

[0083] Figure 5 is another flow chart representing exemplary machine readable instructions 500 that may be executed to implement Figure 1A-Figure 2 The robot manager 100 performs the assembly task. The machine-readable instructions begin at box 502, where the robot manager 100 selects an assembly task of interest to be processed. For example, the workflow guide 210 ( Figure 2 ) can select an assembly task, including coupling the first of the collars 114 to Figure 1A-1B The first of the bolts 116 to be processed.

[0084] At block 504, the robot manager 100 uses the model as ground truth to determine (one or more) estimated robot poses at the assembly location corresponding to the assembly item. For example, the model processor 220 ( Figure 2 ) can determine the coordinates of the first one of the collars 114 based on the model 118, Figure 1A-1B The model processor 220 may determine a trajectory including a first joint state of the robot 102 at a first position (e.g., a home position, a first temporary position, etc.), a second joint state associated with the robot 102 at a second position (e.g., a pre-task position associated with the first of the bolts 116), etc., and / or any other joint states associated with one or more other positions or waypoints along the trajectory. The model processor 220 may determine coordinates associated with a pose of the robot 102 at the second position based on the trajectory and identify the pose as a first estimated pre-task pose.

[0085] At block 506, the robot manager 100 navigates the robot to the assembly location based on the estimated robot pose(s). For example, the command generator 240 ( Figure 2 ) can generate and send commands to the robot 102 to move the robot 102 from a first position to a second position to reach a first estimated pre-task pose, thereby moving the robot 102 to a first actual pre-task position.

[0086] At block 508, the robot manager 100 determines whether the positioning uncertainty satisfies a threshold. For example, the posture engine 230 ( Figure 2 ) can determine that the positioning uncertainty associated with the correlation between the first joint state and the second joint state of the robot 102 satisfies the positioning uncertainty threshold.

[0087] If at block 508, the robot manager 100 determines that the positioning uncertainty does not meet the threshold, control proceeds to block 516 to manipulate the assembly item at the assembly location. If at block 508, the robot manager 100 determines that the positioning uncertainty meets the threshold, then at block 510, the robot manager 100 determines whether a correction factor has been determined. For example, the gesture engine 230 may query the database 250 ( Figure 2 ) to determine whether the correction factor has been calculated. In such examples, gesture engine 230 can retrieve and / or otherwise obtain the correction factor from database 250 in response to the query. In other examples, gesture engine 230 can determine that the correction factor has not been determined based on retrieving and / or otherwise obtaining a null value from database 250 in response to the query.

[0088] If at box 510, the robot manager 100 determines that the correction factor has been determined, control proceeds to box 514 to adjust the robot pose to the assembly position using the correction factor. If at box 510, the robot manager 100 determines that the correction factor has not been determined, at box 512, the robot manager 100 uses an assistance-based system to adjust the robot pose to the assembly position. For example, the posture engine 230 can instruct the robot 102 to enter an assistance mode. The user can move the robot 102 to the correct position to perform the first assembly task. In other examples, when such a system is available for the robot 102, the posture engine 230 can call the command generator 240 to move the robot 102 to the correct position using a camera-based system or any other sensor-based system.

[0089] In response to adjusting the robot pose to the assembly position using the assistance-based system at block 512 or adjusting the robot pose to the assembly position using the correction factor at block 512, the robot manager 100 manipulates the assembly item at the assembly position at 516. For example, the command generator 240 may generate and send a command to the robot 102 to direct the robot 102 to couple the collar 114 to the bolt 116 at the second position.

[0090] At block 518, the robot manager 100 determines whether the sensor measurements satisfy the manipulation threshold. For example, the command generator 240 may select Figure 1A-1B The sensor 112 obtains a torque measurement, compares the torque measurement to a torque threshold, and generates a command to stop coupling the collar 114 and the bolt 116 when the torque measurement is greater than the torque threshold.

[0091] If at block 518, the robot manager 100 determines that the sensor measurements do not satisfy the manipulation threshold, control waits for the robot 102 to complete the articulation at block 518. If at block 518, the robot manager 100 determines that the sensor measurements satisfy the manipulation threshold, the robot manager 100 determines the actual robot pose at block 520. For example, when the torque measurements satisfy the torque threshold, the command generator 240 may call the pose engine 230 to determine a pose of the robot 102 that corresponds to a first actual post-task pose of the robot 102.

[0092] At block 522, the robot manager 100 determines a correction factor based on the actual robot pose and the estimated robot pose. For example, the pose engine 230 may determine and / or otherwise calculate a positioning adjustment based on the first actual pre-task pose, the first actual post-task pose, the first estimated pre-task pose, the first estimated post-task pose, etc., using the example of equation (1) above.

[0093] At block 524, the robot manager 100 updates the model based on the correction factor. For example, the model processor 220 may update the model using Figure 4 The process updates the model 118 based on the correction factor. In some examples, the model processor 220 updates the model 118 by updating information associated with one or more subsequent assembly tasks. For example, the model processor 220 can update the model 118 to generate a set of trajectories to be attached and / or otherwise applied to the trajectory for performing the assembly task.

[0094] At block 526, the robot manager 100 determines whether to select another assembly task of interest to be processed. For example, the workflow director 210 may determine to perform another assembly task including coupling the second collar 114 to the second bolt 116 at the second assembly position. In other examples, the workflow director 210 may determine that there is no other assembly task of interest to be processed. If at block 526, the robot manager 100 determines to select another assembly task of interest to be processed, control returns to block 502 to select another assembly task of interest to be processed, otherwise Figure 5 The machine readable instructions 500 end.

[0095] Figure 6 is constructed to execute Figure 3-Figure 5 Instructions to implement Figure 1A-Figure 2 The processor platform 600 can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smartphone, a tablet computer such as an iPad, etc.). TM ), a robotic computing platform, or any other type of computing device.

[0096] The processor platform 600 of the illustrated example includes a processor 612. The processor 612 of the illustrated example is hardware. For example, the processor 612 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor can be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 612 implements the example workflow director 210, the example model processor 220, the example gesture engine 230, the example command generator 240, and / or more generally, Figure 1A-Figure 2 An example robot manager 100 is shown.

[0097] The processor 612 of the illustrated example includes a local memory 613 (e.g., a cache). The processor 612 of the illustrated example communicates with a main memory including a volatile memory 614 and a non-volatile memory 616 via a bus 618. The volatile memory 614 may be comprised of synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), Dynamic Random Access Memory The non-volatile memory 616 may be implemented by flash memory and / or any other desired type of memory device. Access to the main memory 614, 616 is controlled by a memory controller.

[0098] The processor platform 600 of the illustrated example also includes an interface circuit 620. The interface circuit 620 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), a Bluetooth interface, a near field communication (NFC) interface, and / or a PCI express interface.

[0099] In the example shown, one or more input devices 622 are connected to the interface circuit 620. The input devices 622 allow a user to enter data and / or commands into the processor 612. The input devices 622 may be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, buttons, a mouse, a touch screen, a track pad, a trackball, an isometric device, and / or a voice recognition system.

[0100] One or more output devices 624 are also connected to the interface circuit 620 of the illustrated example. The output device 624 can be implemented, for example, by a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker). Therefore, the interface circuit 620 of the illustrated example typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0101] The interface circuitry 620 of the illustrated example also includes communication devices, such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate data exchange with external machines (e.g., any type of computing device) via a network 626. In this example, the network 626 implements Figure 2 An example network 260. Communications may be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a field wireless system, a cellular telephone system, or the like.

[0102] The processor platform 600 of the illustrated example also includes one or more mass storage devices 628 for storing software and / or data. Examples of such mass storage devices 628 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives. In this example, the one or more mass storage devices 628 implement Figure 2 An example database 250 is provided.

[0103] Figure 3-Figure 5The machine-executable instructions 632 may be stored in the mass storage device 628, in the volatile memory 614, in the non-volatile memory 616, and / or on a removable, non-transitory computer-readable storage medium (eg, a CD or DVD).

[0104] As can be appreciated from the foregoing, example methods, apparatuses, and articles of manufacture for improving robot control have been disclosed. The disclosed methods, apparatuses, and articles of manufacture reduce a certain amount of sensors and sensor-based systems to facilitate the operation of the robot by using information associated with the robot's end effector to determine correction factors to be applied to assembly tasks performed by the robot. Sensor-based systems such as camera-based systems require an increased amount of computing and storage resources to function, and the disclosed methods, apparatuses, and articles of manufacture reduce and / or otherwise eliminate the need for such systems. Compared to existing robots, the disclosed methods, apparatuses, and articles of manufacture can reduce the amount of computing and / or storage resources required to facilitate the operation of the robot. For example, the disclosed methods, apparatuses, and articles of manufacture can allocate unused resources to perform additional processing tasks, and thus improve and / or otherwise increase the capabilities or functions of one or more processors associated with the robot. Therefore, the disclosed methods, apparatuses, and articles of manufacture relate to one or more improvements in computer functions.

[0105] The following relates to other examples disclosed herein.

[0106] Example 1 includes an apparatus comprising a command generator for: instructing a robot to move an end effector from a temporary position to an estimated pre-task position to perform a first task based on a first posture of the robot, the first posture being based on a model; when the robot is to move to a second actual pre-task position, adjusting the robot to a first actual pre-task position to perform the first task, the first actual pre-task position being close to the estimated pre-task position; and guiding the robot to perform a second task based on a correction factor, the correction factor being determined by: determining a second posture of the robot, the second posture corresponding to position information associated with the robot at a post-task position, the post-task position corresponding to position information associated with the robot after performing the first task and calculating the correction factor based on the first posture and the second posture.

[0107] Example 2 includes the apparatus of Example 1, wherein the first task includes the robot coupling a first assembly item to a second assembly item.

[0108] Example 3 includes the apparatus of Example 1 or 2, wherein the first pose is a first homogeneous matrix and the second pose is a second homogeneous matrix, and the correction factor is calculated by multiplying the second homogeneous matrix by the inverse of the first homogeneous matrix.

[0109] Example 4 includes the apparatus of Example 3, wherein the first homogeneous matrix and the second homogeneous matrix are 4×4 homogeneous matrices.

[0110] Example 5 includes the apparatus described in any of Examples 1-4, further comprising a model processor for updating the model based on a correction factor by generating a set of trajectories associated with a second task, determining one of the trajectories in the set whose target state is closest to the second posture, and generating one or more joint states associated with one of the trajectories by applying the correction factor to the target state.

[0111] Example 6 includes the apparatus of any one of Examples 1-5, further comprising a model processor for determining the estimated pre-task position by using the model as a ground truth to determine coordinates of the estimated pre-task position.

[0112] Example 7 includes the apparatus of any of Examples 1-6, wherein the command generator is used to adjust the robot to the first actual pre-task position by at least one of obtaining output from a camera system or obtaining sensor measurements from an encoder.

[0113] Example 8 includes a non-transitory computer-readable storage medium containing instructions, which, when executed, cause the machine to at least: instruct a robot to move an end effector from a temporary position to an estimated pre-task position to perform a first task based on a first posture of the robot, the first posture being based on a model; when the robot is to move to a second actual pre-task position, adjust the robot to a first actual pre-task position to perform the first task, the first actual pre-task position being close to the estimated pre-task position; after performing the first task, determine a second posture of the robot, the second posture corresponding to position information associated with the robot at a post-task position, the post-task position corresponding to the position of the robot after performing the first task; calculate a correction factor based on the first posture and the second posture; and guide the robot to perform a second task based on the correction factor.

[0114] Example 9 includes the non-transitory computer-readable storage medium of Example 8, wherein the first task includes the robot coupling a first assembly item to a second assembly item.

[0115] Example 10 includes the non-temporary computer-readable storage medium described in Example 8 or 9, wherein the first posture is a first homogeneous matrix and the second posture is a second homogeneous matrix, and the non-temporary computer-readable storage medium also includes instructions that, when executed, cause the machine to at least calculate the correction factor by multiplying the second homogeneous matrix with the inverse of the first homogeneous matrix.

[0116] Example 11 includes the non-transitory computer-readable storage medium of Example 10, wherein the first homogeneous matrix and the second homogeneous matrix are 4×4 homogeneous matrices.

[0117] Example 12 includes the non-transitory computer-readable storage medium described in any of Examples 8-11, further comprising instructions that, when executed, cause the machine to at least update the model based on the correction factor by generating a set of trajectories associated with a second task, determining one of the trajectories in the set whose target state is closest to the second posture, and generating one or more joint states associated with one of the trajectories by applying the correction factor to the target state.

[0118] Example 13 includes the non-transitory computer-readable storage medium described in any of Examples 8-12, which also includes instructions that, when executed, cause the machine to at least determine the coordinates of the estimated pre-task position by using the model as a basis for determining the estimated pre-task position.

[0119] Example 14 includes the non-transitory computer-readable storage medium described in any of Examples 8-13, which also includes instructions that, when executed, cause the machine to at least adjust the robot to the first actual pre-task position by obtaining at least one of output from a camera system or sensor measurements from an encoder.

[0120] Example 15 includes a method comprising: instructing a robot to move an end effector from a first position to a second position to perform a first task based on a first posture of the robot, the first posture being based on a model; in response to the robot moving to a third position, adjusting the robot to a fourth position to perform the first task, the fourth position being close to the second position; in response to performing the first task, determining a second posture of the robot, the second posture corresponding to position information associated with the robot at a fifth position, the fifth position corresponding to the position of the robot after performing the first task; calculating a correction factor based on the first posture and the second posture; and guiding the robot to perform a second task based on the correction factor.

[0121] Example 16 includes the method of Example 15, wherein the first task includes the robot coupling a first assembly item to a second assembly item.

[0122] Example 17 includes the method of Example 15 or 16, wherein the first pose is a first homogeneous matrix and the second pose is a second homogeneous matrix, and calculating the correction factor includes multiplying the second homogeneous matrix by the inverse of the first homogeneous matrix.

[0123] Example 18 includes the method described in any of Examples 15-17, further comprising updating the model based on the correction factor, wherein the updating includes generating a set of trajectories associated with the second task, determining one of the trajectories in the set whose target state is closest to the second posture, and generating one or more joint states associated with one of the trajectories by applying the correction factor to the target state.

[0124] Example 19 includes the method of any of Examples 15-18, wherein the second location is determined by using the model as a ground truth to determine the coordinates of the second location.

[0125] Example 20 includes the method of any of Examples 15-19, wherein adjusting the robot to the fourth position comprises at least one of obtaining an output from a camera system or obtaining sensor measurements from an encoder.

[0126] Although certain exemplary methods, apparatus, and articles have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus, and articles that fully fall within the scope of the claims of this patent.

Claims

1. A device comprising: Command generator for: instructing a robot to move an end effector from a temporary position to an estimated pre-task position to perform a first task based on a first pose of the robot, wherein the first task includes the robot coupling a first assembly item to a second assembly item, and the first pose is based on a model; When the robot is to move to a second actual pre-task position, adjusting the robot to a first actual pre-task position to perform the first task by at least one of obtaining output from a camera system or obtaining sensor measurements from an encoder, the first actual pre-task position being close to the estimated pre-task position; as well as The robot is guided to perform a second task based on a correction factor, wherein the correction factor is determined by: determining a second pose of the robot, the second pose corresponding to position information associated with the robot at a post-task position corresponding to position information associated with the robot after performing the first task; and calculating the correction factor based on the first posture and the second posture; Wherein the first pose is a first homogeneous matrix and the second pose is a second homogeneous matrix, and the correction factor is calculated by multiplying the second homogeneous matrix with the inverse of the first homogeneous matrix. 2 . The apparatus of claim 1 , wherein the first homogeneous matrix and the second homogeneous matrix are 4×4 homogeneous matrices. 3 . The apparatus of claim 1 , further comprising a model processor for determining the estimated pre-task position by using the model as a ground truth to determine coordinates of the estimated pre-task position.

4. The apparatus according to any one of claims 1 to 3, further comprising a model processor for updating the model based on the correction factor by: generating a set of trajectories associated with the second task; determining one of the trajectories in the set having a target state closest to the second posture; and One or more joint states associated with the one of the trajectories are generated by applying the correction factor to the target state.

5. A non-transitory computer-readable storage medium comprising instructions that, when executed, cause a machine to at least: instructing a robot to move an end effector from a temporary position to an estimated pre-task position to perform a first task based on a first pose of the robot, wherein the first task includes the robot coupling a first assembly item to a second assembly item, and the first pose is based on a model; When the robot is to move to a second actual pre-task position, adjusting the robot to a first actual pre-task position to perform the first task by at least one of obtaining output from a camera system or obtaining sensor measurements from an encoder, the first actual pre-task position being close to the estimated pre-task position; determining a second pose of the robot after performing the first task, the second pose corresponding to position information associated with the robot at a post-task position corresponding to a position of the robot after performing the first task; calculating a correction factor based on the first posture and the second posture; and directing the robot to perform a second task based on the correction factor; Wherein the first pose is a first homogeneous matrix and the second pose is a second homogeneous matrix, and the non-transitory computer-readable storage medium further comprises instructions that, when executed, cause the machine to at least calculate the correction factor by multiplying the second homogeneous matrix by the inverse of the first homogeneous matrix. 6 . The non-transitory computer-readable storage medium of claim 5 , wherein the first homogeneous matrix and the second homogeneous matrix are 4×4 homogeneous matrices.

7. The non-transitory computer-readable storage medium of claim 5, further comprising instructions that, when executed, cause the machine to at least determine the estimated pre-task position by using the model as a ground truth to determine coordinates of the estimated pre-task position.

8. The non-transitory computer-readable storage medium of any one of claims 5-7, further comprising instructions that, when executed, cause the machine to at least update the model based on the correction factor by: generating a set of trajectories associated with the second task; determining one of the trajectories in the set having a target state closest to the second posture; and One or more joint states associated with the one of the trajectories are generated by applying the correction factor to the target state.

9. A method for implementing the device of any one of claims 1 to 4, comprising: instructing a robot to move an end effector from a first position to a second position to perform a first task based on a first pose of the robot, wherein the first task includes the robot coupling a first assembly item to a second assembly item, and the first pose is based on a model; In response to the robot moving to a third position, adjusting the robot to a fourth position to perform the first task by at least one of obtaining an output from a camera system or obtaining a sensor measurement from an encoder, the fourth position being proximate to the second position; In response to performing the first task, determining a second pose of the robot, the second pose corresponding to position information associated with the robot at a fifth position, the fifth position corresponding to a position of the robot after performing the first task; calculating a correction factor based on the first posture and the second posture; as well as directing the robot to perform a second task based on the correction factor; Wherein the first pose is a first homogeneous matrix and the second pose is a second homogeneous matrix, and the correction factor is calculated by multiplying the second homogeneous matrix with the inverse of the first homogeneous matrix.

Images