Variable Payload Robot

Abstract

A variable payload robot is disclosed. In various embodiments, the robot includes two or more joints, each actuated by an associated joint motor, each joint motor having a different capacity, and the robot includes an end effector configured to grasp an object. A processor coupled to the robot is configured to determine a plan and trajectory for moving the object from a source location to a destination location based at least in part on the respective capacities of at least some of the joint motors and payload-related attributes of the object.

Description

CROSS-REFERENCE TO OTHER APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 422,334, entitled "VARIABLE PAYLOAD ROBOT," filed November 3, 2022, which is incorporated herein by reference for all purposes. [Background technology]

[0002] Industrial robot arms are typically designed to perform a variety of precision tasks with predetermined motions and payloads. Manufacturers conservatively create design and operational specifications that set uniform performance standards across the robot's work envelope. These specifications are typically set as hard constraints in the hardware, firmware, and / or drivers. This results in the robot having variable performance capabilities across the work envelope, resulting in consistently underutilized hardware. [Brief explanation of the drawings]

[0003] Various embodiments of the present invention are disclosed in the following detailed description and the accompanying drawings.

[0004] [Figure 1A] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0005] [Figure 1B] FIG. 1 is a block diagram illustrating an embodiment of a variable payload robotic system.

[0006] [Figure 2A] FIG. 1 illustrates an embodiment of a conventional non-variable payload robotic system.

[0007] [Figure 2B] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0008] [Figure 3A]FIG. 1 illustrates an embodiment of a conventional non-variable payload robotic system.

[0009] [Figure 3B] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0010] [Figure 3C] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0011] [Figure 3D] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0012] [Figure 4] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0013] [Figure 5] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0014] [Figure 6A] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0015] [Figure 6B] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0016] [Figure 7] 1 is a flow chart illustrating one embodiment of a process for operating a variable payload robotic system.

[0017] [Figure 8] 1 is a flow chart illustrating an embodiment of a process for planning a trajectory for operating a variable payload robotic system.

[0018] [Figure 9A] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0019] [Figure 9B] FIG. 1 illustrates an embodiment of a variable payload robotic system.

[0020] [Figure 10] 1 is a flow chart illustrating one embodiment of a process for executing a planned trajectory for operating a variable payload robotic system. DETAILED DESCRIPTION OF THE INVENTION

[0021] The present invention may be embodied in various forms, including as a process, an apparatus, a system, a composition of matter, a computer program product embodied on a computer-readable storage medium, and / or a processor configured to execute instructions stored in and / or provided by a memory coupled to the processor. These embodiments, or any other form the present invention may take, may be referred to herein as technology. In general, the order of steps in a disclosed process may be varied within the scope of the present invention. Unless otherwise noted, components, such as a processor or memory, described as configured to perform a task may be implemented as general components temporarily configured to perform the task at a given time, or as specific components manufactured to perform the task. As used herein, the term “processor” refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0022] The following is a detailed description of one or more embodiments of the present invention with reference to figures that illustrate the principles of the invention. While the present invention has been described in connection with such embodiments, it is not limited to any particular embodiment. The scope of the present invention is limited only by the claims, and the present invention includes many alternatives, modifications, and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are for the purpose of example, and the present invention may be practiced according to the claims without some or all of these specific details. For simplicity's sake, technical matters that are well known in the art related to the present invention have not been described in detail so as not to unnecessarily obscure the present invention.

[0023] Techniques are disclosed for defining robot design and operational parameters consistent with the physical ability to operate with variable performance across a work envelope. In various embodiments, the techniques disclosed herein are utilized to enable robots to perform precision and non-precision tasks with dynamically calculated motion and variable or unknown payloads.

[0024] In various embodiments, robot physics, kinematics, movement payload, robot motor capabilities, control system, joints, limbs, grippers, and / or multiple other capabilities are considered when determining (e.g., dynamically, in real time) how the capabilities of the robot can be utilized within appropriate constraints to perform a given task. Advantages achieved in various embodiments include one or more of the following: · It enables the construction of lightweight, highly capable robots that are focused on performance for a specific set of tasks, but with a wide variety of motions, operating conditions, and payloads. Intelligent control and machine learning systems enable maximum utilization of the robot's capabilities across the entire work envelope. Dynamically improve the design and behavior of multiple robotic arms working in collaboration.

[0025] In various embodiments, the techniques disclosed herein enable one or more of the following: building and programming less precise robots, reducing the weight of robotic machines, reducing power consumption, targeting robots to specific tasks, identifying and executing physical actions that minimize control errors, and improving force and contact control. While particular applications such as robotic package sorting or trailer loading are discussed, in various embodiments, the techniques disclosed herein may be utilized in a variety of other contexts.

[0026] In some contexts, a robot may have knowledge of payload mass (weight). Examples include knowledge of payload mass from task planning software, weigh stations at pickup points, wrist-mounted force sensors, payload (or type) identification via computer vision and lookup, etc. Using knowledge of payload mass, a motion planner component of the robotic system can apply the techniques disclosed herein to select an appropriate path and destination. For example, the planner may choose to hang a heavier payload from its “elbow” or other penultimate joint, thereby reducing stress on the last (e.g., “wrist”) joint and its connected link or limb.

[0027] 1A illustrates one embodiment of a variable payload robotic system. In the example shown on the left side of FIG. 1A, a robotic arm with a base segment 102, a motorized shoulder joint 104, a motorized elbow joint 106, a motorized wrist joint 108, and a suction-type gripper 110 is shown in a position where an object 112 hangs below and is moved closer to the robot, for example, to reduce stress on the "wrist" joint 108 (e.g., no torque required) and the upper links of the wrist (e.g., only tension loads and no bending).

[0028] 1A , in the example shown on the right, the object 112 is held and / or moved further away from the base segment 102. In various embodiments, for example, with respect to the example shown on the right of FIG. 1A , a control system configured to control the robotic arms of FIG. 1A may have determined that, given a known or estimated weight of the object 112, the robotic arms 102, 104, 106, 108, 110 can hold the object 112 in the position and orientation shown. For example, the system may have determined that the robotic arms and their components can hold the object 112 in the position and orientation shown based on the weight or torque limits of the motors driving the object 112, the joints 104, 106, and / or 108, and the strength of the links connecting the joints. For example, it may have instead been determined that a heavier object needs to be held in the position shown on the right of FIG. 1A .

[0029] In various embodiments, the robotic systems disclosed herein are configured to recognize their hardware components and their respective capabilities and limitations. In traditional approaches, the maximum torque of each motor (and associated gearbox), the maximum designed payload of the limbs (links) that make up the robot arm, etc. are considered in determining the maximum ratings for the robot at a system level for different configurations, poses, etc. These limitations are hard-coded into firmware, resulting in a "stall" if the robot control system attempts to exceed those limits. Typically, traditional approaches result in strict enforcement of limitations or guardrails that result in the robot being severely underutilized.

[0030] In contrast, in the approach disclosed herein, the robotic control system knows the capabilities and limitations of the individual components that make up the robot and takes that information into account when making real-time decisions about how to best utilize the robot's capabilities to perform a given task. This approach, also referred to herein as "hardware-aware" robotic control, allows for fuller utilization of the robot's capabilities. For example, in FIG. 1A, the robotic system may determine that the robot can lift a much heavier payload in the manner shown on the left than it can in the pose shown on the right.

[0031] FIG. 1B is a block diagram illustrating one embodiment of a variable payload robotic system. In the illustrated example, system 120 includes a robotic arm 122 positioned and configured to pick and place a box 124 stacked near robotic arm 122. Robotic arm 122 communicates with control computer 126, for example, via conventional wireless communication. Control computer 126 may include one or more processors, computer memory, and a wireless communication interface. In the illustrated example, control computer 126 includes a control stack 128 including one or more processing modules that cooperate to provide commands to robotic arm 122 to manipulate box 124. For example, control stack 128 may include a motion planner configured to determine a trajectory for moving an end effector of robotic arm 122 through a determined trajectory in three-dimensional space (after being used to grasp box 124) to place box 124 at a destination location with a predetermined orientation.

[0032] In the illustrated example, the control stack 128 uses the robot model 130 and the object attribute store 132 to determine how to pick and place the box 124 as disclosed herein. For example, the robot model 130 may include a kinematic model of the robot arm 122, which the control stack 128 may use to determine the physical reach of the robot arm 122, various combinations of joint positions to which the arm 122 can be moved, etc. Additionally, the capabilities of each joint of the robot arm 122 may be stored in the robot model 130. For example, the torque capacity or torque limit of each joint motor may be stored. In various embodiments, the control stack 128 may use the joint- and / or link-level capabilities and / or limitations of the elements that make up the robot arm 122 and the attributes of a given box 124 loaded from the object attribute store 132 to determine a set of one or more feasible trajectories for the robot arm 122 to use to move the given box 124 from a source position to a destination position.

[0033] 1B , in the illustrated example, control computer 126 further includes a computer vision stack 134 configured to receive and process image data received from 3D camera 136, which is shown positioned in the workspace along with a diagram of robotic arm 122 and box 124. In various embodiments, vision stack 134 includes components configured to extract data from images generated by camera 136 that can be used to look up attributes of a given box 124 from object attribute store 132. For example, text or other information printed on box 124 may be recognized and used to perform the lookup. In another example, vision stack 134 may use image data from camera 136 to more directly determine attributes, such as by recognizing a weight printed on the side of box 124 and / or a printed warning indicating that box 124 is “heavy.” In various embodiments, the control stack 128 determines a plan and trajectory for moving a given box 124 from a source location to a destination location using object attribute data determined at least in part from image data processed by the vision stack 134.

[0034] In various embodiments, trajectories with different payloads and maneuvers are simulated prior to runtime (i.e., before beginning to plan and execute a given task) to allow the robotic system to learn its capabilities and / or limitations to those capabilities for a given payload, situation, task, etc. In some embodiments, the robot design is iterated at a box (payload) level, given the detailed and / or fine-grained capabilities of the robot and its components, as described above. For example, the robot may learn through simulation that it cannot lift or move a payload over a certain weight in the configuration shown on the right above, but can in the “hanging” configuration on the left. In some embodiments, the robot uses the simulation at runtime to decide in real time between different locations for placing a box or other payload, or between different strategies for grasping the payload and / or different plans for moving the payload, for example, subject to physical joint limitations. In some embodiments, the robot uses simulations at run time to react in real time to detected conditions, such as an object being heavier than expected, a joint being less capable than expected (e.g., due to heat or other environmental conditions, wear, impending failure, etc.), encountering an obstacle along the way, etc.

[0035] In various embodiments, one or more of the following technical problems are overcome using the techniques disclosed herein: 1. The system can calculate the robot's ability to move with multiple payloads while performing a specific task (instead of having a pre-set ability for any task). Intelligent and dynamic control to calculate the feasibility of achieving robot motion and force control for various desired motion trajectories and payloads, then use this information to scale capabilities. 2. Reduce the cost, weight, and electromechanical design requirements of multi-degree-of-freedom robots designed to operate with a variety of payloads. · Actuator sizing, which leads to cost, weight, and electromechanical design requirements for multi-degree-of-freedom robots designed to operate with a variety of payloads. For redundant robots, trade off the actuator specifications of the redundant joints to distribute weight to joints closer to the base than to joints further from the base (which would otherwise contribute to poor inertial performance due to gravity in robot acceleration). 3. Using task null-space control, active error compensation, and / or force control, reduce the weight of the robot links by allowing them to be slightly flexible (effectively reducing their stiffness) while maintaining the ability to manipulate and position objects. · Task null space control, active error compensation, and force control are used to reduce the weight of the robot links by making them slightly flexible while maintaining positioning capabilities. 4. Enhance the robot's "object manipulation capabilities" to be useful for very heavy payloads by using alternative techniques to achieve the same manipulation results without using too much of the robot's physical capabilities. · When faced with a heavy payload, perform pick-and-place tasks using object manipulation strategies such as sliding, rolling, or knocking over items. 5. Taking into account payload information and task type, to calculate and evaluate the merits of multiple motion paths by selecting a path from multiple available paths so as to achieve lower errors when the task is performed, or by selecting an appropriate robot from multiple robots so as to achieve lower errors when the task is performed. · When scoring different task execution methods, consider the predictability of the control and model based on the payload and type of task being performed. 6. Accelerate robot movement by utilizing available electromechanical power and time more effectively by acquiring payload information to identify (potentially indirect or non-intuitive) paths. · Reduce power consumption for robots carrying variable payloads. For example, use the robot's null space to change the overall power consumption profile (e.g., by repositioning joints to move). For example, picking / placing heavier payloads closer to the robot's base can save power.

[0036] FIG. 2A illustrates one embodiment of a conventional non-variable payload robotic system. In the illustrated example, the robot 202 is shown with fixed specifications (in this example, a maximum payload of 5 kg). As a result, the robot 202 may be hardwired to be limited to manipulating a maximum payload 204 of no more than 5 kg anywhere within a reachable zone 206 that is physically reachable by the robot 202. An unreachable area 208 beyond the reach of the robot 202, in this example, cannot be physically reached by the robot 202, regardless of the payload. Note that in the illustrated configuration / orientation, the torque limit (or other limitation) of the wrist joint 210 of the robot 202 may be the “weak link,” requiring that the payload 204 be limited to 5 kg within the entire reachable zone 206 of the robot 202.

[0037] 2B illustrates one embodiment of a variable payload robotic system. In the illustrated example, the techniques disclosed herein are used to enable a robot 202 to be utilized to manipulate larger payloads within a particular portion of the robot's 202 physical reachable zone 206. In portions of the robot's 202 closer reachable zone 206, for example, the robot 202 may be capable of grasping, lifting, and moving an object 204 having a mass of up to 25 kg. For example, the robot 202 may be positioned in a pose similar to that shown on the left side of FIG. 1A, thereby reducing (or eliminating) the torque required from the wrist joint 210.

[0038] In various embodiments, the techniques disclosed herein allow the reachable zone 206 of the robot 202 to be transformed from a uniform reachable zone for manipulating objects up to 5 kg anywhere in the reachable zone 206 to a "probably" reachable zone 206 where the reachable portion of a given object can vary and can be dynamically determined at runtime.

[0039] In various embodiments, the robotic systems disclosed herein are transitioned from a fixed and potentially overly conservative set of specifications to a regime of variable and / or dynamically determined specifications that maximize the utility of robotic actuators and other elements in situations where the environment and / or task are dynamic, as shown, for example, in FIG. 2B.

[0040] In some embodiments, zones of "probably reachable" configurations and / or "probably feasible" task capabilities are generated and defined, where the feasibility of manipulating a particular object to perform a particular task of interest is determined by software calculations in real time (rather than by pre-computed hardware capabilities).

[0041] In some embodiments, for a given task that is strictly or clearly defined or conceived, notification of task completion may be modified to restore reachability or feasibility even if the former is compromised. For example, instead of "place item I on surface S at position (X, Y, Z) with current / specified orientation," the task may be understood to allow for various solutions, such as placing the item at other positions on the surface and / or in other orientations that are stable.

[0042] In various embodiments, the fundamental criteria for calculating a robot's capabilities is changed from the physical capabilities of the robot hardware (e.g., motor speed, etc.) to the robot's ability to perform a given task. Techniques are disclosed that allow a weaker, less precise / less repeatable robot to perform the same task that would normally be performed using a much heavier, stronger, more precise robot.

[0043] In some embodiments, the techniques disclosed herein provide asymmetric capabilities. For example, a joint (actuator) may be able to generate more torque and / or operate at a higher speed in one direction than in another. In another example, a motor may be able to operate in a first manner in cooler climates, but only in a second manner (e.g., lower current / torque, slower speed, shorter duty cycle, etc.) in warmer environments. In some embodiments, sensors on the robot (e.g., strain gauges on the limbs, force sensors, temperature sensors in each motor, etc.) and / or mathematical / computer models of the robot and / or its components may be used to inform hardware-aware decisions made by the robot control system at any time (in any situation) about the (best) way to perform a given task.

[0044] In various embodiments, moving to a task-oriented basis for determining the capabilities of a robot may do one or both of the following: First, it influences the design and choice of hardware used in the robot, or the type of robot itself, so that lighter weight motors, gearboxes, and limbs can be used, lighter limbs that can handle compressive or tensile loads rather than bending, etc., so that the full range of component capabilities can be realized and utilized. Second, the hardware / robot selection will affect how the robot is utilized.

[0045] Both types of effects are discussed herein with appropriate examples.

[0046] Intelligent dynamic control to ensure task completion

[0047] 3A illustrates one embodiment of a conventional non-variable payload robotic system. In the illustrated example, a robotic arm 302 is used to place an object 304 at a specific position and orientation on top of an object 306. The robot 302, in this example, is hardwired to be limited to placing objects up to 5 kg anywhere within the reachable area of ​​the robot 302 (shown as a dot in FIG. 3A). For example, a 10 kg object cannot be manipulated in any part of the reachable area of ​​the robot 302.

[0048] FIG. 3B illustrates one embodiment of a variable payload robotic system. In the illustrated example, the ability of the robot 302 to lift a heavier payload is enhanced by changing the robot's posture (e.g., placing the wrist joint in a less stressed position) or by changing the item's final position and / or orientation at the time of placement (i.e., closer to the robot 302, standing it on end rather than laying it flat). In various embodiments, the systems disclosed herein may be configured to redefine or request adjustments to the task definition to determine a feasible plan and / or trajectory. For example, in the illustrated example, the task may be specified to place object 304 on top of object 306 without specifying a position or orientation. Alternatively, the system may determine that placing object 304 in the originally specified position (e.g., the position and orientation shown in FIG. 3A ) is not feasible given the weight of object 304 and may request a different placement and / or orientation. A higher level (or peer level) algorithm / module (such as one configured to determine how to stack items to achieve dense and stable packing) may be required to determine a different or updated packing plan, for example, to allow object 304 to be placed as shown in FIG. 3B rather than as shown in FIG. 3A.

[0049] 3C illustrates one embodiment of a variable payload robotic system. In the illustrated example, the robot 302 is placed on a movable base to reduce torque requirements on relatively weak joints (such as the wrist joint of the robot 302). In the example shown in FIG. 3C, the movement of the movable base is utilized to reduce the torque that needs to be provided by the relatively weak joints (such as the wrist joint of the robot 302) when moving the object 304 to the position and orientation shown.

[0050] 3D illustrates one embodiment of a variable payload robotic system. In the illustrated example, robot 302 and / or a control computer associated with robot 302 has determined that robot 302 cannot independently manipulate object 304. In response, robot 302 is shown to have sought assistance from another robot 308. In this example, the ability to dynamically seek assistance from robot 308 enhances the operational capabilities of the robotic system of FIG. 3D.

[0051] In various embodiments, the robot disclosed herein first determines whether the task of picking and playing an object is feasible within the robot's payload capacity across its workspace. If so, a traditional approach to trajectory planning and execution may be used. To pick and place heavier objects and expand the robot's variable payload capacity, in various embodiments, the system dynamically calculates how a particular object can be picked and placed. In some embodiments, this is done by integrating a controller with a task planner to accommodate three resulting "capability-changing" scenarios: 1. Changing the way the robot performs a task. This can include changing the pose, gripping style, placing style, or the order in which a series of tasks are performed. See, for example, Figure 3B. 2. Use mobility as a tool to increase payload capacity by moving the robot to positions where weaker joints experience less load. See, for example, Figure 3C. 3. Use robot collaboration to enable multiple robots to interact with each other. See, for example, Figure 3D.

[0052] Intelligent dynamic control to ensure task completion

[0053] Various embodiments reduce the cost, weight, and electromechanical design requirements of multi-degree-of-freedom robots designed to operate with a variety of payloads: it is no longer necessary to utilize robots in which every joint and limb is rated to handle the heaviest loads that the robot may be asked to handle in every pose.

[0054] Actuator sizing leads to cost, weight, and electromechanical design requirements for multi-degree-of-freedom robots designed to operate with a variety of payloads.

[0055] For redundant robots, trade off the actuator specifications of the redundant joints to distribute weight to joints closer to the base than to joints further from the base (which would otherwise contribute to poor inertial performance due to gravity in the robot's acceleration).

[0056] No longer does every link / joint need to be able to handle a single load rating. Size / weight / cost are determined by task feasibility, not a single / static maximum load rating.

[0057] Further use cases include: Traditional 6 degrees of freedom ("DOF") serial manipulator arm Adding a seventh DOF to a serial manipulator 8DOF serial manipulators as shown in the examples of Figures 3A to 3D

[0058] Adding additional DOFs to an existing serial chain manipulator design often means adding dense motor / gearbox assemblies at the additional joints. In various embodiments, the techniques disclosed herein are used to minimize the power, size, and mass required for the new actuators. For example, not all joints need to be capable of providing enough torque to move the remaining lower limb elements of the arm and the robot's maximum rated payload. This approach allows for the reallocation of work capacity for existing base actuators that currently have additional DOFs (i.e., additional joints and / or motors and other structures that include additional DOFs at existing joints) so that they can carry the additional mass of the new DOFs while still handling task-based payloads as disclosed herein.

[0059] In various embodiments, this optimization generalizes to 6DOF robots, 7DOF, 8DOF, and even 12-16DOF systems. For high degree of freedom systems, this is realized in some embodiments as a two-arm or three-arm system. Each (smaller) arm by itself may be independent when tackling smaller payloads within its workspace, but when combined to perform a two-handed pick, the available workspace and payload limitations become significantly greater.

[0060] This can be seen when one robot helps another robot lift a heavy object, or lifts it in an out-of-specification orientation, only in a specific zone, but not along the entire trajectory.

[0061] In various embodiments, motor size and weight are reduced to the extent that they are still capable of completing the overall task required by the robot. However, the same methodology applies to reducing motor power. For example, in some embodiments, motor size is kept large, but overall system power is dynamically allocated to individual motors. This effectively limits motor strength (as well as reducing motor size and weight), creating the same utilization zones shown above. Even better, dynamically reducing power per motor allows for on-the-fly zone changes. This is another optimization used to reduce the size and capabilities of the system's power electronics (rather than having to size the electronics to power the robot to 100% full power at every joint).

[0062] Efficiently operating flexible, low-weight robots

[0063] In various embodiments, task null-space control, active error compensation, and / or force control are used to reduce the weight of robot links by allowing at least some components to be slightly flexible (effectively reducing their stiffness) while maintaining the ability to manipulate and position objects.

[0064] Task null-space control, active error compensation, and force control are used to reduce the weight of the robotic links by making at least some components (e.g., limbs) slightly flexible while maintaining positioning capabilities.

[0065] For example, as described above, a heavier payload is "hanged" to allow for the use of more flexible (less stiff or rigid) materials in the links / limbs connected to the wrist. At run time, simulations can be used to understand that, given the weight of the payload, using the robot in a "hanging" position minimizes strain on the more flexible limbs.

[0066] In some embodiments, joints or other structures other than links / limbs may be more flexible than conventional robots. For example, for certain tasks (such as picking and placing large boxes or packages), less precision (e.g., due to more flexible links / limbs and / or joints) may be acceptable, allowing control as disclosed herein to be performed despite possible bending and / or vibration due to the use of lightweight but flexible links / limbs, joints, etc.

[0067] In some embodiments, simulation is utilized to investigate the appropriate stiffness or mechanical properties required for a robot based on the design requirements for performing a given task or set of tasks within the expanded set of capabilities discussed above.

[0068] Utilizing payload information for dynamic path and motion control

[0069] In various embodiments, the "object handling capabilities" of a robot useful for very heavy payloads are enhanced by using the alternative approaches disclosed herein to achieve the same manipulation result without using much of the robot's physical capabilities. When faced with a heavy payload, pick-and-place tasks may be performed using non-traditional object handling strategies, such as sliding, rolling, or tipping items. Different "utilization zones" may be defined to identify reachable areas where the robot's capabilities (e.g., maximum payload) may be expanded.

[0070] Figure 4 illustrates one embodiment of a variable payload robotic system. In the illustrated example, system 400 includes a robotic arm 402 that defines different operational zones, including zone 1 (404), zone 2 (406), and zone 3 (408 and 410).

[0071] Each zone (404, 406, 408, and 410) may have different operating characteristics / limits, for example: Zone 1: Maximum payload with limited wrist orientation. Zone 2: Smaller payloads and maximum flexibility in wrist orientation. Zone 3: Same as Zone 2, but the joints are constrained by the maximum torque required by the joints to hold the target object (+ end effect mass).

[0072] The example in Figure 4 shows three task / execution regimes, but more or fewer regimes may be utilized. For example, a small number of regimes is most beneficial when optimizing motor / gearbox selection at each joint over a small number of standard motor / gearbox combinations. However, depending on the nature of the task and the type of payload, more or fewer zones may be defined and utilized.

[0073] Calculate the useful working space with an emphasis on task capability, not just reachability.

[0074] In various embodiments, payload information and the type of task are taken into account to calculate and evaluate the merits of multiple motion paths and / or to select a path from multiple available paths to achieve lower error when the task is performed, or by selecting an appropriate robot from multiple robots to achieve lower error when the task is performed.

[0075] In various embodiments, the systems disclosed herein determine feasibility and / or select a plan / trajectory based on the payload and the type of task being performed. A score may be assigned to each of the different possible ways to perform a given task, and the score is used to select a plan.

[0076] In some embodiments, the mass / inertia of the object being manipulated may be combined with the weight of the last link and / or joint of the robot, which changes the effective dynamic behavior of the combined system, effectively changing the instantaneous acceleration profile and the resulting optimal overall trajectory path profile. This effect is particularly enhanced when the object's inertia is non-negligible compared to the robot's inertia (as is the case when a lighter robot lifts a heavier payload and / or when multiple robots lift an item).

[0077] For example, if you want to move a heavy box, there are two valid paths: (a) bring the box close to the base (tightly) and perform a complex motion to prevent the arm from extending, or (b) balance the box against a shelf and slide it to the target location. Both will work, but if you choose a robot with a rigid body (expensive) but no force control (cheap), option (a) will be better, whereas if you choose a robot with a flexible body (cheap, lightweight) but good force control (expensive), option (b) will be better.

[0078] The robot may have knowledge of the payload mass, for example from task planning software, a weigh station at the pickup point, a wrist-mounted force sensor, etc. Using the payload mass knowledge and the task payload map (see, e.g., Figure 4), the motion planner selects an appropriate path and destination.

[0079] The software (e.g., motion planner) utilized to provide the capabilities described herein, in some embodiments, generates robot motions that fit within a (dynamically / variably defined) envelope of the robot's maximum specifications. Knowledge of the payload mass is required to plan motion within each joint's constraints (electrical constraints, torque constraints, etc.).

[0080] In some embodiments, this same software is used to simulate trajectories with different payloads and maneuvers. The robot design may be iterated, in some embodiments, given detailed capabilities at a box-by-box level. During operation, the robot can also use the simulation to decide between different positions to place the box, subject to physical joint limitations.

[0081] Dynamic load cost calculation is used to optimize the robot's power consumption.

[0082] In various embodiments, the robot's available electromechanical force, time, and velocity capabilities are used more effectively, for example, to accelerate the robot's movement by considering payload information to identify (potentially indirect or non-intuitive) paths, for example: Reducing power consumption of robots carrying variable payloads, for example, by planning movements to minimize power consumption given knowledge of the robot's variable capacity and power consumption for a given payload in different poses / zones / trajectories. For example, using the robot's null space to change the overall power consumption profile (e.g., by rearranging joints to move). Choose the least power-intensive way to perform a task from among many other ways. Using two robots working together can sometimes use less power than using one robot. Consider the number of robots (if any) available to assist, whether any / all power is battery or mains power, time of day, or maximum current / utilization limits on the circuits, etc.

[0083] For example, a single robot plugged into an otherwise non-battery powered wall outlet may be operated without as much consideration for power consumption as a single battery-powered robot, in which case real-time considerations such as battery charge level, charger availability and utilization, charging downtime, etc. may need to be taken into account when determining how to utilize the robot's capabilities to perform a task. For example, a lower-power method of performing a task more slowly may be selected.

[0084] In another scenario, multiple robots may be used cooperatively to minimize collective power usage, for example, using a single robot may be more energy efficient for some tasks, or using many robots simultaneously may increase speed but exceed instantaneous power consumption limits.

[0085] 5 illustrates one embodiment of a variable payload robotic system. In the illustrated example, the robotic system 500 includes a track loader robot with left and right robotic arms 502, 504 disposed on a robotically controlled mobile chassis 506 with a longitudinal central conveyor 508 disposed between the robotic arms 502, 504. During an unloading operation, the robots 502, 504, 506, 508 pick boxes 510 from within a truck (or other container) and place them on the central conveyor 508, which transports the boxes 510 to the rear of the track loader (e.g., to a human or other robotic worker, to a transport structure that transports the boxes 510 further downstream, etc.).

[0086] During a loading operation, boxes 510 may arrive at the rear end of the track loader and be transported by conveyor 508 to a location from which they may be picked using one or both of robotic arms 502, 504 and placed in a position that achieves, for example, a desired density and stability.

[0087] In various embodiments, the techniques disclosed herein are used to operate the robotic system of FIG. 5. For example, one of the robotic arms 502, 504 may be used to pick and place smaller items, while both may be used cooperatively to pick and place larger and / or heavier items. In some embodiments, variable payload technology is applied, for example, to allow heavier items to be unloaded to or from a given location than a track loader can load. For example, an item that may be too heavy for a track loader to lift from the conveyor 508 and place in a given location (e.g., at the top of a stack or layer) may be considered within the capabilities of the track loader if the same object is being unloaded instead. During unloading, the robotic arms 502, 504 can be utilized to pull heavy objects from the top of the stack and lower them onto the conveyor 508 in a controlled drop manner, with gravity being utilized to bring the items onto the conveyor 508 and braking force / torque provided by the robotic arms 502, 504 slowing the rate of descent to prevent damage to the objects.

[0088] Figure 6A illustrates one embodiment of a variable payload robotic system. In the illustrated example and state, the track loader of Figure 5 is used to initiate movement of box 510 along track 602, at least in part using gravity, from the position shown to conveyor 508 (e.g., to the end position shown in Figure 6B).

[0089] In some embodiments, the distal weaker joints of the robotic arms 502, 504 may be locked in place, allowing gravity, countered in part by the stronger joints of the robotic arms 502, 504, to be utilized to bring the box 510 to the position shown in FIG. 6B.

[0090] In various embodiments, the techniques described above and illustrated in FIGS. 6A and 6B may enable the track loader of FIG. 5 to unload much heavier objects than could be stacked in a similar location (e.g., the location shown in FIG. 6A forming the location shown in FIG. 6B).

[0091] FIG. 7 is a flowchart illustrating one embodiment of a process for operating a variable payload robotic system. In various embodiments, process 700 of FIG. 7 may be implemented by a control computer (such as control computer 126 of FIG. 1B). In the illustrated example, at step 702, a task to be performed and one or more attributes of the object on which the task is to be performed are determined. For example, a higher-level planner / scheduler may assign a task to pick a particular object from a source location and place the object at a destination location with a predetermined orientation or in one of a set or range of allowable positions and / or orientations. At step 704, a trajectory and plan are determined for grasping the object and moving it along a trajectory to complete (or properly, sufficiently, and / or acceptably complete) the task. In various embodiments, techniques disclosed herein are used at step 704 to determine the plan and trajectory based at least in part on the object attributes determined at step 702. At step 706, the plan determined at step 704 is implemented. If the object is successfully placed, it is determined whether any other tasks are to be performed in step 708. If so, the process proceeds via step 710 to determine and perform the next task. If no other tasks are to be performed (step 708), the process 700 ends.

[0092] FIG. 8 is a flowchart illustrating one embodiment of a process for planning a trajectory for operating a variable payload robotic system. In various embodiments, the process of FIG. 8 may be used to perform step 704 of process 700 of FIG. 7. In the illustrated example, at step 802, a feasible solution space is determined. For example, a set of solutions may be determined that includes only solutions that are feasible for a given payload. Alternatively, a zone associated with solutions that are feasible given the payload is selected (see, for example, FIG. 4). At step 804, the detailed capabilities and limitations of the components that make up the robot are considered to determine a particular trajectory and / or plan for performing the task. For example, the torque capabilities and / or limitations of each joint and / or weight limitations for joints in different configurations, orientations, and / or the links between them may be considered. At step 806, simulation and / or cost calculations are performed to determine and select the best (or good enough, or at least better than the best so far) solution for picking and placing the object. At step 808, the selected plan and / or trajectory is returned as a result.

[0093] In various embodiments, supervised and / or other machine learning may be used to learn strategies, trajectories, etc. for executing pick / place tasks for objects having various attributes (size, weight, etc.) given the joint / link-level capabilities and / or limitations of the elements that make up the robot. For example, for a given potential plan and / or trajectory, heavier objects may be moved through different trajectories in different operational zones to train a model that can be used to predict whether the plan and / or trajectory is or may be feasible and assign a confidence, fitness, or other score to the plan and / or trajectory.

[0094] 9A illustrates one embodiment of a variable payload robotic system. In the illustrated example, system 900 includes a robotic arm 902 utilized to move a box 904 through a trajectory that moves the robotic arm 902 through positions and orientations 902a, 902b, and 902c to place the box 904 on a pallet 906 at a target position and orientation 904c. The trajectory illustrated by FIG. 9A may be an example of a potentially feasible plan and trajectory, such as those that may be considered in steps 804 and / or 806, in some embodiments.

[0095] 9B is a diagram illustrating one embodiment of a variable payload robotic system. In the illustrated example, robotic system 920 includes the same robotic arm 902 as in FIG. 9A being moved through alternative trajectories 922a, 922b, 922c to place box 904 at alternative location 924c as shown. In various embodiments, the trajectories illustrated by FIG. 9B include further examples of potentially feasible plans and trajectories, such as those that may be considered in steps 804 and / or 806.

[0096] In various embodiments, the systems disclosed herein may consider alternative trajectories, configurations, and orientations, such as those shown in FIGS. 9A and 9B, to determine and select a viable preferred plan and trajectory. Aspects of each plan and the results achieved (e.g., the cost of grasping, moving, and placing the item, the impact of each plan on the downstream costs of placing other items around the item, etc.) may be considered to determine and select a trajectory and / or plan. For example, the trajectory and plan of FIG. 9A may be preferred for a lighter box 904, especially if the resulting configuration promotes low-cost packing (or higher value (e.g., higher density, higher stability)) of other boxes on or around box 904, whereas the trajectory and plan of FIG. 9A may be preferred if box 904 is too heavy to move and / or place as shown in FIG. 9A, for example, due to torque limitations of the wrist joint of the robotic arm 902.

[0097] FIG. 10 is a flowchart illustrating one embodiment of a process for implementing a planned trajectory for operating a variable payload robotic system. In various embodiments, the process of FIG. 8 may be used to perform step 706 of process 700 of FIG. 7. In the illustrated example, at step 1002, the robot and its components (e.g., joint motors, gripper sensors) are monitored as the robot is used to move an object through the planned trajectory. If at step 1004 an unexpected condition is detected (e.g., the object is heavier than expected or the robot is unable to maintain the originally planned trajectory), at step 1006 a revised trajectory and plan is determined based on the dynamically assessed conditions and the fine-grained capabilities of the components that make up the robot (e.g., torque capacity at each joint). For example, the robotic system may determine to move the load more quickly toward the destination, similar to how a human might do if a heavy item is slipping out of hand or becoming difficult to continue carrying, and then rapidly increase torque as needed near the end of the trajectory to lift the object and place it in place. Alternatively, the plan may be modified to place the item at a different target location and / or orientation, or in yet another example, assistance may be received from another robot or a human worker. Once the unexpected condition is addressed (steps 1004, 1006), monitoring continues (step 1002), and further adjustments are made as needed / if required (steps 1004, 1006) until the task is determined to be complete in step 1008, at which point the process of FIG. 10 ends.

[0098] In some embodiments, step 1006 is performed at least in part by a feedback control component or module that, if the actual trajectory along which the object is moved deviates from the plan, generates commands to return the object along the originally planned trajectory, or to abandon it and move along a newly calculated trajectory to the destination, taking into account the joints and other fine-level capabilities of the components that make up the robot.

[0099] In various embodiments, the techniques disclosed herein enable robotic systems to perform tasks more efficiently and with payloads that may exceed what is traditionally defined as a single, static maximum payload, and / or enable robot designs that provide the same capabilities with lighter weight, lower cost, and / or less power consumption.

[0100] Although the above-described embodiments have been described in some detail for ease of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and are not intended to be limiting.

Claims

1. 1. A system comprising: a robot having two or more joints, each joint actuated by an associated joint motor, each joint motor having a different capacity, the robot having an end effector configured to grasp an object; a processor coupled to the robot and configured to determine a plan and trajectory for moving the object from a source location to a destination location based at least in part on the capacitances of each of at least some of the joint motors and payload-related attributes of the object; A system comprising:

2. The system of claim 1 , wherein the payload-related attribute comprises a weight of the object.

3. The system of claim 2 , wherein the processor is further configured to determine the weight of the object.

4. The system of claim 3 , wherein the weight is determined based on sensors that configure the robot.

5. The system of claim 3 , wherein the processor is configured to determine the weight of the object based at least in part on image data.

6. 3. The system of claim 2, wherein the processor is configured to determine one or more operational zones within a maximum operational reach of the robot within which the robot can move an object.

7. 3. The system of claim 2, wherein the processor is configured to determine a set of feasible trajectories for moving the object based at least in part on the capacities of each of the at least some of the joint motors and the payload-related attributes of the object.

8. 8. The system of claim 7, wherein the set of feasible trajectories includes only trajectories within a portion of an operating space physically reachable by the robot, the portion being determined based at least in part on the payload-related attributes of the object.

9. The system of claim 1 , wherein the processor is further configured to monitor the robot as the object is moved along the trajectory.

10. The system of claim 9 , wherein the processor is configured to detect a condition and, in response to detecting the condition, determine a corrective trajectory.

11. 10. The system of claim 1, wherein the processor is configured to determine the plan and trajectory at least in part by utilizing a stored model that reflects the capacitance of each of the joint motors.

12. 12. The system of claim 11, wherein the model is trained at least in part by supervised machine learning.

13. 2. The system of claim 1, wherein the processor is configured to use simulation to determine a set of feasible trajectories for moving the object from the source position to the destination position given the capacities of each of the joint motors.

14. 14. The system of claim 13, wherein the processor is further configured to utilize real-time simulation to adapt the plan and trajectory in response to conditions detected while moving the object from the source location to the destination location.

15. The system of claim 1 , wherein the processor is configured to determine one or both of an alternative position and an alternative orientation in which to place the object.

16. 10. The system of claim 1, wherein the determined plan and trajectory includes moving a movable chassis to which the robot is mounted to impart momentum to the object.

17. 10. The system of claim 1, wherein the object is being moved to a destination location that is closer to the ground than the source location, and the determined plan and trajectory relies at least in part on gravity to move the object through the trajectory.

18. 10. The system of claim 1, wherein the robot includes a first robot, and the processor is further configured to receive assistance from a second robot to move the object through at least a portion of the trajectory.

19. 1. A method for programmatically controlling a robot, the robot comprising two or more joints, each joint actuated by an associated joint motor, each joint motor having a different capacity, the robot comprising an end effector configured to grasp an object, the method comprising: receiving an instruction to move an object from a source location to a destination location; using a processor to determine a plan and trajectory for moving the object from the source location to the destination location based at least in part on the capacity of each of at least some of the joint motors and payload-related attributes of the object; A method comprising:

20. 1. A computer program product for programmatically controlling a robot, the robot comprising two or more joints, each joint actuated by an associated joint motor, each joint motor having a different capacity, the robot comprising an end effector configured to grasp an object, the computer program product embodied in a non-transitory computer readable medium; computer instructions for receiving instructions to move an object from a source location to a destination location; computer instructions for, with a processor, determining a plan and trajectory for moving the object from the source location to the destination location based at least in part on the respective capacities of at least some of the joint motors and payload-related attributes of the object; A computer program product comprising:

Images