A robot system using dynamic motion planning for transferring unregistered objects

Abstract

This specification discloses a robotic system (and related systems, devices, and methods) that uses dynamic motion planning for transferring unregistered objects. In one embodiment, a method for operating the robotic system includes (i) receiving sensor data representing a distance between a sensor of the robotic system and a target object engaged by an end effector of the robotic system, and (ii) determining a height of the target object based at least in part on the sensor data. The method may further include updating a motion plan for placing the target object at a destination location based at least in part on the height of the target object. The updated motion plan may include commands, settings, or combinations thereof for operating a robotic arm and end effector to (i) approach the destination location and (ii) disengage the target object to place the target object at the destination location.

Description

Technical Field

[0001] Cross - reference to related applications (multiple possible) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 418,637, filed on October 24, 2022, which is hereby incorporated by reference in its entirety.

[0002] This technology generally relates to robotic systems, and more particularly, to systems, processes, and techniques for object detection. For example, some embodiments of this technology are directed to robotic systems that use dynamic approach, departure, and / or return path motion planning based on sensor data acquired using upward sensors, for example, robotic systems that use dynamic motion planning to transfer unregistered objects (e.g., objects having initially unknown dimensions), such as robotic systems for transferring unregistered objects (e.g., objects having initially unknown dimensions).

Background Art

[0003] With continuously improving performance and reduced costs, many robots (e.g., machines configured to automatically / autonomously perform physical actions) are now widely used in many fields. For example, robots can be used to perform various tasks (e.g., manipulate or transfer objects through space), such as in manufacturing and / or assembly, packing and / or packaging, conveying and / or shipping. When performing tasks, robots can replace or reduce the human intervention that would normally be required to perform dangerous or repetitive tasks because they can reproduce human actions.

[0004] However, despite technological advancements, in many cases, robots lack the sophistication necessary to reproduce the human interactions required for performing larger - scale and / or more complex tasks. Therefore, there is still a need for improved techniques and systems for managing the operations and / or interactions between robots.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7A

Figure 7B

Figure 8

Figure 9

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0006] This specification discloses a robotic system (and related systems, devices, and methods) that uses dynamic motion planning for transporting unregistered objects. Unregistered objects can include objects that have one or more characteristics or traits that are not included in, not stored in, or not registered in the master data of the robotic system used to transport the unregistered object between a source location and a destination location. Additionally, or alternatively, unregistered objects can include objects that have one or more characteristics or traits that can be misdetected, occluded, altered, and / or determined in another way as being different from the characteristics included in the master data. As a result, unregistered objects can be “unknown” (at least initially) to the robotic system. The unknown characteristics or traits of an unregistered object can include physical dimensions (e.g., the length and / or width of one or more sides of the object of interest), shape, position of the center of gravity, weight, SKU, vulnerability assessment, and the like. A specific example of a characteristic of an unregistered object of interest that can be unknown to the robotic system is the height of the object of interest.

[0007] Without knowledge of the characteristics of the target object, it may be difficult for a robotic system to place the target object at the destination position. For example, (i) at the source position, using the end effector of the robotic system to engage the upper surface of the target object, and (ii) it may be possible to transfer the target object towards the destination position (e.g., based on the maximum and / or minimum possible heights with respect to the target object), but the robotic system cannot recognize the position of the bottom surface of the target object. Therefore, the robotic system may not be able to determine the distance by which it must lower the target object towards the destination position before removing (e.g., dropping) the target object at the destination position. Releasing a shorter object at a higher position increases the drop distance and may increase the risk of damaging the object and its contents. Alternatively, lowering the grasped object too much may crush the grasped object and its contents.

[0008] To address this problem, the robotic system of the present technology can include a sensor (e.g., a distance sensor) having a vertically oriented field of view. The robotic system of the present technology can present the target object to the vertically oriented sensor by positioning the target object within the vertically oriented field of view of the sensor while transferring the unregistered target object between the source position and the destination position. As a result, the sensor can be used to determine the distance (e.g., a second distance) between the target object and the sensor. Additionally, assuming that (i) the position of the sensor and (ii) the position of the end effector that grasps the target object are known to the robotic system, the robotic system can determine the distance (e.g., a first distance) between the end effector and the sensor at the time when the target object is presented to the sensor. Therefore, the robotic system can determine the height of the target object by determining the difference between the first distance and the second distance.

[0009] Based on the knowledge of the height of the target object and the position of the end effector, this robotic system can determine the position of the bottom surface of the target object. As a result, this robotic system can determine an approach path for the robotic arm and end effector of the robotic system to place the target object at the transfer destination position. In some embodiments, this robotic system can optimize the approach path and / or the speed at which the robotic arm and end effector move along the approach path, for example, to reduce or minimize the time it takes for the robotic system to place the target object at the transfer destination position.

[0010] In addition, in some embodiments, this robotic system can determine the height (e.g., release height) above the transfer destination position where the end effector of the robotic system can safely remove (e.g., drop) the target object to place it at the transfer destination position. The release height may depend on one or more characteristics of the target object. For example, this robotic system can determine a lower release height for a heavier or more fragile target object and / or a higher release height for a lighter or less fragile target object.

[0011] Furthermore, based on the knowledge of the height of the target object, the present robot system can determine the future position of the end effector corresponding to the time when the bottom surface of the target object is positioned at the release height of the target object. Therefore, the present robot system can dynamically calculate a return path for returning the end effector to the starting position directly from the future position of the end effector. As a result, the time taken by the robot system to return the end effector to the starting position can be shorter than the time taken by a robot system that first raises the end effector to a pre-calculated height / pre-determined height (e.g., to avoid a horizontal line sensor or other components of the robot system) and then moves the end effector along the return path to the starting position following the placement of the target object at the transfer destination position.

[0012] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the technology of the present disclosure. In other embodiments, the techniques introduced herein may be practiced without these specific details. In other instances, well-known features such as specific functions or routines are not described in detail herein to avoid obscuring the present disclosure unnecessarily. References to "embodiments," "one embodiment," etc. in the context of the embodiments for carrying out the present invention mean that the specific features, structures, materials, or characteristics described are included in at least one embodiment of the present disclosure. Thus, the appearance of such phrases in this specification does not necessarily all refer to the same embodiment. On the other hand, such references are not necessarily mutually exclusive. Furthermore, the specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that the various embodiments shown in the figures are merely illustrative representations and are not necessarily drawn to scale.

[0013] It is well known and often associated with robotic systems and subsystems, but some details explaining structures or processes that can unnecessarily obscure some important aspects of the disclosed technology are not described in the following description for clarity. Further, in the following disclosure, some embodiments of different aspects of the present technology are shown, but some other embodiments can have different configurations or different components from those described in this section. Thus, the disclosed technology can have other embodiments with additional elements or without some of the elements described below.

[0014] Many embodiments or aspects of the present disclosure described below may take the form of computer-executable instructions or controller-executable instructions that include routines executed by a programmable computer or controller. Those skilled in the art will understand that the disclosed technology can be implemented in computer systems or controller systems other than those shown and described below. The techniques described herein can be embodied in a special-purpose computer or data processor specifically programmed, configured, or constructed to execute one or more of the computer-executable instructions described below. Accordingly, the terms "computer" and "controller" as generally used herein refer to any data processor and can include Internet appliances and handheld devices (including palm-top computers, wearable computers, cellular phones or mobile phones, multiprocessor systems, processor-based or programmable consumer electronics, network computers, minicomputers, etc.). The information processed by these computers and controllers can be presented on any suitable display medium, including a liquid crystal display (LCD). Instructions for performing computer-executable tasks or controller-executable tasks can be stored or carried on any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. The instructions can be included in any suitable memory device, such as, for example, a flash drive, a USB device, and / or other suitable media.

[0015] The terms "coupled" and "connected," along with their derivatives, can be used herein to describe the structural relationships between components. It should be understood that these terms are not intended to be synonyms for each other. Rather, in certain embodiments, "connected" can be used to indicate that two or more elements are in direct contact with each other. Unless the context clearly indicates otherwise, the term "coupled" can be used to indicate that two or more elements are in contact with each other directly or indirectly (with other intervening elements therebetween), that two or more elements are in cooperation or interaction with each other (such as in a causal relationship like signal transmission / reception or function calls), or both.

[0016] appropriate environment FIG. 1 is a partial schematic perspective view of an exemplary environment 150 in which a robot system 100 with a cooperative transfer mechanism can operate, according to various embodiments of the present technology. The robot system 100 can include one or more units (e.g., robots) configured to perform one or more tasks and / or communicate with those units. Aspects of the cooperative transfer mechanism can be practiced or implemented by various units.

[0017] In the illustrated embodiment, the robot system 100 can include a unloading unit 102, a transfer unit 104 (e.g., a palletizing robot and / or a piece picker robot), a conveying unit 106, a loading unit 108, or a combination thereof, in a warehouse or a logistics / shipping hub. Each unit within the robot system 100 can be configured to perform one or more tasks. The tasks can be combined in sequence to perform actions to achieve goals, such as unloading objects from a truck or van and storing them in a warehouse, or unloading objects from a storage location and preparing them for shipping. In some embodiments, the tasks can include placing an object at a destination location (e.g., on a pallet and / or inside a bin / cage / box / case). As will be described in detail below, the robot system 100 can derive individual placement positions / orientations and calculate corresponding motion plans, or a combination thereof, for placing and / or stacking objects. Each unit can be configured to perform a series of actions to perform the task (e.g., operating one or more of its components).

[0018] In some embodiments, the task can include an operation (e.g., movement and / or reorientation) of the target object 112 (e.g., one of a package, box, case, cage, pallet, etc. corresponding to the task being executed) from the start / transfer source position 114 to the task / transfer destination position 118. For example, the unloading unit 102 (e.g., a deboning robot) can be configured to transfer the target object 112 from a position within a transport vehicle (e.g., a truck) to a position on the conveyor 107. Also, the transfer unit 104 can be configured to transfer the target object 112 between one position (e.g., the conveyor 107, a pallet, or a bin) and another position (e.g., a pallet, a bin, another conveyor, etc.). For example, the transfer unit 104 (e.g., a palletizing robot) can be configured to transfer the target object 112 from a transfer source position (e.g., a pallet, a bin, a pickup area, and / or a conveyor where the transfer unit 104 engages the target object 112) to a transfer destination position (e.g., a pallet, a bin, a drop-off area, and / or a conveyor where the transfer unit 104 places or disengages the target object 112). The transport unit 106 (e.g., a conveyor, an automated guided vehicle (AGV), a shelf transport robot, etc.) can transfer the target object 112 between (a) an area associated with the transfer unit 104 and (b) an area associated with the loading unit 108. The loading unit 108 can transfer the target object 112 between the transfer unit 104 and a storage location (e.g., a location on a shelf) (e.g., by moving a pallet on which the target object 112 is placed).

[0019] In some embodiments, the robot system 100 can include sensors 116 such as two-dimensional imaging sensors and three-dimensional imaging sensors. For example, the robot system 100 can include sensors 116 disposed above the transfer source position, such as one or more top-down facing sensors 6. The sensors 116 disposed above the transfer source position can be used, for example, to recognize an object 112 (e.g., an unknown object, an unregistered object, a known object, and / or a registered object) at the transfer source position and / or to calculate the dimensions of the object 112 (e.g., the length and / or width of the upper surface). In some embodiments, the robot system 100 can process the sensor information of the upper surface of the target object 112 captured using the sensors 116 to calculate a detection result that may or may not correspond to a registered object (e.g., an object having corresponding information included in the master data).

[0020] For illustrative purposes, robot system 100 is described in the context of a packing and / or shipping center. However, it will be understood that robot system 100 can be configured to perform tasks in other environments / for other purposes, such as manufacturing, assembly, storage / warehousing, healthcare, and / or other types of automation. It will also be understood that robot system 100 can include other units, not shown in FIG. 1, such as manipulators, service robots, modular robots, etc. For example, in some embodiments, robot system 100 includes a loading unit (e.g., unloading unit 102), a depalletizing unit (e.g., transfer unit 104) for transferring an object from a cage cart or pallet to a conveyor (e.g., conveyor 107 or another conveyor) or another pallet, a container replacement unit for transferring an object from one container to another, a packing unit for wrapping / packaging an object, a sorting unit for grouping objects according to one or more characteristics of the objects, a piece picking unit (e.g., unloading unit 102, transfer unit 104, or another unit) for operating on the objects separately according to one or more characteristics of the objects (e.g., for sorting, grouping, and / or transferring), or combinations thereof.

[0021] appropriate system FIG. 2 is a partial schematic block diagram of a robot system 200 (e.g., the robot system 100 of FIG. 1 or another robot system) configured according to various embodiments of the present technology. In some embodiments, the robot system 200 (e.g., one or more of the units and / or robots described above) can include electronic / electrical devices such as one or more processors 202, one or more storage devices 204, one or more communication devices 206, one or more input / output devices 208, one or more actuating devices 212, one or more conveyance motors 214, one or more sensors 216, or combinations thereof. The various devices can be coupled to each other via wired connections and / or wireless connections. For example, the robot system 200 can include a communication path 218 (e.g., a bus) such as a system bus, a Peripheral Component Interconnect (PCI) bus or a PCI-Express bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a Small Computer System Interface (SCSI) bus, a Universal Serial Bus (USB), an IIC (I2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (also referred to as "FireWire"). Also, for example, the robot system 200 can include bridges, adapters, processors, or other signal-related devices to provide wired connections between devices. The wireless connection can be based on, for example, cellular communication protocols (e.g., 3G, 4G, LTE, 5G, etc.), wireless local area network (LAN) protocols (e.g., Wi-Fi (Wireless Fidelity)), peer-to-peer or device-to-device communication protocols (e.g., Bluetooth, Near Field Communication (NFC), etc.), Internet of Things (IoT) protocols (e.g., NB-IoT, LTE-M, etc.), and / or other wireless communication protocols.

[0022] The processor 202 may include a data processor (e.g., a central processing unit (CPU), a dedicated computer, and / or an on-board server) configured to execute instructions (e.g., software instructions) stored in a memory device 204 (e.g., a computer memory). In some embodiments, the processor 202 may be included in a separate / standalone controller operably coupled to other electronic / electrical devices shown in FIG. 2 and / or the robot unit shown in FIG. 1. The processor 202 can implement program instructions for controlling other devices / interface with other devices, thereby causing the robot system 200 to execute actions, tasks, and / or operations.

[0023] The memory device 204 may include a non-transitory computer-readable medium in which program instructions (e.g., software 210) are stored. Some examples of the memory device 204 can include volatile memory (e.g., cache and / or random access memory (RAM)) and / or non-volatile memory (e.g., flash memory and / or magnetic disk drive). Other embodiments of the memory device 204 may include portable memory and / or cloud memory devices.

[0024] In some embodiments, the memory device 204 can be used to further store processing results and / or predetermined data / thresholds and provide access thereto. For example, the memory device 204 can store master data 246 that includes descriptions of objects (e.g., boxes, cases, and / or products) that can be manipulated by the robotic system 200. In one or more embodiments, the master data 246 can include dimensions, shapes (e.g., templates for potential postures and / or computer-generated models for recognizing objects in different postures), color schemes, images, identification information (e.g., barcodes, quick response (QR) codes, logos, etc., and / or their expected positions), expected weights, other physical / visual characteristics, or combinations thereof of the objects that are expected to be manipulated by the robotic system 200. In some embodiments, the master data 246 can include operation-related information about the objects, such as the center of mass (CoM) position of each object, expected sensor measurements (e.g., for forces, torques, pressures, and / or contact measurements) corresponding to one or more actions / operations, or combinations thereof.

[0025] The communication device 206 can include circuitry configured to communicate with external or remote devices via a network. For example, the communication device 206 can include communication input / output devices 248 such as receivers, transmitters, transceivers, modulators / demodulators (modems), signal detectors, signal encoders / decoders, connector ports, network cards, etc. The communication device 206 can be configured to transmit, receive, and / or process electrical signals according to one or more communication protocols (e.g., Internet Protocol (IP), wireless communication protocols, etc.). In some embodiments, the robotic system 200 can use the communication device 206 to exchange information between units of the robotic system 200 (e.g., for reporting, data collection, analysis, and / or troubleshooting purposes) and / or to exchange information with systems or devices external to the robotic system 200.

[0026] The input / output device 208 may include a user interface device configured to communicate information to and / or receive information from a human operator. For example, the input / output device 208 may include a display 250 and / or other output devices (such as speakers, tactile circuits, or tactile feedback devices, etc.) for communicating information to a human operator. Also, the input / output device 208 may include control devices or receiving devices such as keyboards, mice, touchscreens, microphones, user interface (UI) sensors (such as cameras for receiving motion commands), wearable input devices, etc. In some embodiments, the robot system 200 may interact with a human operator using the input / output device 208 when performing actions, tasks, operations, or combinations thereof.

[0027] The robot system 200 may include physical or structural members (such as a robot manipulator arm) connected by joints for motion (such as rotational displacement and / or translational displacement). The structural members and joints may form a kinematic chain configured to manipulate an end effector (such as a gripper) configured to perform one or more tasks (such as grasping, rotating, welding, etc.) depending on the use / operation of the robot system 200. An actuation device 212 (such as a motor, actuator, wire, artificial muscle, electroactive polymer, etc.) may be configured to drive or manipulate (such as displace and / or reorient) the structural members around or at the corresponding joints. In some embodiments, the transport motor 214 may be configured to transport the corresponding unit / chassis back and forth.

[0028] Sensor 216 can be configured to obtain information used to perform various tasks such as the operation of structural members and / or the conveyance of objects. Sensor 216 can include a device configured to detect or measure one or more physical characteristics of the robot system 200, one or more objects (e.g., individual object 112 in FIG. 1), and / or the surrounding environment (e.g., the state, condition, and / or position of one or more structural members / their joints). Some examples of sensor 216 can include an accelerometer, a gyroscope, a force sensor, a weight sensor or transducer, a distance sensor, an image sensor, a strain gauge, a tactile sensor, a torque sensor, a position encoder, etc.

[0029] In some embodiments, for example, sensor 216 can include one or more imaging devices 222 configured to detect the surrounding environment (e.g., a visual camera and / or an infrared camera, a 2D and / or 3D imaging camera, a distance measurement device such as a lidar or radar, etc.). The imaging device 222 can generate a representation of the detected environment, such as a digital image and / or a point cloud, which can be processed via machine / computer vision (e.g., for automatic inspection, robot guidance, or other robot applications). The robot system 200 can process the digital image and / or the point cloud (e.g., via processor 202) to identify the target object, one or more dimensions of the target object (e.g., length, width, and / or height dimensions), the pick-up / start / transfer source position, the drop / end / transfer destination / task position, the pose of the target object, a reliability measure regarding the start position and / or pose, or a combination thereof.

[0030] To manipulate the target object, the robot system 200 can capture and analyze image data within a specified range (e.g., a pick-up position such as within a track, on a pallet, or on a belt conveyor) (e.g., via the various circuits / devices described above) to identify the target object and its starting position. Similarly, the robot system 200 can capture and analyze image data within another specified range (e.g., a drop position for placing the target object on a conveyor, a position for placing the target object within a container, or a position on a pallet for stacking) to identify the task position of the target object. For example, the imaging device 222 can include one or more cameras configured to generate image data of the pick-up range and / or one or more cameras configured to generate image data of the task range (e.g., the drop range). Based on the image data, as described below, the robot system 200 can determine the starting position, task position, associated posture, packing / placement position, and / or other processing results.

[0031] In some embodiments, the sensor 216 can include a contact sensor 226 (e.g., a pressure sensor, a force sensor, a strain gauge, a piezoresistive / piezoelectric sensor, a capacitance sensor, a piezoresistive sensor, and / or other tactile sensors) configured to measure one or more characteristics related to direct contact between a plurality of physical structures or surfaces. The contact sensor 226 can measure characteristics corresponding to the gripping of an end effector (e.g., a gripper) on the target object. Thus, the contact sensor 226 can output a contact quantity representing a quantified quantity (e.g., a measured force, torque, position, etc.) corresponding to the degree of contact or adhesion between the gripper and the target object. For example, the contact quantity can include measurements of one or more forces or torques related to the force applied to the target object by the end effector.

[0032] In these and other embodiments, for example, sensor 216 can include a position sensor 224 (such as a position encoder, a potentiometer, a distance sensor, etc.) configured to detect structural members of the robotic system 200 (such as the robotic arm and / or corresponding end effector of the robotic system 200), the position of corresponding joints, and / or other objects (such as the individual objects 112 of FIG. 1, the target object, other obstacles, etc.). The robotic system 200 can use the position sensor 224 to track the position and / or orientation of structural members, joints, and / or other objects during the execution of various tasks. In these and still other embodiments, sensor 216 can include a weight sensor (such as a weight transducer) for determining, for example, the weight of a target object grasped by the end effector of the robotic system 200.

[0033] Operation of the System FIG. 3 is a partial schematic view of a motion plan 330 of a robotic system 300 (such as the robotic system 100 of FIG. 1, the robotic system 200 of FIG. 2, or another robotic system) configured according to various embodiments of the present technology. The motion plan 330 can represent a series of actions or movements to be executed by the robotic system 300 (such as by one of the above-described units such as the robotic arm 305 and / or end effector 309 of the transfer unit 304) to achieve a goal or complete a task. As shown in FIG. 3, for example, the motion plan 330 can be generated and / or implemented to move the target object 312 from a source position 314 (such as a position above or within a conveyor, pallet, bin, etc.) to a task or destination position 318 (such as another position above or within a conveyor, pallet, bin, etc.).

[0034] In some embodiments, the robot system 300 may generate a detection result corresponding to an object at the source position 314. For example, the robot system 300 may image or monitor a predetermined area to identify and / or specify the source position 314. As a specific example, the robot system 300 may include a source sensor (e.g., an example of the sensor 116 in FIG. 1 and / or the sensor 216 in FIG. 2) directed toward a pickup area such as a sorting pallet, a sorting bin, and / or an area designated for a sorting area on the receiving side of the conveyor. The robot system 300 may use the source sensor to generate image data (e.g., captured images and / or point clouds) of the pickup area and / or other sensor data. The robot system 300 may perform computer vision and / or other processes on the image and / or other sensor data to identify different objects (e.g., boxes or cases) located in the pickup area and / or determine one or more dimensions of the object (e.g., length, width, etc. related to the upper surface). From the recognized objects, the robot system 300 may select an object as the target object 312 (e.g., according to a predetermined order or a series of rules and / or a template of the object contour). For the selected target object 312, the robot system 300 may further process the image and / or other sensor data to determine the source position 314 and / or the initial posture of the target object 312.

[0035] The robot system 300 may further image or monitor another predetermined area to identify the destination position 318. In some embodiments, for example, the robot system 300 may include a destination sensor (e.g., another example of the sensor 116 in FIG. 1 and / or the sensor 216 in FIG. 2) configured to generate image data and / or other sensor data of an arrangement area such as a destination pallet on the feed side of the conveyor, a destination bin, and / or an area designated for the destination area. The robot system 300 may use the destination sensor to generate image data (e.g., captured images and / or point clouds) of the arrangement area and / or other sensor data. The robot system 300 may perform computer vision and / or other processes on the image and / or other sensor data to identify the destination position 318 and / or the corresponding posture for placing the target object 312. In some embodiments, the robot system 300 may identify the destination position 318 (based on or without based on the image and / or other sensor data) according to a predetermined order or a series of rules for stacking, arranging, and / or placing one or more objects.

[0036] Using the identified source location 314 and / or the identified destination location 318, the robot system 300 may perform a task of moving the selected target object 312 from the source location 314 to the destination location 318 by operating one or more structures of the corresponding unit (e.g., the transfer unit 304), such as the robot arm 305 and / or the end effector 309. More specifically, the robot system 300 may derive or calculate (e.g., via motion planning rules or algorithms) a motion plan 330 corresponding to one or more actions to be performed by the corresponding unit to perform the task. Generally, the motion plan 330 may include a source trajectory related to gripping the target object 312 at the source location 314, a transfer trajectory related to transferring the target object 312 from the source location 314 to the destination location 318, a destination trajectory related to releasing the target object 312 at the destination location 318, and / or a return trajectory related to returning the subsequent motion plan and / or the corresponding unit to the starting position.

[0037] In the specific example shown in FIG. 3, the motion plan 330 of the transfer unit 304 includes a transfer approach path 331 that specifies one or more trajectories of the robot arm 305 of the transfer unit 304 and / or the end effector 309 to position the end effector 309 at the transfer source approach position, a gripping approach path 332 that specifies one or more trajectories and / or operations of the robot arm 305 and / or the end effector 309 to position and / or operate the end effector 309 to grip or otherwise engage the target object 312 at the transfer source position 314, and / or a gripping release path 333 that specifies one or more trajectories of the robot arm 305 and / or the end effector 309 to move the target object 312 away from the transfer source position 314. The motion plan further includes transfer paths 334 and 335 that specify one or more trajectories to move the robot arm 305 and / or the end effector 307 to transfer the target object 312 toward the transfer destination position 318. Additionally, the motion plan 330 includes a transfer destination approach path 336 that specifies one or more trajectories and / or operations of the robot arm 305 of the transfer unit 304 and / or the end effector 309 to position and / or operate the end effector 309 to place the target object 312 at the transfer destination position 318 or otherwise disengage / release it, a transfer destination release path 337 that specifies one or more trajectories of the robot arm 305 and / or the end effector 309 of the transfer unit 304 to position the end effector 309 at the release position, and / or a return path 338 that specifies one or more trajectories of the robot arm 305 and / or the end effector 309 to position the end effector 309 of the transfer unit 304 at the start position (e.g., in preparation for or as part of the execution of the next task, which includes transferring another object from the transfer source position 314 to the transfer destination position 318).

[0038] In some embodiments, the starting position can be the default position of the end effector 309. For example, the starting position can be the position to which the end effector 309 is returned by default settings after placing the target object 312 at the destination position 318. As another example, the starting position can be a storage location or an idle position where the end effector 309 is displaced laterally / off-track, and / or the position where the transfer unit 304 positions the end effector 309 while the robot system 300 derives or waits for further commands (e.g., to transfer the next target object between the source position and the destination position).

[0039] In these and other embodiments, the starting position can be the position at which the transfer unit 304 positions the end effector 309 to implement (or as part of implementing) the next transfer approach path and / or the next gripping approach path of the next motion plan derived to transfer the next object between the source position and the destination position. For example, the starting position can be the starting position of the next transfer approach path and / or the next gripping approach path that can be implemented by the robot system 300 to transfer the next object between the source position and the destination position according to the next motion plan. In other words, the return path 338 can be linked to the start of one or more paths of the next motion plan. Thus, after placing the object 312 at the destination position 318, the robot system 300 can implement the return path 338 in the motion plan 330 so as to be able to implement the next transfer approach path and / or the next gripping approach path of the next motion plan to transfer the next object. As another example, the next transfer approach path and / or the next gripping approach path of the next motion plan for the next object can include at least a portion of the return path of the motion plan 330 for the object 312. Thus, when the robot system 300 implements the return path 338 of the motion plan 330, the robot system 300 can also implement at least a portion of the next transfer approach path and / or the next gripping approach path of the next motion plan for the next object. In any of these examples, the starting position specified by the return path 338 can depend at least in part on the next object (e.g., the position, orientation, characteristics of the next object) and / or the next motion plan. Additionally, or alternatively, the next motion plan can depend at least in part on the return path 338.

[0040] In some embodiments, the robot system 300 can derive or calculate a motion plan 330 by determining a series of commands and / or settings for one or more actuating devices (e.g., the actuating device 212 of FIG. 2) that operate the robot arm 305 and / or the end effector 309. For example, the robot system 300 uses a processor to calculate commands and / or settings for the actuating device to operate the end effector 309 and / or the robot arm 305, to place the end effector 309 (e.g., a gripper) at an approach position around the source position 314, to engage and grip the target object 312 with the end effector 309, to place the end effector 309 at a specific position around the destination position 318, to release the target object 312 from the end effector 309 at or near the destination position 318, and / or to return the end effector 309 to the starting position. All or a subset of the series of commands and / or settings can be derived or calculated in advance (e.g., before the robot system 300 executes all or a subset of the motion plan 330). In these and other embodiments, all or a subset of the series of commands and / or settings can be derived and / or calculated dynamically (e.g., in real time and / or when the robot system 300 executes all or a subset of the motion plan 330). In these and still other embodiments, all or a subset of the series of commands and / or settings can be re-derived or re-calculated (e.g., taking into account new information determined or made available to the robot system 300, such as the actual height of an unregistered object, as described in more detail below). The robot system 300 can execute actions to complete a task by operating the actuating device according to the determined series of commands and / or settings.

[0041] When performing operations related to the motion plan 330, the robot system 300 may track the current position (e.g., a set of coordinates corresponding to the grid used by the robot system 300) and / or the current orientation of the target object 312. For example, the robot system 300 may (e.g., via one or more processors such as the processor 202 of FIG. 2) track the current position / orientation according to data from a position sensor (e.g., the position sensor 224 of FIG. 2). As a specific example, the robot system 300 can identify one or more parts of the robot arm 305 (e.g., structural members and / or its joints) within the kinematic chain according to data from the position sensor. The robot system 300 can further calculate, for example, based on the position and orientation of the robot arm 305, the position and / or orientation of the end effector 309 (and thus the current position of at least the upper surface of the target object 312 held by the end effector 309). In some embodiments, the robot system 300 can track the current position of the robot arm 305 and / or the end effector 309, for example, according to a dead reckoning mechanism, based on the processing of other sensor measurements (e.g., force measurements or accelerometer measurements), executed actuation commands / settings, and / or related timings, or a combination thereof.

[0042] (Registered and / or unregistered) object transfer FIG. 4 is a partial schematic perspective view of another environment 450 in which a robot system 400 equipped with a cooperative transfer mechanism can operate according to various embodiments of the present technology. As shown, the robot system 400 includes a transfer unit 404 having a robot arm 405 and an end effector 409 (e.g., a gripper). The robot system 400 can be the robot system 100, 200, and / or 300 of FIGS. 1-3, or another robot system of the present technology. In some embodiments, the robot system 400 can be used to transfer the object 412 from the transfer source position 414 to the transfer destination position 418. In the illustrated embodiment, the transfer destination position 418 includes a designated area on the feed side of the conveyor 407.

[0043] The object 412 at the source position 414 may include registered and / or unregistered objects. Registered objects include objects having one or more characteristics or traits that are included in, stored in, or registered in the master data of the robot system 400 (e.g., the master data 246 of FIG. 2), and thus are "known" to the robot system 400. Unregistered objects may include objects having one or more characteristics or traits that are not included in, not stored in, or not registered in the master data of the robot system 400, and thus are "unknown" to the robot system 400 (at least initially). One or more characteristics or traits of the registered and / or unregistered objects may include physical dimensions (e.g., the length, width, and / or height dimensions of one or more sides of the object), shape, center of gravity position, weight, SKU, vulnerability assessment, and the like.

[0044] As shown in FIG. 4, the objects 412 at the source position 414 may have one or more different characteristics and / or traits from each other. In other embodiments, the objects 412 at the source position 414 may have uniform characteristics and / or traits. If there are unregistered objects 412 at the source position 414, in some embodiments, the robot system 400 may be provided with the maximum and / or minimum possible values for one or more characteristics or traits of the unregistered objects 412 (e.g., the maximum and / or minimum possible dimensions of the unregistered objects 412). As will be described in more detail below, the robot system 400 may derive a motion plan for transferring the unregistered object 412 from the source position 414 to the destination position 418 based at least in part on the maximum and / or minimum possible values.

[0045] Robot system 400 can generate a detection result corresponding to an object at the source position 414, consistent with the above discussion. For example, robot system 400 may include a scanner or sensor 416 disposed at, on, or around the source position 414. As a specific example, robot system 400 may include a two-dimensional and / or three-dimensional imaging sensor 416 disposed above the source position 414 such that the object 412 at the source position 414 is within the field of view(s) of the imaging sensor 416. Robot system 400 may utilize the sensor 416 at the source position 414 to determine one or more characteristics or traits of the object 412 at the source position 414 and / or to detect or identify the target object 412 at the source position 414.

[0046] For example, in the case of a registered object 412 at the source position 414, robot system 400 may utilize information corresponding to the registered object 412 (e.g., information captured by the sensor 416 at the source position 414) to detect or identify the registered object 412 and / or to obtain corresponding characteristics and / or traits from the master data. Continuing with this example, robot system 400 may derive a motion plan (e.g., a motion plan similar to the motion plan 330 of FIG. 3) for transferring the registered object 412 from the source position 414 to the destination position 418 based at least in part on the detected, obtained, and / or known characteristics or traits corresponding to the registered object 412.

[0047] In the case of an unregistered object 412 at the source position 414, the robot system 400 can detect or identify the unregistered object 412 and / or calculate one or more characteristics of the unregistered object 412 by using information corresponding to the unregistered object 412 captured by the sensor 416 at the source position 414. As a specific example, the robot system 400 can use the sensor 416 at the source position 414 to image the unregistered object 412 (e.g., the upper surface) at the source position 414, and use the image to estimate the dimensions of the unregistered object 412 (e.g., the length and / or width of the upper surface). As a result, the robot system 400 can derive a motion plan (e.g., a motion plan similar to the motion plan 330 in FIG. 3) for transferring the unregistered object 412 from the source position 414 to the destination position 418 based at least in part on the estimated dimensions of the unregistered object 412.

[0048] In some embodiments, the robot system 400 can calculate a motion plan for gripping the object 412, with and / or without knowledge of one or more characteristics or traits of the object 412, transfer the object 412 to or near the destination position 418, and / or place the object 412 at the destination position 418. As a specific example, it may be difficult to accurately determine the height of the target object 412 at the source position 414. Continuing with this example, therefore, the robot system 400 engages the target object 412 at the source position 414, transfers the target object 412 from the source position 414 towards the destination position 418, and / or derives a motion plan for placing the target object 412 at the destination position 418 based at least in part on the maximum possible height value and / or the minimum possible height value of all the objects 412 at the source position 414 provided to the robot system 400.

[0049] To clarify, consider a partial schematic side view of a robot system 400 shown in FIG. 5 that illustrates an example of an end effector 409 positioned above or around a destination position 418 on a conveyor 407. In the example shown, the robot system 400 is provided with (i) the maximum possible height of the object 412 (represented by the line segment H1) at the source position 414 (FIG. 4), and / or (ii) the minimum possible height of the object 412 at the source position 414 (represented by the line segment H2). Continuing with this example, the robot system 400 can derive one or more default or pre-calculated motion trajectories and / or corresponding motion speeds for a motion plan (similar to the motion plan 330 of FIG. 3), based on, for example, the maximum possible height H1 and / or the minimum possible height H2, where this motion plan can be implemented by the robot system 400 to transfer the target object 412 from the source position 414 to the destination position 418. The default motion trajectory can include a default grasping approach path for engaging the target object 412, a default grasping release path for moving the target object 412 away from the source position 414, one or more default transfer paths for positioning the end effector 409 (and / or the object 412 engaged by the end effector 409) in the position shown in FIG. 5, a default destination approach path 536 for moving the target object 412 towards and / or positioning the target object 412 at the destination position 418, a default destination release path 537 for moving the end effector 409 away from the destination position 418, and / or a default return path 538 for returning the end effector 409 to the starting position. In some embodiments, the robot system 400 can further calculate (e.g., pre-calculate) a default speed corresponding to one or more of the above-described default motion paths. The default speed can specify the speed at which the end effector 409 and / or the robot arm 405 (FIG. 4) of the robot system 400 are moved while implementing the corresponding default motion path(s).

[0050] However, without knowing the actual height of the target object 412, it may be difficult for the robot system 400 to place the target object 412 at the transfer destination position 418 even using a pre-calculated default motion path / speed. For example, it may be difficult for the robot system 400 to determine how far to lower the robot arm 405 of the transfer unit 404 along the default transfer destination approach path 536 towards the transfer destination position 418 before releasing the target object 412 without knowing the height of the target object. Additionally, since the robot system 400 does not recognize the position of the bottom surface of the target object 412 relative to the end effector 409, it is difficult to calculate an optimized gripping approach path, an optimized gripping release path, an optimized transfer path, an optimized transfer destination approach path, an optimized transfer destination release path, an optimized return path, and / or one or more corresponding optimized motion speeds that reduce or minimize the time spent transferring the target object 412 to the transfer destination position 418 and / or returning the end effector 409 to the starting position.

[0051] Accordingly, in some embodiments, the robotic system 400 may include one or more sensors to determine the height measurement of the object 412 and / or the position of the bottom surface of the object 412. For example, FIG. 6 is a partial schematic side view of the robotic system 400 that positions the target object 412 at the destination position 418 on the conveyor 407. More specifically, FIG. 6 shows an example where the end effector 409 of the robotic system 400 grips the target object 412 such that the target object 412 is positioned above or around the destination position 418 above the rollers of the conveyor 407. The robotic system 400 is further shown as including an upper horizontal line sensor 617a and a lower horizontal line sensor 617b. In some embodiments, the position of the upper horizontal line sensor 617a and / or the position of the lower horizontal line sensor 617b (e.g., the position relative to the conveyor 407 and / or the position relative to each other) may be known and / or tracked by the robotic system 400.

[0052] When the robotic system 400 lowers the target object 412 along the default destination approach path 536 towards the destination position 418, the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b may be used to detect the bottom surface of the target object 412 and / or determine the height of the target object 412. For example, as described above, the robotic system 400 may track the position of the end effector 409 (e.g., the bottom surface). Thus, when (i) the robotic system 400 lowers the target object 412 towards the destination position 418 and (ii) the upper horizontal line sensor 617a detects the target object 412 (e.g., the bottom surface), the height of the target object 412 can be determined using Equation 1 below using the known vertical positions of the end effector 409 and the upper horizontal line sensor 617a at the time when the upper horizontal line sensor 617a detects the bottom surface of the target object 412. Equation 1: Height of target object = Vertical position of end effector - Vertical position of horizontal line sensor In these and other embodiments, in addition to, or instead of, the upper horizontal line sensor 617a, the robotic system 400 may use a lower horizontal line sensor 617b to determine the height of the target object 412 based on the known positions of the end effector 409 and the lower horizontal line sensor 617b at the time the lower horizontal line sensor 617b detects the bottom surface of the target object 412.

[0053] In addition, or alternatively, when the robotic system 400 lowers the target object 412 along the destination approach path 536 toward the destination position 418, the robotic system 400 may use the lower horizontal line sensor 617b to determine when to release or disengage the target object 412 in order to place the target object 412 at the destination position 418. For example, the lower horizontal line sensor 617b may be positioned at a location above the conveyor 407. This location may correspond to a specific distance (e.g., release height) above the conveyor 407 at which the robotic system 400 can safely release the target object 412 (e.g., without damaging the target object 412, without risking the target object 412 falling from the conveyor 407, etc.) and place the target object 412 at the destination position 418. Continuing with this example, when the robotic system 400 lowers the target object 412 along the destination approach path 536 toward the destination position 418, the lower horizontal line sensor 617b may detect that the bottom surface of the target object 412 is positioned at the specified distance above the conveyor 407. At this point, the robotic system 400 can release or disengage the target object 412 and place the target object 412 at the destination position 418.

[0054] However, there are several drawbacks to using horizontal line sensors similar to the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b to detect the height of the target object 412 and / or to determine when to release the target objects 412 and place them at the destination position 418. For example, the robotic system 400 cannot detect the bottom surface of the target object 412 and / or calculate the height of the target object 412 until the bottom surface of the target object 412 is lowered to the height level of the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b and detected by them. Thus, prior to lowering or otherwise positioning the target object 412 within the field of view of the upper horizontal line sensor 617a and / or within the field of view of the lower horizontal line sensor 617b, the robotic system 400 cannot calculate an optimized trajectory (e.g., a destination approach path, a destination departure path, and / or a return path) and / or a corresponding optimized motion speed that reduces or minimizes the time spent placing the target object 412 at the destination position 418 and / or returning the end effector 409 to the starting position.

[0055] In addition, the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b are generally installed near or closest to the destination position 418 on the conveyor 407. Thus, there may not be enough time for the robotic system 400 to dynamically recalculate or adjust a default or pre-calculated trajectory (e.g., a pre-calculated destination approach path 536, a pre-calculated destination departure path, and / or a pre-calculated return path) and / or a corresponding motion speed to optimize such trajectory / speed before (i) the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b detect the target object 412 and (ii) the robotic system 400 can use the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b to determine the height of the target object 412.

[0056] Note that the position of the lower horizontal line sensor 617b with respect to the conveyor 407 is usually fixed. Therefore, without adjusting the position of the lower horizontal line sensor 617b with respect to the conveyor 407, the robot system 400 can be configured to release each target object 412 from the same height above the conveyor 407. In other words, the robot system 400 cannot adjust or regulate the release height of the target object 412 based on one or more characteristics or properties of the target object 412 (e.g., weight, center of gravity position, size, shape, etc.).

[0057] Furthermore, when the positions of the upper horizontal line sensor 617a and the lower horizontal line sensor 617b are above the conveyor 407, the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b can act as obstacles when returning the end effector 409 to the starting position. Therefore, before moving the end effector 409 along the return path to return the end effector 409 to the starting position, the robot system 400 may first be required to move the end effector 409 along a pre-calculated transfer destination departure path to raise the end effector 409 to a specific height beyond the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b. Such movement of the end effector 409 may correspond to a delay in the process of returning the end effector 409 to the starting position after placing the target object at the transfer destination position 418.

[0058] To address one or more of these issues, the robotic system 400 may use one or more vertically oriented sensors in addition to, or instead of, the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b. FIGS. 7A and 7B are, respectively, a partial schematic side perspective view and a top perspective view of an example of such a vertically oriented sensor 745. In some embodiments, the sensor 745 may be a distance sensor or another suitable type of sensor. In the illustrated embodiment, the sensor 745 may be positioned below the conveyor 407 and the destination location 418. More specifically, the sensor 745 may be arranged such that the field of view of the sensor 745 is (i) directed upward (vertically) and (ii) not at least partially blocked by the rollers of the conveyor 407. In some embodiments, the position of the sensor 745 below the conveyor 407 may be fixed (e.g., such that the distance between the destination location 418 and the sensor 745 on the upper surface of the rollers of the conveyor 407 is constant and / or known). As will be described in more detail below, the sensor 745 may be configured to monitor (e.g., determine a height measurement of the target object 412) an object (e.g., the target object 412 (FIG. 4), the end effector 409 (FIG. 4), etc.) disposed above the conveyor 407 and / or the destination location 418 through one or more gaps in the rollers of the conveyor 407.

[0059] Although the sensor 745 is shown under the conveyor 407 in the illustrated embodiment, it may be positioned at other locations within the robot system 400. For example, in some embodiments, the sensor 745 may be positioned at a location between the source position 414 (FIG. 4) and the destination position 418. As another example, the sensor 745 may be positioned at or near the source position 414. As yet another example, the sensor 745 may be positioned at or near the destination position 418, such as at a location not under the conveyor 407 and / or the destination position 418. In any of these other embodiments, the robot system 400 can derive a motion plan (e.g., similar to the motion plan 330 of FIG. 3) that presents the target object 412 within the field of view of the sensor 745 so that the robot system 400 can determine the actual height of the target object 412 while the robot system 400 transfers the target object 412 between the source position 414 and the destination position 418.

[0060] For example, consider FIGS. 8A - 8C, which are partial schematic side views of the end effector 409 of the robot system 400 that uses the sensor 745 to place a target object 812 (e.g., one of the objects 412 of FIG. 4) at the destination position 418 according to various embodiments of the present technology. The target object 812 can be a registered or unregistered object. Additionally, or alternatively, the height of the target object 812 may or may not be known to the robot system 400.

[0061] As shown in FIG. 8A, the end effector 409 is positioned above the conveyor 407 and the destination position 418 such that the bottom surface of the target object 812 is within the field of view of the sensor 745 through the gap within the rollers of the conveyor 407. As described above, the robot system 400 can track the position of the end effector 409 (e.g., the bottom surface). Thus, if both the position of the end effector 409 and the position of the sensor 745 are known to the robot system 400, the distance between the bottom surface of the end effector 409 and the sensor 745 (represented by arrow D1 in FIG. 8A) can also be known to the robot system 400. Further, when the target object 812 is presented within the field of view of the sensor 745, the robot system 400 can determine the distance between the bottom surface of the target object 812 and the sensor 745 (represented by arrow D2 in FIG. 8A). Once (i) the distance D1 between the end effector 409 and the sensor 745 is known and (ii) the distance D2 between the bottom surface of the target object 812 and the sensor 745 is known, the robot system 400 can determine the actual height measurement of the target object 812 (represented by arrow H3 in FIG. 8A) using Equation 2 below. Equation 2: Height of target object = Distance between end effector and sensor - Distance between target object and sensor That is, in the example shown in FIG. 8A, H3 = D1 - D2. In some embodiments, the robot system 400 can calculate the actual height measurement H3 of the target object 812 before moving the target object 812 towards the destination position 418 (e.g., before implementing a default or pre - calculated destination approach path such as the default destination approach path 536 of FIG. 5). In other embodiments, the robot system 400 can calculate the actual height measurement H3 of the target object 812 while moving the target object 812 towards the destination position 418 (e.g., while implementing a default or pre - calculated destination approach path such as the default destination approach path 536 of FIG. 5).

[0062] Referring now to FIG. 8B, once the actual height measurement H3 of the target object 812 is known, the robot system 400 can proceed to dynamically calculate a destination approach path 836 for moving the target object 812 toward the destination position 418. In some embodiments, dynamically calculating the destination approach path 836 can include dynamically adjusting or recalculating a pre-calculated / default destination approach path (e.g., the default destination approach path 536 of FIG. 5) to place the target object 812 at the destination position 418. For example, considering the actual height measurement H3 of the target object 812, the robot system 400 can determine the position of the bottom surface of the target object 812 (e.g., with respect to the bottom surface of the end effector 409, the upper surface of the roller of the conveyor 407, and / or the sensor 745). Using this information, the robot system 400 can lower the end effector 409 by a determined distance to position the bottom surface of the target object 812 at a specific distance above the destination position 418 on the upper surface of the roller of the conveyor 407 (represented by line segment D5 in FIG. 8B) and / or at a specific distance above the sensor 745 (represented by line segment D4 in FIG. 8B), and can determine a motion path (represented by the destination approach path 836 in FIG. 8B). Such a specific distance(s) is also referred to herein as the clearance height.

[0063] In some embodiments, the release height D5 may be constant across the placement of the plurality of target objects at the destination position 418. For example, the release height D5 may be invariant across the placement of all target objects (including the target object 812) at the destination position 418. As another example, the release height D5 may correspond to a group of target objects (including the target object 812) such that the robotic system 400 is configured to release all target objects of the group from the release height D5. In both of these examples, the release height D5 corresponds to a specified distance above the conveyor 407 such that the robotic system 400 can safely release the plurality of target objects (e.g., without damaging those target objects, without having the risk of those target objects falling from the conveyor 407, etc.) and place those target objects at the destination position 418.

[0064] In some embodiments, the release height D5 may vary across the placement of different target objects at the destination location 418. For example, the release height D5 may be variable and / or may depend at least in part on one or more characteristics or properties of a given target object (e.g., weight, shape, center of gravity position, fragility assessment, etc.). As a specific example, the release height D5 for the target object 812 may be smaller if the weight of the target object 812 is heavier and / or more fragile, and may be larger if the weight of the target object 812 is lighter and / or less fragile. As another specific example, the release height D5 of the target object 812 may be smaller if the shape of the target object 812 and / or the size / shape of the bottom surface of the target object 812 create a risk that the target object 812 will roll off the conveyor 407 or otherwise fall, and may be larger if the shape of the target object 812 and / or the size / shape of the bottom surface of the target object 812 are relatively flat or do not create a significant risk that the target object 812 will fall off the conveyor 407. In other words, in some embodiments, the release height D5 may be specific to the target object 812 and / or may correspond to one or more characteristics / properties of the target object 812. In some embodiments, the robotic system 400 may utilize one or more sensors (e.g., weight sensors, force sensors, imaging sensors, etc.) to determine one or more of the characteristics or properties of the target object, and / or may (e.g., dynamically) determine the release height of the target object based on the characteristics / properties of the target object.

[0065] The robot system 400 may employ any one or more of several possible methods for determining when the bottom surface of the target object 812 is at the release height D5. For example, the robot system 400 may determine that the bottom surface of the target object 812 is at the release height D5 by monitoring the motion of the end effector 409. For example, the position of the transfer destination position 418 above the roller of the conveyor 407 may be known to the robot system 400. Therefore, the robot system 400 can know the vertical distance between the bottom surface of the end effector 409 and the upper part of the roller of the conveyor 407. Therefore, when the distance obtained by subtracting the actual height measurement value H3 of the target object 812 from the vertical distance of the bottom surface of the end effector 409 above the roller of the conveyor 407 is equal to the release height D5, the robot system 400 can determine that the bottom surface of the target object 812 is at the release height D5. This is represented by Equation 3 below. Equation 3: Vertical height of the target object at the transfer destination position = Vertical height of the bottom surface of the end effector at the transfer destination position - Actual height measurement value of the target object Therefore, using the above Equation 3, when the value of the vertical height of the target object at the transfer destination position is equal to the specified and / or determined release height D5, the robot system 400 can determine that the bottom surface of the target object 812 is at the release height D5.

[0066] In addition, or alternatively, the robot system 400 may monitor the motion of the end effector 409 with respect to the position of the end effector 409 at the time (t0) when the robot system 400 uses the sensor 745 to determine the actual height measurement value H3 of the target object 812 (e.g., with respect to the position of the end effector 409 shown in FIG. 8A) to determine that the bottom surface of the target object 812 is at the release height D5. In such an embodiment, the robot system 400 may use Equation 4 below to determine that the bottom surface of the target object 812 is at the release height D5. Equation 4: Vertical height of the target object at the destination position = Vertical height of the bottom surface of the end effector at the destination position at time t0 - Vertical distance traversed by the end effector along the destination approach path after time t0 - Measured value of the actual height of the target object Therefore, using the above Equation 4, the robot system 400 can determine that the bottom surface of the target object 812 is at the release height D5 when the value of the vertical height of the target object at the destination position is equal to the specified and / or determined release height D5.

[0067] In these embodiments and still other embodiments, the distance between the sensor 745 and the destination position 418 at the top of the roller of the sensor 745 (represented by the line segment D3 in FIG. 8B) may be known to the robot system 400. Additionally, or alternatively, the robot system 400 can utilize the sensor 745 to determine the distance between the sensor 745 and the bottom surface of the target object 812. Therefore, the robot system 400 can determine that the bottom surface of the target object 812 is at the release height D5 using the following Equation 5 and / or Equation 6. Equation 5: Vertical height of the target object at the destination position = Distance between the sensor and the bottom surface of the target object - Distance between the sensor and the destination position Equation 6: Vertical height of the target object at the destination position = Distance between the end effector and the sensor - Measured value of the actual height of the target object - Distance between the sensor and the destination position Therefore, using the above Equation 5 and / or Equation 6, the robot system 400 can determine that the bottom surface of the target object 812 is at the release height D5 when the value of the vertical height of the target object at the destination position is equal to the specified and / or determined release height D5.

[0068] Returning to the description of the destination approach path 836 shown in FIG. 8B, the robot system 400 may (e.g., dynamically) determine the speed at which to lower the target object 812 along the destination approach path 836 toward the destination position 418 using knowledge of the position of the bottom surface of the target object 812. For example, without knowing the height of the target object 812 and / or the position of the bottom surface of the target object 812, the robot system 400 may: (a) to reduce damage to the target object 812 and / or the robot system 400 during a collision between the target object 812 and the robot system 400; (b) to provide sufficient time for the robot system 400 to determine the position of the bottom surface of the target object 812 and / or the height of the target object 812 (e.g., using the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b of FIG. 6) before the target object 812 reaches the conveyor 407; and / or (c) to provide sufficient time for the robot system 400 to calculate (e.g., recalculate) the destination approach path, the destination departure path, and / or the return path, it may be required to lower the target object 812 slowly toward the destination position 418. However, as described above, the robot system 400 can use the sensor 745 to determine the actual height measurement H3 of the target object 812 and the position of the bottom surface of the target object 812 before and / or at a relatively early stage of implementing the default destination approach path. In addition, the robot system 400 can use the sensor 745 to (e.g., continuously) monitor the position of the target object 812 (e.g., the bottom surface) while the end effector 409 is lowering the target object 812 toward the destination position 418. Therefore, since the robot system 400 can know and / or monitor the height of the target object 812 above the destination position 418 and / or the position of the bottom surface of the target object 812, the risk of collision between the target object 812 and the robot system 400 can be significantly reduced, minimized, and / or eliminated.Also, before or relatively early in the process of moving / lowering the target object 812 towards the destination position 418, the robot system 400 can determine the actual height measurement H3 of the target object 812 and / or the position of the bottom surface of the target object 812. Thus, the robot system 400 can provide sufficient time (e.g., dynamically) to calculate / recalculate the destination approach path 836, the destination departure path, and / or the return path. Therefore, by knowing the height of the target object 812 above the destination position 418 and / or the position of the bottom surface of the target object 812, the robot system 400 can lower the target object 812 along the destination approach path to the release height D5 (e.g., at an increased speed) earlier than would be possible without knowing the height of the target object 812 and / or the position of the bottom surface of the target object 812. In some embodiments, the robot system 400 can dynamically determine this increased speed when it knows the actual height measurement H3 of the target object 812 and / or the position of the bottom surface. In some scenarios, the increase in the speed at which the robot system 400 lowers the target object 812 towards the destination position 418 can lead to a reduction in the time it takes for the robot system 400 to place the target object 812 at the destination position 418.

[0069] Furthermore, knowledge of the actual height measurement H3 of the target object 812 can facilitate the robot system 400 to dynamically calculate (e.g., recalculate dynamically) the transfer destination departure path and / or return path of the robot system 400. For example, by knowing the actual height H3, when the bottom surface of the target object 812 is positioned at the release height D5, the robot system 400 may be able to determine the position of the upper surface of the target object 812 (and thus the bottom surface of the end effector 409). Therefore, knowledge of the actual height H3 of the target object 812 can facilitate the calculation of the transfer destination departure path and / or return path starting from the position where the end effector 409 will be positioned when the bottom surface of the target object 812 is positioned at the release height D5 and / or when the end effector 409 disengages (e.g., drops) the target object 812. Furthermore, in embodiments where the actual height H3 of the target object 812 is calculated by the robot system 400 at or near the start of the transfer destination approach path 836 (e.g., before or while the robot system 400 moves the target object 812 along the transfer destination approach path 836), the robot system 400 can have sufficient time to dynamically calculate the transfer destination departure path and / or return path.

[0070] For purposes of illustration and clarity, referring to FIG. 8C, the robot system 400 can use the actual height measurement H3 of the target object 812 to determine that when the bottom surface of the target object 812 is positioned at the release height D5 (FIG. 8B), the bottom surface of the end effector 409 is positioned at a position corresponding to the intersection of the default transfer destination departure path 537 and the arrow 839. Thus, in some embodiments (e.g., in embodiments where the robot system 400 includes an upper horizontal line sensor 617a and / or a lower horizontal line sensor 617b that act as obstacles to the end effector 409), the robot system 400 can (e.g., dynamically) recalculate the default transfer destination departure path 537 to generate an updated transfer destination departure path 837 (representing an upper portion or segment of the default transfer destination departure path 537). In these embodiments, after moving the end effector 409 along a path corresponding to the updated transfer destination departure path 837 (e.g., to position the end effector 409 at a specific height so as to avoid the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b), the robot system 400 can proceed to return the end effector 409 to the starting position by moving the end effector 409 along the default return path 538.

[0071] (i) Embodiments in which the robot system 400 does not include the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b, or (ii) Embodiments in which the upper horizontal line sensor 617a and / or the lower horizontal line sensor 617b do not act as obstacles to the end effector 409. In other embodiments, the robot system 400 may calculate (e.g., dynamically) a hybrid return path 839. As shown in FIG. 8C, the hybrid return path 839 represents a combination of the updated transfer destination departure path 837 and the default return path 538, or a recomputation that combines the default transfer destination departure path 537 and the default return path 538. In other words, the hybrid return path 839 may represent a "shortcut" between the start of the updated transfer destination departure path 837 and the end of the default return path 538. Thus, rather than first lifting the end effector 409 to a particular height corresponding to the updated transfer destination departure path 837 shown in FIG. 8C, the robot system 400 may begin to move (e.g., immediately) (e.g., horizontally) the end effector 409 along the hybrid return path 839 toward the starting position (e.g., the source position 414 shown in FIG. 4 or in the vicinity thereof). This can reduce the time required to return the end effector 409 to the starting position after placing the target object 812 at the transfer destination position 418.

[0072] In other words, the use of the sensor 745 of the robot system 400 to determine the actual height measurement value H3 of the target object 812 may facilitate the robot system 400 to calculate (e.g., dynamically) an optimized transfer destination approach path, an optimized transfer destination approach speed, an optimized transfer destination departure path, an optimized return path, and / or an optimized hybrid "shortcut" return path. Additionally, in embodiments where the sensor 745 is positioned at other locations (e.g., at or near the source position 414, between the source position 414 and the destination position 418, etc.), the robot system 400 can utilize the sensor 745 to determine the actual height measurement value H3 of the target object 812 at a point further upstream in the corresponding motion plan of the end effector 409. In such embodiments, the robot system 400 can calculate (e.g., dynamically) or optimize other paths (e.g., a source approach path, a grasping approach path, a grasping departure path, and / or a transfer path) for transferring the target object 812 from the source position 414 to the destination position 418.

[0073] FIGS. 9A - 9C are partial schematic side views of the end effector 409 of the robot system 400 for placing another target object 912 (e.g., another one of the objects 412 of FIG. 4) at the destination position 418 using the sensor 745, according to various embodiments of the present technology. The target object 912 can be a registered or unregistered object. Additionally, the height of the target object 912 may or may not be known to the robot system 400. Additionally, or alternatively, one or more characteristics or traits (e.g., weight, length, width, height, center of gravity position, vulnerability assessment, etc.) can be the same as, similar to, or different from the corresponding characteristics / traits of the target object 812 discussed above with reference to FIGS. 8A - 8C.

[0074] As shown in FIG. 9A, the end effector 409 is positioned above the conveyor 407 and the transfer destination position 418 such that the bottom surface of the target object 912 is within the field of view of the sensor 745 through the gap within the rollers of the conveyor 407. The position of the end effector 409 in FIG. 9A may be the same as or different from the position of the end effector 409 in FIG. 8A. Using the sensor 745, the robot system 400 may determine the actual height measurement H4 of the target object 912 and / or the position of the bottom surface of the target object 912 in a manner consistent with the above discussion. For example, the robot system 400 may use (i) the known distance D6 between the end effector 409 and the sensor 745, and (ii) the measured distance D7 between the bottom surface of the target object 912 and the sensor 745 to determine the actual height measurement H4 of the target object 912.

[0075] Referring now to FIG. 9B, once the actual height measurement H4 of the target object 912 is known, the robot system 400 may proceed to calculate / recalculate (e.g., dynamically) (i) a destination approach path 936 for moving the target object 912 toward the destination position 418 and / or (ii) a destination approach speed for moving / lowering the target object 912 toward the destination position 418, in accordance with the discussion of FIGS. 8A-8C above. The destination approach path 936 may be the same as, similar to, or different from the destination approach path 836 for the target object 812 of FIG. 8B. Additionally or alternatively, the destination approach speed for placing the target object 912 at the destination position 418 may be the same as, similar to, or different from the destination approach speed used to place the target objects 812 of FIGS. 8A and 8B at the destination position 418. After or while calculating the destination approach path 936 and / or the destination approach speed, the robot system 400 may begin to execute the destination approach path 936 to move / lower the target object 912 toward the destination position 418 (e.g., to position the bottom surface of the target object 912 at a release height D9 above the destination position 418 at the top of the rollers of the conveyor 407 and / or at a release height D8 above the sensor 745).

[0076] In some embodiments, the robot system 400 may determine (e.g., dynamically) the release height D9 and / or the release height D8. For example, the robot system 400 may determine the release height D9 and / or the release height D8 based at least in part on one or more characteristics or properties of the target object 912, in accordance with the discussion of FIGS. 8A-8C above. The release height D9 and / or the release height D8 of the target object 912 may be the same as or different from the release height D5 and / or the release height D4 of the target objects 812 of FIGS. 8A and 8B, respectively.

[0077] Referring to FIG. 9C, once the robot system 400 knows the release altitude D9 or D8 of the target object 912, and / or knows the position of the upper surface of the target object 912 when the bottom surface of the target object 912 is located at the release altitude D9 / D8, the robot system 400 can (e.g., dynamically) calculate / recalculate a transfer destination departure path 937 for raising the end effector 409 to a specific height after placing the target object 912 at the transfer destination position 418, a return path 538 for returning the end effector 409 to the starting position after raising the end effector 409 to a specific height along the transfer destination departure path 937, and / or a hybrid "shortcut" return path 939 for returning the end effector 409 to the starting position after placing the target object 912 at the transfer destination position 418. The transfer destination departure path 937, the default return path 538, and / or the hybrid return path 939 can be the same as, similar to, or different from the transfer destination departure path 837, the default return path 538, and / or the hybrid return path 839 described above with reference to FIG. 8C.

[0078] Therefore, using the sensor 745 in the robot system 400 can facilitate realizing several advantages compared to a robot system lacking such a sensor. For example, the robot system 400 can use the sensor 745 to determine the actual height of the target object at an initial stage of the corresponding motion plan (e.g., before or during moving the target object along the transfer destination approach path). Therefore, the robot system 400 can provide sufficient time to (e.g., dynamically) calculate, recalculate, and / or optimize various motion paths and / or corresponding speeds (e.g., transfer path, transfer destination return path, transfer destination approach speed, release altitude, transfer destination departure path, return path, hybrid return path, etc.) included in the motion plan. As a result, the time spent by the robot system 400 to place the target object at the transfer destination position 418 can be reduced and / or minimized compared to a robot system without a sensor similar to the sensor 745.

[0079] Furthermore, by using a vertically oriented sensor 745 to determine the actual height measurement value and / or position of the bottom surface of the object to be transferred with respect to the transfer destination position 418 above the roller of the conveyor 407, the robot system 400 can change, adjust, regulate, and / or customize the release height for different objects to be transferred (e.g., based on one or more characteristics or traits of these objects to be transferred) without the need to adjust the position of the sensor 745.

[0080] Furthermore, the sensor 745 can be disposed under the conveyor 407 and / or outside the trajectory of the end effector 409, and / or can serve as an obstacle for returning the end effector 409 to the starting position, and can be used instead of a horizontal line sensor (e.g., one or both of the upper horizontal line sensor 617a and the lower horizontal line sensor 617b in FIG. 6). Therefore, the use of the sensor 745 can facilitate omitting such a horizontal line sensor from the robot system 400, which can facilitate moving the end effector toward the starting position along a hybrid "shortcut" return path (e.g., immediately) after placing the object to be transferred at the transfer destination position 418 (e.g., without the need to first move the end effector 409 to a specific height). As a result, the time required for the robot system 400 to transfer the object to be transferred between the transfer source position 414 and the transfer destination position 418 can be shortened and / or minimized.

[0081] Operation Flow FIG. 10 shows a flowchart showing a method 1070 of operating a robot system according to various embodiments of the present technology. For example, method 1070 can be a method of operating a robot system to transfer (registered and / or unregistered) objects between a source location and a destination location. The robot system can be the robot system 100 of FIG. 1, the robot system 200 of FIG. 2, the robot system 300 of FIG. 3, the robot system 400 of FIGS. 4 to 9C, and / or another robot system of the present technology. Method 1070 is shown as a set of steps or blocks 1071 to 1076 having corresponding sub-blocks 1081 to 1093. All or a subset of one or more of blocks 1071 to 1076, and / or all or a subset of one or more of sub-blocks 1081 to 1093 can be executed by various components of the robot system (e.g., by the various components shown in any one or more of FIGS. 1 to 9C described above). Further, all or a subset of one or more of blocks 1071 to 1076, and / or all or a subset of one or more of sub-blocks 1081 to 1093 can be executed according to the above discussion.

[0082] Method 1070 begins in block 1071 by detecting a target object at the source location. The target object can be a registered or unregistered object. Additionally, or alternatively, the source location can be a pallet, a bin, a designated area on a conveyor, a stack of objects including the target object, etc.

[0083] Detecting the target object may include detecting the target object using one or more sensors of the robot system. For example, detecting the target object may include imaging a specified area using one or more imaging sensors and identifying the source position. As another example, detecting the target object may include using one or more imaging sensors to image the target object. Based on one or more images of the specified area and / or one or more images of the target object, the robot system may identify the source position and / or the target object at the source position.

[0084] As shown in sub-block 1081, detecting the target object may include estimating at least a part of the dimensions of the target object. For example, detecting the target object may include using one or more imaging sensors to image a part (e.g., the upper surface) of the target object. Continuing with this example, detecting the target object may include estimating the dimensions (e.g., length, width, etc.) of a part of the target object based at least in part on an image of the target object.

[0085] In block 1072, method 1070 continues by deriving a motion plan for moving the object of interest to a destination location, such as from a source location to a destination location. In some embodiments, deriving the motion plan may include deriving the motion plan based on one or more characteristics or traits of the object of interest registered in the master data of the robotic system. In these and other embodiments, deriving the motion plan may include deriving the motion plan based on default values (e.g., provided to the robotic system), such as a maximum possible height value for the object of interest and / or a minimum possible height value for the object of interest. Additionally or alternatively, deriving a motion plan for moving the object of interest may include determining one or more motion paths and / or one or more corresponding motion speeds for moving the robotic system (e.g., the robotic arm and / or end effector of the robotic system) and / or the object of interest toward the destination location.

[0086] For example, referring to sub-blocks 1082-1084, deriving a motion plan may include deriving a source approach path for moving the end effector to a position at or near the source position, deriving a gripping approach path for operating the end effector on the target object to cause the end effector to engage (e.g., grip) the target object, and / or deriving a gripping release path for moving / lifting the target object away from the source position after the target object has been engaged by the end effector. Additionally, or alternatively, referring to sub-block 1085, deriving a motion plan may include deriving one or more transfer paths for moving the target object between the source position and the destination position. In these and other embodiments, referring to sub-blocks 1086-1089, deriving a motion plan may include deriving a destination approach path for placing the target object at the destination position, deriving a destination release path for moving the end effector to the destination position and / or a specified height, and / or deriving a return path for moving the end effector to the starting position (e.g., at or near the source position, such as for transferring another target object from the source position to the destination position).

[0087] In block 1073, method 1070 continues by implementing a first portion of a motion plan for moving the object of interest to the destination location. Implementing the first portion of the motion plan can include moving a robotic system (e.g., a robotic arm and / or end effector) toward the source location along a source approach path, moving the robotic system toward the object of interest such that the end effector engages the object of interest along a grasping approach path and / or operating the robotic system, and / or moving the robotic system and the object of interest away from the source location along a release path. Additionally, or alternatively, implementing the first portion of the motion plan can include moving a robotic system (e.g., a robotic arm and / or end effector) toward the destination location along a (one or more) transfer path(s). In these and still other embodiments, implementing the first portion of the motion plan can include moving the object of interest toward the destination location along at least a portion of a destination approach path.

[0088] As shown in sub-block 1089, implementing the first part of the motion plan may include presenting the target object to a sensor such as a distance sensor similar to the distance sensor 745 discussed in detail above. Presenting the target object to the sensor may include positioning the target object above the sensor and / or within the field of view of the sensor. In an embodiment where the sensor is positioned below the destination position located on the upper surface of the conveyor roller, presenting the target object to the sensor may include positioning the target object above the destination position and within the field of view of the sensor that extends unobstructed through the gap between the conveyor rollers. Alternatively, in an embodiment where the sensor is positioned at another position, such as a position between the source position and the destination position, presenting the target object to the sensor may include positioning the target object at a position within the field of view of the sensor at the other position. In these and other embodiments, presenting the target object to the sensor includes positioning the target object such that (i) the target object is within the field of view of the sensor and (ii) the end effector of the robotic system is positioned on the side of the target object opposite the sensor.

[0089] In block 1074, method 1070 continues by determining the height of the target object. Determining the height of the target object can include determining a first distance between a part of the robotic system and the sensor. For example, determining the height of the target object can include determining a first distance between the bottom surface of the end effector of the robotic system and the sensor. Continuing with this example, determining the first distance can include tracking the position of the bottom surface of the end effector or determining it in some other way. Determining the height of the target object can additionally or alternatively include determining a second distance between the target object and the sensor. For example, determining the second distance can include receiving sensor data (e.g., from the sensor) that indicates the second distance. Additionally or alternatively, determining the second distance can include determining the second distance based at least in part on the sensor data and / or the bottom surface of the target object and the sensor. In these and other embodiments, determining the height of the target object can include determining the height of the target object based at least in part on the first distance and / or the second distance. For example, determining the height of the target object can include determining the height of the target object as the difference between the first distance and the second distance.

[0090] In block 1075, method 1070 continues by calculating (e.g., deriving) or updating (e.g., adjusting, changing, recalculating, etc.) a second portion of the motion plan for moving the target object to the destination position. Calculating or updating the second portion of the motion plan can include calculating or updating the second portion of the motion plan based at least in part on the height of the target object determined in block 1074. In these and other embodiments, calculating or updating the second portion of the motion plan can include dynamically calculating or updating all or a subset of the second portion of the motion plan. In these and still other embodiments, calculating or updating the second portion of the motion plan includes calculating or updating the second portion of the motion plan before all or a first subset of the second portion of the motion plan is implemented and / or while all or a second subset of the second portion of the motion plan is being implemented.

[0091] As shown in sub-block 1091, calculating or updating the second portion of the motion plan can include calculating or updating the destination approach path and / or the corresponding destination approach velocity. Calculating or updating the destination approach path can include determining the release altitude of the target object. Determining the release altitude of the target object can include determining the release altitude based at least in part on one or more characteristics or traits of the target object. Calculating or updating the destination approach path and / or the corresponding destination approach velocity can include optimizing the destination approach path and / or the corresponding destination approach velocity to minimize or shorten the time it takes for the robotic system to place the target object at the destination position.

[0092] As shown in sub - block 1092, calculating or updating the second part of the motion plan may include calculating or updating the destination departure path and / or the corresponding destination departure speed. Calculating or updating the destination departure path may include determining the height and / or position at which the end - effector is raised after the robotic system places the object of interest at the destination position. Determining the height and / or position may include determining the height and / or position that avoids the horizontal line sensor and / or other components of the robotic system. Calculating or updating the destination departure path and / or the corresponding destination departure speed may include optimizing the destination departure path and / or the corresponding destination departure speed so as to minimize or shorten the time taken for the robotic system to move to the determined height and / or position of the end - effector after placing the object of interest at the destination position.

[0093] As shown in sub - block 1093, calculating or updating the second part of the motion plan may include calculating or updating the return path and / or the corresponding return speed. Calculating or updating the return path may include determining or updating the path by which the end - effector of the robotic system returns to the starting position (e.g., after raising the end - effector to the height and / or position specified by the destination departure path). Calculating or updating the return path and / or the corresponding return speed may include optimizing the return path and / or the corresponding return speed so as to minimize or shorten the time required for the robotic system to move the end - effector from the height / position specified by the destination departure path to the starting position.

[0094] Alternatively, calculating or updating the return path and / or the corresponding return speed may include determining a path for returning the end effector of the robotic system to the starting position after placing the object of interest at the destination position. Calculating or updating the return path may include determining a path that starts at the position of the end effector and ends at the starting position (e.g., the source position or near it) when the end effector disengages (e.g., drops) the object of interest at the destination position. For example, calculating or updating the return path may include calculating or updating a hybrid “shortcut” return path that represents a combination of the departure path from the destination and the return path. In such embodiments, sub-block 1092 may be omitted. As another example, calculating or updating the return path may include calculating or updating the return path such that the end effector is moved (e.g., immediately) (e.g., horizontally) toward the starting position after placing the object of interest at the destination position. In these and other embodiments, calculating or updating the return path may include calculating or updating the return path directly from the position where the end effector disengages the object of interest to the starting position. Additionally, or alternatively, calculating or updating the return path and / or the corresponding return speed may include optimizing the return path and / or the return speed to minimize or shorten the time it takes for the robotic system to move the end effector from the position at the time the end effector disengages the object of interest to the starting position.

[0095] As described above, the starting position can be (i) the default position and / or (ii) a position that positions the end effector to execute all or a subset of the next motion plan (or as part of the execution) to transfer, for example, the next object between the source position and the destination position. If the starting position is the default position (e.g., at the time when sub-block 1088 is executed), calculating or updating the return path may include determining or updating the path to return the end effector to the default position. Alternatively, calculating or updating the return path may include (i) updating the starting position from the default position to another position different from the default position (e.g., a position that facilitates executing all or a subset of the next motion plan), and / or (ii) determining or updating the path to move the end effector to position the end effector at another position. If the starting position is a position that positions the end effector to execute the next motion plan (or as part of the execution) (e.g., at the time when sub-block 1088 is executed), calculating or updating the return path may include determining or updating the path to move the end effector to position the end effector at the starting position (e.g., such that the return path links to one or more paths derived for the next motion plan).

[0096] In block 1076, method 1070 continues by implementing a second part of the motion plan for moving the target object to the destination position. Implementing the second part of the motion plan can include moving the target object towards the destination position according to the destination approach path and / or the destination approach speed calculated and / or updated in sub-block 1090. Implementing the second part of the motion plan can include lowering a part of the target object (such as the bottom surface) to the release height. As shown in sub-block 1093, implementing the second part of the motion plan can include placing the target object at the destination position, for example, by disengaging (such as dropping, releasing) the target object at the release height. Implementing the second part of the motion plan can include raising the end effector to the height and / or position specified by the destination departure path and / or according to the destination departure speed. Implementing the second part of the motion plan can include moving the end effector from the height and / or position specified by the destination departure path and / or according to the return path and / or the return speed to the starting position. Alternatively, implementing the second part of the motion plan can include moving the end effector to the starting position according to the hybrid "shortcut" return path and / or the associated return speed. For example, implementing the second part of the motion plan can include moving the end effector along the hybrid "shortcut" return path from the position of the end effector when it disengages the target object to the starting position. In an embodiment where the starting position is initially (for example, when sub-block 1088 is executed) the first position or the default position and then updated to a different position (for example, when sub-block 1092 is executed), implementing the second part of the motion plan can include moving the end effector to a different position, in contrast to the first / default position and along the return path / hybrid return path.In these and other embodiments, implementing the second portion of the motion plan may include moving the end effector to a starting position to facilitate or as part of implementing the next motion plan for the next object of interest.

[0097] The steps of method 1070 are described and illustrated in a particular order, but method 1070 of FIG. 10 is not so limited. In other embodiments, the steps of method 1070 may be performed in a different order. In these and other embodiments, any step of method 1070 may be performed before, between, and / or after any other step of method 1070. Further, one of ordinary skill in the art will recognize that the illustrated method 1070 may be modified and still remain within the scope of these and other embodiments of the present technology. For example, one or more of blocks 1071-1076 and / or one or more of sub-blocks 1081-1093 of method 1070 shown in FIG. 10 may be omitted and / or repeated in some embodiments.

[0098] Examples Some aspects of the present technology are illustrated in the following examples. Some aspects of the present technology are illustrated in examples directed particularly to methods, computer-readable media, and systems, but these aspects of the present technology may equally be illustrated in examples directed to any of systems, devices, methods, and computer-readable media in other embodiments. 1. A method for operating a robotic system, comprising: (i) receiving sensor data representing a distance between (i) a sensor of the robotic system and (ii) an object of interest engaged by an end effector of the robotic system; determining a height of the object of interest based at least in part on the sensor data; updating a motion plan for placing the object of interest at a destination location based at least in part on the height of the object of interest; and The method, wherein the updated motion plan includes commands, settings, or a combination thereof for operating the robotic arm and the end effector so that (i) it approaches the destination position and (ii) disengages the object to be placed at the destination position. 2. Determining the height of the object to be placed at least partially based on the sensor data includes: Determining a first distance between the position of the end effector and the sensor; Determining, at least partially based on the sensor data, a distance between the sensor and the object to be placed, wherein the distance between the sensor and the object to be placed is a second distance; Determining a difference between the first distance and the second distance; The method according to Example 1, comprising the above steps. 3. Determining the first distance includes determining or tracking the position of the end effector, the method according to Example 2. 4. Updating the motion plan includes determining a release height above the destination position where the end effector disengages the object to be placed, the method according to Examples 1 to 3. 5. Determining the release height includes determining the release height at least partially based on one or more characteristics of the object to be placed, the method according to Example 4. 6. The one or more characteristics include the weight of the object to be placed, the method according to Example 5. 7. Updating the motion plan includes determining a speed at which the robotic arm and the end effector move the object to be placed towards the destination position, the method according to Examples 1 to 6. 8. Further comprising deriving the motion plan, Deriving the motion plan includes, at least in part, pre-calculating a first command, a first setting, or a first combination thereof for operating the robotic arm and the end effector based on the maximum possible height value of the object of interest and / or the minimum possible height value of the object of interest. Updating the motion plan includes updating the first command, the first setting, or the first combination thereof to a second command, a second setting, or a second combination thereof, at least in part based on the height of the object of interest, according to the method described in Examples 1-7. 9. Updating the motion plan includes updating the motion plan before the robotic system executes the first command, the first setting, or the first combination thereof, according to the method described in Example 8. 10. Updating the motion plan includes updating the motion plan while the robotic system is executing the first command, the first setting, or the first combination thereof, according to the method described in Example 8. 11. The command, the setting, or the combination thereof is a first command, a first setting, or a first combination thereof. The updated motion plan further includes a second command, a second setting, or a second combination thereof for operating the robotic arm or the end effector such that the end effector directly returns from the position where it disengages the object of interest to place the object of interest at the destination position to the starting position, according to the method described in Examples 1-10. 12. Further including deriving the motion plan. Deriving the motion plan After disengaging the target object to place the target object at the transfer destination position, at least partially pre-calculate a third command, a third setting, or a third combination thereof for operating the robotic arm and the end effector to raise the end effector to a specified height, based on the maximum possible height value of the target object and / or the minimum possible height value of the target object. After raising the end effector to the specified height, pre-calculate a fourth command, a fourth setting, or a fourth combination thereof for operating the robotic arm and the end effector to return the end effector to the starting position. including Updating the motion plan includes, at least partially based on the height of the target object, updating the third command, the fourth command, the third setting, and / or the fourth setting to the second command, the second setting, or the second combination thereof, as described in Example 11. 13. The sensor data is first sensor data. The method further While the end effector approaches the transfer destination position according to the command, the setting, or the combination thereof, receive second sensor data representing a second distance between (i) the sensor and (ii) the target object. Determine the second distance, at least partially based on the second sensor data. including, as described in Examples 1 to 12. Example 14. Further including deriving the motion plan. The method according to any one of examples 1 to 13, wherein the motion plan includes a second command, a second setting, or a second combination thereof for operating the robotic arm and the end effector to position the object within the field of view of the sensor such that (i) the object is positioned above the sensor and (ii) the end effector is positioned on the side of the object opposite the sensor. 15. The method according to any one of examples 1 to 14, wherein the object is an unregistered object having an initially unknown height for the robotic system before determining the height of the object based at least in part on the sensor data. 16. A non-transitory computer-readable medium storing processor instructions that, when executed by one or more processors of a robotic system, cause the robotic system to execute a method, wherein the method comprises: instructions for determining the height of the object based at least in part on sensor data representing a distance between a sensor and the object engaged by an end effector of the robotic system; and instructions for updating a motion plan for placing the object at a destination location based at least in part on the height of the object, the updated motion plan including commands, settings, or combinations thereof for operating the robotic arm and the end effector to (i) approach the destination location and (ii) disengage the object to place the object at the destination location. A non-transitory computer-readable medium comprising executing the above. 17. A robotic sensor, comprising a robotic arm, an end effector attached to the robotic arm, a distance sensor having a vertically oriented field of view, wherein the robotic system comprises: ​Transferring the target object between a transfer source position and a transfer destination position using the robot arm and the end effector; Presenting the target object within the vertically oriented field of view of the distance sensor before placing the target object at the transfer destination position using the robot arm and the end effector; A robot system configured to perform the above. 18. The robot system according to Example 17, wherein the distance sensor is positioned at a position between the transfer source position and the transfer destination position. 19. The transfer destination position is positioned on the upper surface of a roller of a conveyor, The robot system according to Example 17, wherein the distance sensor is positioned under the transfer destination position and the roller of the conveyor. 20. The robot system according to Example 19, wherein at least a part of the vertically oriented field of view of the distance sensor is not blocked by the roller of the conveyor.

[0099] Conclusion The above detailed description of the embodiments of the present technology is not intended to be exhaustive or to limit the present technology to the precise forms disclosed above. Specific embodiments of the present technology, and specific examples for the present technology, have been described above for purposes of illustration, but as will be recognized by those skilled in the art, various equivalent modifications are possible within the scope of the present technology. For example, although the steps are presented in a given order above, in alternative embodiments, the steps can be executed in a different order. Further, the various embodiments described herein may also be combined to provide further embodiments.

[0100] From the foregoing, specific embodiments of the present technology have been described herein for purposes of illustration, but it will be understood that well-known structures and functions have not been shown or described in detail in order to avoid unnecessarily obscuring the description of the embodiments of the present technology. To the extent that the material incorporated herein by reference conflicts with the present disclosure, the present disclosure shall govern. Where the context permits, the singular or plural terms may also, respectively, include the plural or singular terms. In addition, unless the word "or" is explicitly limited to mean only a single item exclusive of the other items in a list of two or more items, the use of "or" in such a list shall be construed to include (a) any single item in the list, (b) all of the items in the list, or (c) any combination of the items in the list. Further, as used herein, the phrase "and / or" in "A and / or B" refers to only A, only B, and both A and B. In addition, the terms "comprising", "including", "having", and "with" are used throughout the present disclosure to mean including at least the recited feature(s) such that any greater number of the same features and / or additional types of other features are not excluded. Further, as used herein, the phrases "based on", "dependent on", "as a result of", and "in response to" shall not be construed as referring to a closed set of conditions. For example, an exemplary step described as "based on condition A" can be based on both condition A and condition B without departing from the scope of the present disclosure. In other words, as used herein, the phrase "based on" shall be construed in the same manner as the phrase "based at least in part on" or the phrase "based at least partially on". Also, the terms "connect" and "couple" are used interchangeably herein and refer to both direct and indirect connection or coupling.For example, where context permits, an element A that is "connected" or "coupled" to an element B can refer to (i) A being directly "connected" or directly "coupled" to B and / or (ii) A being indirectly "connected" or indirectly "coupled" to B.

[0101] From the foregoing, it will also be understood that various modifications may be made without departing from the disclosure or the technology. For example, those skilled in the art will understand that various components of the technology can be further divided into sub-components, or that various components and functions of the technology can be combined and integrated. Additionally, certain aspects of the technology described in the context of specific embodiments may also be combined or excluded in other embodiments. Further, the advantages associated with certain embodiments of the technology are described in the context of those embodiments, but other embodiments may also exhibit such advantages, and not all embodiments necessarily exhibit such advantages so as to fall within the scope of the technology. Accordingly, the disclosure and the related technology can encompass other embodiments not explicitly shown or described herein.

Claims

1. A method for operating a robot system, comprising: receiving sensor data representing a distance between (i) a sensor of the robot system and (ii) a target object engaged by an end effector of the robot system; determining a height of the target object based at least in part on the sensor data; updating a motion plan for placing the target object at a destination position based at least in part on the height of the target object; wherein the updated motion plan includes commands, settings, or combinations thereof for operating a robot arm and the end effector to (i) approach the destination position and (ii) disengage the target object to place the target object at the destination position.

2. Determining the height of the target object based at least in part on the sensor data includes: determining a first distance between a position of the end effector and the sensor; determining the distance between the sensor and the target object based at least in part on the sensor data, wherein the distance between the sensor and the target object is a second distance; and determining a difference between the first distance and the second distance. The method of claim 1.

3. Determining the first distance includes determining or tracking the position of the end effector. The method of claim 2.

4. Updating the motion plan includes determining a release height above the destination position at which the end effector disengages the target object. The method of claim 1.

5. Determining the release height includes determining the release height based at least in part on one or more characteristics of the target object. The method of claim 4.

6. The one or more characteristics include the weight of the target object. The method of claim 5.

7. Updating the motion plan includes determining a speed at which the robot arm and the end effector move the target object toward the destination position. The method of claim 1.

8. ​ further including deriving the motion plan, deriving the motion plan includes, at least in part, pre-calculating a first command, a first setting, or a first combination thereof for operating the robotic arm and the end effector based on a maximum possible height value of the object of interest and / or a minimum possible height value of the object of interest, updating the motion plan includes updating the first command, the first setting, or the first combination thereof to a second command, a second setting, or a second combination thereof, at least in part based on the height of the object of interest, according to the method of claim 1.

9. updating the motion plan includes updating the motion plan before the robotic system executes the first command, the first setting, or the first combination thereof, according to the method of claim 8.

10. updating the motion plan includes updating the motion plan while the robotic system is executing the first command, the first setting, or the first combination thereof, according to the method of claim 8.

11. the command, the setting, or the combination thereof is a first command, a first setting, or a first combination thereof, the updated motion plan further includes a second command, a second setting, or a second combination thereof for operating the robotic arm or the end effector such that the end effector directly returns from a position where it disengages the object of interest to place the object of interest at the destination position to a starting position, according to the method of claim 1.

12. further including deriving the motion plan, deriving the motion plan is After disengaging the target object to place the target object at the transfer destination position, at least partially pre-calculate a third command, a third setting, or a third combination thereof for operating the robot arm and the end effector to raise the end effector to a specified height based on the maximum possible height value of the target object and / or the minimum possible height value of the target object. After raising the end effector to the specified height, pre-calculate a fourth command, a fourth setting, or a fourth combination thereof for operating the robot arm and the end effector to return the end effector to the starting position. comprising Updating the motion plan includes updating the third command, the fourth command, the third setting, and / or the fourth setting to the second command, the second setting, or the second combination thereof, at least partially based on the height of the target object, according to the method of claim 11.

13. The sensor data is first sensor data. The method further While the end effector approaches the transfer destination position according to the command, the setting, or the combination thereof, receive second sensor data representing a second distance between (i) the sensor and (ii) the target object. Determine the second distance, at least partially based on the second sensor data. The method according to claim 1, comprising.

14. Further comprising deriving the motion plan, The motion plan includes a second command, a second setting, or a second combination thereof for operating the robot arm and the end effector to position the target object within the field of view of the sensor such that (i) the target object is positioned above the sensor and (ii) the end effector is positioned on the opposite side of the target object from the sensor, according to the method of claim 1.

15. The method according to claim 1, wherein the target object is an unregistered object having an initially unknown height for the robot system before determining the height of the target object based at least in part on the sensor data.

16. A non-transitory computer-readable medium storing processor instructions that, when executed by one or more processors of a robot system, cause the robot system to execute a method, The method includes: Instructions for determining the height of a target object based at least in part on sensor data representing a distance between a sensor and the target object engaged by an end effector of the robot system; Instructions for updating a motion plan for placing the target object at a destination location based at least in part on the height of the target object, the updated motion plan including commands, settings, or combinations thereof for operating a robot arm and the end effector to (i) approach the destination location and (ii) disengage the target object to place the target object at the destination location; A non-transitory computer-readable medium including executing the above.

17. A robot sensor, A robot arm, An end effector attached to the robot arm, A distance sensor having a vertically oriented field of view, Comprising, The robot system, Using the robot arm and the end effector to transfer a target object between a source location and a destination location; Using the robot arm and the end effector to present the target object within the vertically oriented field of view of the distance sensor before placing the target object at the destination location; A robot system configured to perform the above.

18. The robot system according to claim 17, wherein the distance sensor is positioned at a location between the source location and the destination location.

19. The destination location is positioned on an upper surface of a roller of a conveyor, The robot system according to claim 17, wherein the distance sensor is positioned under the destination location and the roller of the conveyor.

20. The robot system according to claim 19, wherein at least a part of the vertically oriented field of view of the distance sensor is not blocked by the roller of the conveyor.

Images