Robot system with multi-position placement control mechanism

Abstract

The disclosure relates to a robotic system with a multi-position placement control mechanism, including a system and method for operating the robotic system to place an object on a plurality of placement positions on a given surface. A robotic system may use a sequence of such placement locations to control placement of an object onto a placement location.

Description

[0001] This application is a divisional application of invention patent application 202380044046.0, filed on November 27, 2023, and entitled “Robotic system with multi-position placement control mechanism”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 428,110, filed on November 27, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0004] The present technology relates generally to robotic systems and, more particularly, to systems, processes, and techniques for managing placement of objects. Background Art

[0005] With increasing performance and decreasing costs, many robots (e.g., machines configured to automatically / autonomously perform physical actions) are now widely used in many fields. For example, robots can be used to perform various tasks (e.g., manipulating or transferring objects in space) in manufacturing and / or assembly, placement and / or packaging, transportation and / or delivery, etc. In performing tasks, robots are intended to replicate some aspects of human actions, thereby replacing or reducing the human involvement that would otherwise be required to perform the tasks.

[0006] However, despite technological advances, robots often lack the sophistication necessary to replicate the human sensitivity and / or adaptability required to perform more complex and delicate tasks. For example, robots often lack the granularity of control and flexibility in performing actions to fully utilize available resources. Furthermore, the human behaviors and experiences required to successfully perform tasks under a variety of real-world conditions are often difficult to capture or translate to robotic implementation. Therefore, there remains a need for improved techniques and systems to control and manage various aspects of robots to complete tasks regardless of various real-world factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is an illustration of an exemplary environment in which a robotic system having a multi-position placement control mechanism may operate.

[0008] Figure 2 is a block diagram illustrating a robotic system according to one or more embodiments of the present technology.

[0009] Figure 3A is an illustration of a robotic system in accordance with one or more embodiments of the present technology.

[0010] Figure 3Bis an illustration of an exemplary task station for a robotic system in accordance with one or more embodiments of the present technology.

[0011] FIG. 4A to FIG. 4B is an illustration of an exemplary segmentation of placement locations in accordance with one or more embodiments of the present technology.

[0012] FIG. 5A to FIG. 5F is an illustration of an implementation of a placement rule in accordance with one or more embodiments of the present technology.

[0013] FIG. 6A to FIG. 6B is an illustration of an exemplary placement sequence in accordance with one or more embodiments of the present technology.

[0014] Figure 7 is an operation according to one or more embodiments of the present technology Figure 1 Flowchart of a first exemplary method of a robotic system. DETAILED DESCRIPTION

[0015] Systems and methods for a robotic system with a multi-position placement control mechanism are described herein. A robotic system (e.g., an integrated system of devices that perform one or more specified tasks) can be configured to provide enhanced control, usability, and flexibility by placing objects (e.g., packages, boxes, crates, etc.) on placement platforms. For example, the robotic system can stack one or more objects at each of the placement platforms (such as pallets) while complying with the stacking height limit 530 assigned to each placement platform.

[0016] The robotic system can place and stack objects at the placement platform based on identifying different placement areas within the task location. As an illustrative example, the robotic system can control a stacking robot at the task location to place objects on one or more pallets (e.g., placement platforms) located at the task location. Each pallet can have multiple different placement areas, such as for forming multiple separate stacks of objects on one or each of the pallets.

[0017] Different placement areas (e.g., stacking positions) can be calculated based on fixed or matching basic dimensions (e.g., length and width) of the object. In other embodiments, each of the different placement areas can correspond to a unique footprint of one or more objects (e.g., a unique shape, a set of unique lateral dimensions, and / or a corresponding posture of the object). In other words, each different placement area can be calculated or designated for placing or stacking one or more types of objects with matching or otherwise related footprints. In some embodiments, the robotic system can generate a separate stack height limit for each different placement area. In addition, the robotic system can dynamically adjust the stack height limit 530 of the different placement areas.

[0018] When the object is transferred, the robot system can dynamically track the implementation of the motion plan to track and identify the objects (e.g., objects in each stack or the topmost object) that have been placed, the stack height and / or the characteristic properties (e.g., size) of the size of the objects at each of the different placement areas. In some embodiments, the robot system can use the known size of the objects previously placed at each of the different placement areas and the orientation tracked to calculate / estimate the current stack height. In other embodiments, the robot system can use sensor data to determine the stack height at the different placement areas in real time. The robot system can use the current stack height to calculate the candidate stack height, which is generated according to the posture placement target object at the corresponding different placement areas. Using the candidate stack height, the robot system can determine whether the stack height limit 530 and the corresponding rules are followed when the target object is placed in the candidate placement area. Based on the determined results, the robot system can derive the best placement area for the target object, thereby improving stacking efficiency and reducing the possibility of collision between multiple placement areas.

[0019] To simplify the stacking process, the robotic system can sort the different placement areas. In some embodiments, the robotic system can sort the different placement areas according to the distance between the different placement areas and a manipulation robot (e.g., a transfer unit, such as a robot with an arm) configured to place the object in the different placement areas. For example, the robotic system can assign 1 as a sequence identifier to the farthest different placement area and assign a higher number as a sequence identifier 450 to closer different placement areas (e.g., 3 is assigned to the closest area among the three placement areas).

[0020] The robotic system can use this sequence to generate a simplified rule that can guide the object stacking operation. Continuing with the exemplary sequence above, the robotic system can include a stacking rule to require that the stacking height at a lower-ranked placement area is greater than the stacking height at a higher-ranked placement area. This rule can be expressed as: i ≤h j , where i>j. Parameters i and j can represent instances of different placement regions, and h i and h j The current and / or candidate stacking heights at the corresponding different placement areas may be indicated. Detailed information on the sorting and stacking rules is described below.

[0021] By sorting the placement areas, such as according to the lateral separation distance from the operating robot, the robotic system can simplify the stacking rules. For example, the stacking rules can eliminate multiple situation-based comparisons and the corresponding lines of code, memory, and execution time. Based on the simplification, the robotic system can dynamically consider and respond to real-world conditions to form more relevant and efficient object stacking. For example, using simplified rules, the robotic system can group objects with similar footprints together, quickly consider alternative stacking postures for target objects, respond to dynamic / unplanned arrival of objects, etc. In order to make a comparison, a traditional system may need to stack all objects in one location until the resulting stack reaches a limit and then move to another placement area. This approach may produce undesirable stack formation (e.g., the lower stacked object has a smaller lateral size than the upper object), cannot respond to dynamic conditions (e.g., unplanned arrival / sequence of objects), and reduce the overall placement efficiency of the target location (e.g., within a tray).

[0022] The robotic system can be configured to dynamically adjust the motion plan, placement area and / or stacking height limit, such as to deal with unexpected situations (e.g., package anomalies). For example, the placement object may include a top surface that may be deformed, bent, misaligned, partially closed and / or physically different from the expected conditions. Such unexpected conditions may affect the ability to stack additional objects at the placement area and / or the stacking height limit of other placement areas. The robotic system can detect such unexpected situations and dynamically adjust the assigned placement area and / or stacking height limit. In addition, based on dynamic adjustments, the robotic system can update the motion plan to deal with unexpected situations.

[0023] In the following description, numerous specific details are set forth to provide a thorough understanding of the current disclosed technology. In other embodiments, the technology presented herein may be practiced without these specific details. In other examples, well-known features such as specific functions or routines are not described in detail in order to avoid unnecessarily obscuring the present disclosure. References to "embodiment", "one embodiment" and the like in this description mean that the specific features, structures, materials or characteristics described are included in at least one embodiment of the present disclosure. Therefore, the appearance of such phrases in this specification may not all refer to the same embodiment. On the other hand, such references may not necessarily be mutually exclusive. In addition, specific features, structures, materials or characteristics may be combined in one or more embodiments in any suitable manner. It should be understood that the various embodiments shown in the drawings are merely illustrative representations and are not necessarily drawn to scale.

[0024] For the sake of clarity, several details describing structures or processes that are well known and commonly associated with robotic systems and subsystems but that may unnecessarily obscure some important aspects of the disclosed technology are not set forth in the following description. In addition, although the following disclosure sets forth several embodiments of different aspects of the present technology, several other embodiments may have configurations or components that are different from those described in this section. Therefore, the disclosed technology may have other embodiments with additional elements or without several elements described below.

[0025] Many embodiments or aspects of the present disclosure described below may take the form of computer or processor executable instructions, including routines executed by a programmable computer or processor. It will be appreciated by those skilled in the relevant art that the disclosed technology may be practiced on a computer or processor system other than the computer or processor system shown and described below. The technology described herein may be embodied in a special-purpose computer or data processor, which is specifically programmed, configured or constructed to execute one or more computer executable instructions described below. Therefore, the terms "computer" and "processor" as commonly used herein refer to any data processor, and may include Internet appliances and handheld devices (including palmtop computers, wearable computers, cellular or mobile phones, multiprocessor systems, processor-based or programmable consumer electronics, network computers, minicomputers, etc.). The information handled by these computers and processors may be presented on any suitable display medium, including a liquid crystal display (LCD). Instructions for performing tasks executable by a computer or processor may be stored in or on any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. Instructions may be contained in any suitable memory device, including, for example, a flash drive and / or other suitable media.

[0026] The terms "coupled" and "connected" and their derivatives may be used herein to describe the structural relationship between components. It should be understood that these terms are not intended to be synonymous with each other. Instead, in a particular embodiment, "connected" can be used to indicate that two or more elements are in direct contact with each other. Unless otherwise apparent in the context, the term "coupled" can be used to indicate that two or more elements are in direct or indirect contact with each other (with other intermediate elements between them), or that two or more elements cooperate or interact with each other (e.g., interact with each other in a causal relationship, such as for signal transmission / reception or for function calls), or both.

[0027] Applicable environment

[0028] Figure 1is an illustration of an exemplary environment in which a robotic system 100 having a multi-position placement control mechanism may operate. The robotic system 100 may include and / or communicate with one or more units (e.g., robots) configured to perform one or more tasks. Aspects of the multi-position placement control mechanism may be practiced or implemented by the various units.

[0029] for Figure 1 In the example shown, the robot system 100 may include an unloading unit 102, a transfer unit 104 (e.g., a stacking robot and / or a picking robot), a transport unit 106, a loading unit 108, or a combination thereof in a warehouse or a distribution / delivery center. Each of the units in the robot system 100 may be configured to perform one or more tasks. The tasks may be combined in sequence to perform operations to achieve a goal, such as unloading an object from a truck or a van and storing the object in a warehouse, or unloading an object from a storage location and preparing to transport the object. For another example, a task may include placing an object at a target location (e.g., on top of a pallet and / or inside a bin / cage / box / box). As described below, the robot system may derive a plan for placing and / or stacking objects (e.g., a placement area / orientation, a sequence for transferring objects, and / or a corresponding motion plan). Each of the units may be configured to perform a series of actions (e.g., by operating one or more components therein) to perform a task according to one or more of the derived plans.

[0030] In some embodiments, the task may include the manipulation (e.g., movement and / or reorientation) of a target object 112 (e.g., one of a package, box, case, cage, pallet, etc. corresponding to the task being performed) from a starting location 114 to a task location 116. For example, the unloading unit 102 (e.g., a box unpacking robot) may be configured to transfer the target object 112 from a location in a carrier (e.g., a truck) to a location on a conveyor. Additionally, the transfer unit 104 may be configured to transfer the target object 112 from one location (e.g., a conveyor, a pallet, or a bin) to another location (e.g., a pallet, a bin, etc.). For another example, the transfer unit 104 (e.g., a stacking robot) may be configured to transfer the target object 112 from a source location (e.g., a pallet, a pickup area, and / or a conveyor) to a destination pallet. Upon completion of the operation, the transport unit 106 may transfer the target object 112 from an area associated with the transfer unit 104 to an area associated with the loading unit 108, and the loading unit 108 may transfer the target object 112 from the transfer unit 104 to a storage location (e.g., a location on a shelf) (e.g., by moving a pallet carrying the target object 112). Details regarding the tasks and associated actions are described below.

[0031] For illustrative purposes, the robotic system 100 is described in the context of a shipping center; however, it should be understood that the robotic system 100 can be configured to perform tasks in other environments / for other purposes (such as for manufacturing, assembly, packaging, healthcare, and / or other types of automation). It should also be understood that the robotic system 100 may include Figure 1 Other units not shown in the figure, such as manipulators, service robots, modular robots, etc. For example, in some embodiments, the robot system 100 may include: a depalletizing unit for transferring objects from a cage or a pallet to a conveyor or other pallets; a pallet switching unit for transferring objects from one pallet to another pallet; a packaging unit for packaging objects; a sorting unit for grouping objects according to one or more characteristics of the objects; a picking unit for manipulating (e.g., sorting, grouping and / or transferring) objects in different ways according to one or more characteristics of the objects; or a combination thereof.

[0032] The robotic system 100 may include and / or be coupled to physical or structural members (e.g., a robotic manipulator arm) connected at joints for motion (e.g., rotational and / or translational displacement). The structural members and joints may form a power chain configured to manipulate an end effector (e.g., a gripper) configured to perform one or more tasks (e.g., grasping, spinning, welding, etc.) according to the use / operation of the robotic system 100. The robotic system 100 may include an actuation device (e.g., a motor, an actuator, a wire, an artificial muscle, an electroactive polymer, etc.) configured to drive or manipulate (e.g., displace and / or reorient) the structural member around or at a corresponding joint. In some embodiments, the robotic system 100 may include a transport motor configured to transport the corresponding unit / chassis from one place to another.

[0033] The robotic system 100 may include sensors configured to obtain information for performing tasks, such as information for manipulating structural members and / or for transporting the robotic unit 440. The sensors may include devices configured to detect or measure one or more physical properties of the robotic system 100 (e.g., the state, condition, and / or position of one or more structural members / joints thereof) and / or one or more physical properties of the surrounding environment. Some examples of sensors may include accelerometers, gyroscopes, force sensors, strain gauges, tactile sensors, torque sensors, position encoders, etc.

[0034] For example, in some embodiments, the sensor may include one or more imaging devices (e.g., visual and / or infrared cameras, 2D and / or 3D imaging cameras, distance measuring devices such as lidar or radar, etc.) configured to detect the surrounding environment. The imaging device can generate a representation of the detected environment, such as a digital image and / or a point cloud, which can be processed via machine / computer vision (e.g., for automated inspection, robot guidance, or other robotic applications). As described in further detail below, the robotic system 100 can process the digital image and / or point cloud to identify the target object 112, the starting position 114, the task position 116, the pose of the target object 112, a confidence measure about the starting position 114 and / or the pose, or a combination thereof.

[0035] To manipulate the target object 112, the robotic system 100 may capture and analyze images of a designated area (e.g., such as a pickup location inside a truck or on a conveyor) to identify the target object 112 and its starting location 114. Similarly, the robotic system 100 may capture and analyze images of another designated area (e.g., a drop location for placing an object on a conveyor, a location for placing an object on a pallet, or a location on a pallet for stacking purposes) to identify a task location 116. For example, the imaging device may include one or more cameras configured to generate images of the pickup area and / or one or more cameras configured to generate images of a task area (e.g., a drop area). Based on the captured images, the robotic system 100 may determine the starting location 114, the task location 116, the associated poses, the placement plan, the transfer sequence, and / or other processing results, as described below.

[0036] In some embodiments, for example, the sensor may include a position sensor (e.g., a position encoder, a potentiometer, etc.) configured to detect the position of a structural member (e.g., a robotic arm and / or an end effector) and / or a corresponding joint of the robotic system 100. The robotic system 100 may use the position sensor to track the position and / or orientation of the structural member and / or the joint during task performance.

[0037] Applicable systems

[0038] Figure 2is a block diagram illustrating a robot system 100 according to one or more embodiments of the present technology. In some embodiments, for example, the robot system 100 (e.g., at one or more of the above-mentioned units and / or robots) may include electronic / electrical devices, such as one or more processors 202, one or more storage devices 204, one or more communication devices 206, one or more input-output devices 208, one or more actuators 212, one or more transport motors 214, one or more sensors 216, or a combination thereof. The various devices may be coupled to each other via wired connections and / or wireless connections. For example, the robot system 100 may include a bus, such as a system bus, a peripheral component interconnect (PCI) bus or a PCI-Express bus, a HyperTransport or an Industry Standard Architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), an IIC (I2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (also known as "FireWire"). In addition, for example, the robot system 100 may include a bridge, an adapter, a processor, or other signal-related devices for providing a wired connection between devices. The wireless connection can be based on, for example, cellular communication protocols (e.g., 3G, 4G, LTE, 5G, etc.), wireless local area network (LAN) protocols (e.g., Wireless Fidelity (WIFI)), peer-to-peer or device-to-device communication protocols (e.g., Bluetooth, Near Field Communication (NFC), etc.), Internet of Things (IoT) protocols (e.g., NB-IoT, LTE-M, etc.), and / or other wireless communication protocols.

[0039] Processor 202 may include a data processor (e.g., a central processing unit (CPU), a dedicated computer, and / or an onboard server) configured to execute instructions (e.g., software instructions) stored on storage device 204 (e.g., computer memory). In some embodiments, processor 202 may be included in a separate / independent controller that is operably coupled to Figure 2 Other electronic / electrical devices shown and / or Figure 1 The robotic unit 440 is shown. The processor 202 may implement program instructions to control / interface with other devices, thereby causing the robotic system 100 to perform actions, tasks, and / or operations.

[0040] Storage device 204 may include a non-transitory computer-readable medium having program instructions (e.g., software) stored thereon. Some examples of storage device 204 may include volatile memory (e.g., cache and / or random access memory (RAM)) and / or non-volatile memory (e.g., flash memory and / or a disk drive). Other examples of storage device 204 may include a portable memory drive and / or a cloud storage device.

[0041] In some embodiments, the storage device 204 can be used to further store processing results and / or predetermined data / thresholds and provide access thereto. For example, the storage device 204 can store master data 252, which includes descriptions of objects (e.g., boxes, boxes, and / or products) that can be manipulated by the robotic system 100. In one or more embodiments, the master data 252 can include registration data 254 for each such object. The registration data 254 can include the size, shape (e.g., templates for potential poses and / or computer-generated models for identifying objects in different poses), color schemes, images, identification information (e.g., bar codes, quick response (QR) codes, logos, etc., and / or their expected locations), expected weights, other physical / visual characteristics, or combinations thereof of the objects expected to be manipulated by the robotic system 100. In some embodiments, the master data 252 can include manipulation-related information about the objects, such as the center of mass (CoM) position of each of the objects or an estimate thereof, expected sensor measurements corresponding to one or more actions / manipulations (e.g., for force, torque, pressure, and / or contact measurements), or combinations thereof.

[0042] The communication device 206 may include circuits configured to communicate with external or remote devices over a network. For example, the communication device 206 may include a receiver, a transmitter, a modulator / demodulator (modem), a signal detector, a signal encoder / decoder, a connector port, a network card, etc. The communication device 206 may be configured to send, receive, and / or process electrical signals according to one or more communication protocols (e.g., Internet Protocol (IP), wireless communication protocols, etc.). In some embodiments, the robot system 100 may use the communication device 206 to exchange information between units of the robot system 100 and / or to exchange information with systems or devices external to the robot system 100 (e.g., for reporting, data collection, analysis, and / or troubleshooting purposes).

[0043] The input-output device 208 may include a user interface device configured to convey information to an operator and / or receive information from an operator. For example, the input-output device 208 may include a display 210 and / or other output devices (e.g., speakers, tactile circuits or tactile feedback devices, etc.) for conveying information to an operator. In addition, the input-output device 208 may include a control or receiving device such as a keyboard, a mouse, a touch screen, a microphone, a user interface (UI) sensor (e.g., a camera for receiving motion commands), a wearable input device, etc. In some embodiments, the robotic system 100 may use the input-output device 208 to interact with an operator when performing an action, a task, an operation, or a combination thereof.

[0044] The robotic system 100 may include physical or structural members (e.g., a robotic manipulator arm) connected at joints for motion (e.g., rotational and / or translational displacement). The structural members and joints may form a power chain that is configured to manipulate an end effector (e.g., a gripper) configured to perform one or more tasks (e.g., grasping, spinning, welding, etc.) according to the use / operation of the robotic system 100. The robotic system 100 may include an actuation device 212 (e.g., a motor, an actuator, a wire, an artificial muscle, an electroactive polymer, etc.) that is configured to drive or manipulate (e.g., displace and / or redirect) the structural member around or at the corresponding joint. In some embodiments, the robotic system 100 may include a transport motor 214 that is configured to transport the corresponding unit / chassis from one place to another.

[0045] The robotic system 100 may include sensors 216 configured to obtain information for performing tasks, such as information for manipulating structural members and / or for transporting the robotic unit 440. The sensors 216 may include devices configured to detect or measure one or more physical properties of the robotic system 100 (e.g., the state, condition, and / or position of one or more structural members / joints thereof) and / or one or more physical properties of the surrounding environment. Some examples of the sensors 216 may include accelerometers, gyroscopes, force sensors, strain gauges, tactile sensors, torque sensors, position encoders, and the like.

[0046] For example, in some embodiments, the sensors 216 may include one or more imaging devices 222 (e.g., visual and / or infrared cameras, 2D and / or 3D imaging cameras, distance measuring devices such as lidar or radar, etc.) configured to detect the surrounding environment. The imaging devices 222 may generate a representation of the detected environment, such as a digital image and / or a point cloud, which may be processed via machine / computer vision (e.g., for automated inspection, robot guidance, or other robotic applications). As described in further detail below, the robotic system 100 may process (e.g., via the processor 202) the digital image and / or point cloud to identify Figure 1 Target object 112, Figure 1 The starting position 114, Figure 1 The task position 116 , the pose of the target object 112 , a confidence measure regarding the starting position 114 and / or the pose, or a combination thereof.

[0047] To manipulate the target object 112, the robotic system 100 may capture and analyze (e.g., via the various circuits / devices described above) an image of a designated area (e.g., such as a pickup location inside a truck or on a conveyor) to identify the target object 112 and its starting location 114. Similarly, the robotic system 100 may capture and analyze an image of another designated area (e.g., a drop location for placing an object on a conveyor, a location for placing an object on a pallet, or a location on a pallet for stacking purposes) to identify a task location 116. For example, the imaging device 222 may include one or more cameras configured to generate an image of a pickup area and / or one or more cameras configured to generate an image of a task area (e.g., a drop area). Based on the captured images, the robotic system 100 may determine the starting location 114, the task location 116, the associated pose, the placement area, and / or other processing results, as described below. Details regarding the dynamic placement algorithm are described below.

[0048] In some embodiments, for example, the sensors 216 may include position sensors 224 (e.g., position encoders, potentiometers, etc.) configured to detect the position of a structural member (e.g., a robotic arm and / or end effector) and / or a corresponding joint of the robotic system 100. The robotic system 100 may use the position sensors 224 to track the position and / or orientation of the structural member and / or joint during task performance.

[0049] Example Object Transfer and Placement

[0050] Figure 3A According to one or more embodiments of the present technology Figure 1 . The robotic system 100 may include or be communicatively coupled to a robotic arm 302 that includes an end effector 304 (e.g., a gripper). The robotic arm 302 may be Figure 1 The robot unit 440 (e.g., Figure 1 The robot arm 302 may be an industrial robot arm or a portion of a transfer unit 104 (e.g., an instance of a transfer unit 104 of the embodiment of the present invention). For example, the robot arm 302 may include an industrial robot system employed in industrial applications including package handling applications. The robot arm 302 may be articulated along or around multiple axes, such as for a six-axis industrial robot arm structure.

[0051] The robotic arm 302 may be configured to Figure 1 The starting position 114 and Figure 1 The target object 112 is transferred between the task locations 116. Figure 3A As shown, the starting position 114 may correspond to the conveyor 306 (e.g., Figure 1The task location 116 of the robotic arm 302 can be a placement platform 308 (e.g., a pallet) or a location therein. For example, the robotic arm 302 can be configured to pick up objects 112 from the conveyor 306 and place them in or on the placement platform 308 for transport to another destination / task.

[0052] The end effector 304 may include any one or more components coupled to the distal end of the robot arm 302. The end effector 304 may be configured to interact with one or more objects. In some embodiments, the end effector 304 may include a force-torque (FT) sensor (not shown), an arm interface, a gripper system, and / or a gripper interface. For illustrative purposes, the end effector 304 is shown as having multiple rows of suction cups, however it should be understood that the end effector 304 may have a different configuration. For example, the end effector 304 may have a suction pad with an integrated suction channel, a clamp-type gripping device, or any other type of gripping system for gripping an object.

[0053] The robotic system 100 may be used when performing a transfer operation using the robotic arm 302. Figure 2 The robot system 100 may include or be coupled to a set of sensors (e.g., 2D and / or 3D sensors, such as cameras and / or depth sensors) at or near the starting position 114 and / or the task position 116. In some embodiments, the robot system 100 may include or be coupled to a top view sensor 310 above the task position 116 and pointing to the task position and / or a side view sensor adjacent to the task position 116 and pointing laterally to the task position. The robot system 100 may similarly include one or more source sensors 314 pointing to the starting position 114. The sensor may be configured to image and / or analyze the corresponding position. For example, the top view sensor 310 may generate and / or process image data depicting a top view of the placement platform 308 and / or an object thereon. In addition, the side view sensor may generate and / or process image data depicting a side view of the placement platform 308 and / or an object thereon.

[0054] The robotic system 100 can use the image data from the sensor 216 to perform a task, such as for transferring an object from the starting location 114 to the task location 116. Thus, the robotic system 100 can use the image data to derive and implement one or more placement plans (e.g., a description of the number / identity of objects and / or their poses and physical arrangements on one or more platforms) and / or motion plans to perform the task (e.g., a description of physically moving the robotic arm to transfer the corresponding objects). As described in further detail below, the robotic system 100 can derive and / or dynamically adjust placement plans to place objects in multiple stacks on the placement platform 308. The plan can correspond to one or more objects being placed on top of other objects (e.g., stacked).

[0055] Figure 3B According to one or more embodiments of the present technology, Figure 1 350 of the robotic system 100. The example task station 350 can be a location configured for the robotic arm 302 to perform a task / operation. The example task station 350 can have multiple sources 352, such as separate conveyors, which are configured to separately and / or independently provide target objects or packages. The task station 350 can receive the target object via the source 352, and the robotic arm 302 can transfer the target object to one or more destinations 354 (e.g., pallets).

[0056] Each destination 354 can have a plurality of placement locations 356 (e.g., P1-P4, P1-P5, etc.) for the target object. As described in further detail below, the placement locations 356 can be sorted or arranged in order, such as according to the spacing distance between the corresponding location and the robotic arm 302. Figure 3B In the example shown, the furthest placement location on pallet D1 may be assigned the lowest identifier P1 and the closest placement location may be assigned the highest identifier P4. The robotic system 100 may use the sequence and corresponding placement rules to select an appropriate placement / stacking location for an incoming object.

[0057] In some embodiments, when the task station 350 includes multiple placement surfaces, the robotic system 100 can assign and utilize a preferred approach direction 360 to each placement surface. Figure 3B In the example shown, the preferred approach direction 360 of the tray D1 may effectively correspond to a clockwise direction, and the preferred approach direction 360 of the tray D2 may correspond to a counterclockwise direction. During motion planning, the robotic system 100 may provide a higher weight to follow the preferred approach direction 360 when placing an object on the corresponding placement surface. As a result, the robotic system 100 may reduce the likelihood of collision with an object stacked on another placement surface.

[0058] The robotic system 100 may also use the preferred approach direction 360 to sort the placement locations 356 within each of the placement surfaces. For example, when two or more placement locations within a given placement surface are spaced apart by distances within a threshold range from one another, the robotic system 100 may use the preferred approach direction 360 to sort such placement locations. The robotic system 100 may assign lower sequence identifiers 450 to placement locations that are farther apart along the preferred approach direction 360. Figure 3B In the example shown, the robotic system 100 can determine that placement locations P2 and P3 on pallet D1 are within a threshold range. Assuming the clockwise direction is the preferred approach direction 360, the robotic system 100 can assign a lower identifier P2 to locations farther along the preferred approach direction 360. Similarly, the robotic system 100 can sort pallet D2 into P2-P4.

[0059] Using the sequenced placement locations 356, the robotic system 100 can control object placement, such as deriving a packing plan (e.g., the location of an object on a placement surface and within a stack), selecting a target object to be operated on, selecting a placement location for a target object, deriving a motion plan, implementing / executing a motion plan, or a combination thereof. For example, when (1) the arrival time and availability of the target at the source 352 are controllable and (2) the target content is available for each platform, the robotic system 100 can use the sequenced placement locations 356 and placement rules to derive a packing plan. Additionally, for example, when (1) the arrival time is not controllable and (2) the target content is available, the robotic system 100 can use the sequenced placement locations 356 and placement rules to efficiently derive the placement locations and derive a packing plan in real time as the objects are received at the source 352.

[0060] Additionally, when the task station 350 includes multiple sources 352, the robotic system 100 can use the sequential placement positions 356 and placement rules to select which available object to transfer first. Figure 3B In the illustrated example, the robotic system 100 can compare the sorted placement locations of the target objects T1-T3 available at the source 352 (e.g., such as when a placement plan is available or after dynamically deriving the placement locations). Based on the comparison, the robotic system 100 can select one of the target objects that has a placement location farthest from the robotic arm 302 and / or complies with other management rules. Similarly, when the available target object can be placed at two or more placement locations, the robotic system 100 can select one of the placement locations that is farthest from the robotic arm 302.

[0061] Example placement area segmentation

[0062] FIG. 4A to FIG. 4Bis an illustration of an exemplary segmentation of placement locations 450 in accordance with one or more embodiments of the present technology. Figure 4A An exemplary robotic unit 440 (eg, a palletizing robot) is shown operating the robotic arm 302 to place the target package 410 on the base surface 408 of the placement platform 308. Figure 4A As shown, substrate surface 408 may be divided into individual placement locations 430 . Figure 4A Also shown is an exemplary anomalous package 420 having an anomalous object property 402 (eg, a deformed top surface) that interferes with the robotic system 100 (eg, preventing additional packages from being stacked on top). Figure 4A An electronic display 401 is shown depicting a robotic unit 440 , a robotic arm 302 , a placement platform 308 , a target package 410 , and an exception package 420 . Figure 4A Also shown is a pop-up display identifying an anomalous object property 402 (eg, one or more physical properties of the detected object that deviate from a known corresponding metric) of an anomalous package 420 . Figure 4B is a side view of an exemplary placement assembly 400 .

[0063] In some embodiments, Figure 1 The robotic system 100 can be configured to form multiple stacks on one or more placement platforms 308 using the robotic units 440. For example, the robotic system 100 can use one palletizing robot (e.g., one station) to form multiple stacks of one or more pallets. Such multi-stack configurations and implementations can be used to complete multiple tasks and / or complex tasks, such as for completing multiple orders (e.g., each pallet represents an order) and / or for placing a mixed set of stock keeping units (SKUs) or item types on one or more of the platforms.

[0064] When completing multiple and / or complex tasks, the robot system 100 can form multiple separate stacks on one or more of the one or more placement platforms 308. Therefore, the robot system 100 can divide the substrate surface 408 of the corresponding placement platform 308 into multiple placement areas. Each placement area can represent a site or position where the corresponding object is stacked. In some embodiments, the robot system 100 can dynamically divide the substrate surface 408 and calculate the placement area in real time according to the object that enters or can be touched. In other embodiments, the robot system 100 can use a packaging planner (e.g., a group of dedicated processors, engines, software or a combination thereof) to divide the substrate surface 408 according to one or more predetermined goals (e.g., packaging density, pallet height restrictions, shipping lists, relevant thresholds or rules or a combination thereof). The robot unit 440 can receive the incoming objects from one or more sources (e.g., conveyors) and transfer each of the target packages to one of the placement areas, thereby forming multiple stacked objects on one or more of the pallets.

[0065] When forming and managing multiple stacks, the robotic system 100 can use one or more sensor data to detect or identify objects. For example, the robotic system 100 can Figure 3A The source sensor 314 or Figure 2 Other similar sensors 216 receive the depiction Figure 1 The robotic system 100 may analyze the image data (e.g., a 2D image, a 3D depth map, or a combination thereof) of the starting location 114 (e.g., one or more conveyors 306 that provide objects to the station). The robotic system 100 may analyze the image data in real time to determine one or more physical characteristics of the received target package 410, such as size (e.g., length, width, or height), shape, contour, texture (e.g., images, text, or other visual characteristics of the object surface), object pose, and / or other physical properties of the target package 410 and the placement platform 308. In some embodiments, for example, the robotic system 100 may use data from Figure 2 The robot system 100 may use a depth metric (e.g., point cloud data) of one or more imaging devices 222 of the robot system 100. Since the vertical position of the ground and / or the placement platform 308 is known (e.g., the height of the top surface of the placement platform), the robot system 100 may use the depth metric to calculate the height / profile of the platform 308 and the exposed top surface of the target package 410. Additionally or alternatively, the robot system 100 may compare one or more physical characteristics (e.g., sensor data, such as object texture or size) to the master data to detect or identify the received object. Furthermore, the robot system 100 may detect the object by determining the location of the object, the physical edges / boundaries of the object, or a combination thereof.

[0066] When generating or adjusting the packing plan for each platform, the robot system 100 can use the lateral size (e.g., length, width) of the expected or available package to divide the substrate surface of the platform and calculate the placement position 430. Given the expected or desired packaging target (via, for example, upstream sensor information, shipping list, etc.), the robot system 100 can determine the stacking posture and the corresponding substrate area (e.g., occupied space). Using the substrate area of ​​the packaging target, the robot system 100 can determine one or more placement positions 430 on the placement platform 308 by dividing the substrate surface of the placement platform 308 into one or more peripheries associated with the substrate area of ​​the packaging target. The placement position 430 can represent the position where one or more package objects can be placed and / or stacked on the placement platform 308. The robot system 100 can also assign a serial identifier 450 to each placement position. In some embodiments, the length 412 and / or width 414 of the packaging target 410 can be fixed to a specific distance. Therefore, the robot system 100 can divide the substrate area of ​​the placement platform into the periphery of equal size corresponding to each placement position 430. The robotic system 100 may determine the placement location 430 by a three-dimensional coordinate center, a set of known distances (eg, a number of pixels from the edge of the placement platform 308 ), a set of surrounding vectors (eg, coordinate pairs), or some combination thereof.

[0067] In some embodiments, the robotic system 100 may determine one or more sets of placement locations 430 for different height levels above the placement platform 308. The robotic system 100 may be configured to determine a placement location 430 that does not overlap the perimeter of another placement location 430 at the same lateral height level.

[0068] The robotic system 100 can be configured to determine a lateral distance between the placement location 430 and the robotic unit 440. In some embodiments, the robotic system 100 can calculate a distance from a reference point (e.g., a center portion) of the placement location 430 to a reference point of the robotic unit 440 that is lateral (e.g., an xy plane) relative to the center point of the placement location 430. The robotic system 100 can also be configured to arrange or assign sequence identifiers 450 to the placement locations 430 according to a sequence that is ordered based on the calculated distances, such as to assign the lowest identifier to the farthest placement location and higher identifiers to closer placement locations.

[0069] The robotic system 100 can be configured to identify abnormal object properties 402 associated with an incoming or accessible object in real time. For example, the robotic system 100 can be configured to identify deviations in one or more physical properties (e.g., deformation, bending, misalignment, and / or partial closure, etc.) of an incoming object, thereby preventing additional objects from being stacked on top of the incoming object. The robotic system 100 can detect the incoming object using one or more portions depicted in the image data (e.g., portions of the top surface of the package or lateral dimensions). After detection, the robotic system 100 can compare one or more physical properties (such as one or more dimensions, center / overlap area, top surface orientation, or contour shape, etc.) with the master data. In some embodiments, the robotic system 100 can include predetermined one or more physical properties and corresponding analysis processes associated with known deviations. In some embodiments, the robotic system 100 can adjust the stacking height limit 530 based on the abnormal object properties of the incoming object identified.

[0070] Placement rules

[0071] FIG. 5A to FIG. 5E is an illustration of an implementation of a placement rule in accordance with one or more embodiments of the present technology. FIG. 5A to FIG. 5E Various stacking scenarios corresponding to various placement rules are shown. The robotic system 100 can use the placement rules to derive placement locations for objects within a specified placement platform 308. For example, the robotic system 100 can discard or disqualify potential placement locations 430 that fail to meet one or more placement rules.

[0072] Some examples of placement rules may be used to place objects on top of each other, such as for stacking / placing one or more layers of packages on top of one or more layers of other packages. The robotic system 100 may use placement rules to improve / ensure the stability of the stacked objects and prevent any objects from sliding and / or tipping over during movement of the placement platform 308. Additionally, the robotic system 100 may use placement rules to increase the total number of items packaged while reducing collisions that may occur when placing such items. Figure 5A The final state after placing the first item and the initial state after placing the second item are shown. For illustrative purposes, Figure 5B , Figure 5D and Fig. 5F Various scenarios are shown of a top package positioned directly above and supported by (eg, in direct contact with) one or more support packages. FIG. 5B to FIG. 5D Shows the different packing sizes from Figure 5A Various packaging scenarios in which the second item is placed in the initial state are shown. Figure 5E An alternative stacking possibility for the first three items is shown which reduces the height difference at the placement locations. Fig. 5F Shows the deviation FIG. 5B to FIG. 5E Stacking possibilities for the modes shown.

[0073] The robotic system 100 can be configured to determine a target placement location for a target package 410. The robotic system 100 can use the current stacking height 510 of the placement location 430 to derive the target placement location. In some embodiments, the robotic system 100 can determine the current stacking height 510 of the packages stacked on the placement location 430 in real time. The robotic system 100 can use the imaging device 222 to analyze the depth measurement (e.g., point cloud data) between the top surface of the stacked packages and the placement platform 308. In some embodiments, the robotic system 100 can access the recorded height information of each stacked package on the placement location 430 from the master data 252. Therefore, the robotic system 100 can determine the current stacking height 510 of the placement location 430 by summing the recorded height information of each stacked package.

[0074] When determining the appropriate placement location for the target package 410, the robotic system 100 may determine the combined stack height of the placement location 430 as the sum of the current stack height 510 of the placement location 430 and the height 416 of the target package 410. In other words, the robotic system 100 may determine the combined stack height as the height that would result if the target package 410 were placed at the corresponding placement location (e.g., stacked on top of a previously placed package at the corresponding location).

[0075] The robotic system 100 may use placement rules to derive target placement locations 430 for target packages 410. The placement rules may include provisions, requirements, or a combination thereof for controlling or limiting the current stack height 510 between placement locations 430. For example, the placement rules may be based on height requirements, sequence requirements, or a combination thereof.

[0076] The height requirement may include a maximum stack height limit 530 (e.g., a percentage or static distance relative to the size of the placement platform) for the current stack height 510 of the placement location 430. In some embodiments, the height requirement may be a requirement that controls the relative height between adjacent stacks. In other words, the height requirement may require that the current stack height 510 of the placement location 430 be within a predetermined range compared to the current stack heights 510 of other placement locations.

[0077] The sequence requirement may require that the stack height be in accordance with a predetermined sequence or order of the placement positions. For example, if the identifier of a first placement position precedes or is lower in sequence than the identifier of a second placement position, the sequence requirement may require that the stack height of the first placement position be greater than the stack height of the second placement position. In some embodiments, the sequence requirement may be determined by the relationship hi ≤h j , where i>j is used to illustrate, where h i and h j represents the current stack height of the first placement location and the second placement location, and i and j represent the sequence identifiers 450 of the first placement location and the second placement location, respectively. As described above, the placement locations may be sorted or sequenced according to the distance between the corresponding placement location and the robot unit 440.

[0078] In some embodiments, the robotic system 100 can use the stacking height limit 530 to derive the target placement location 430 of the package 410. The stacking height limit 530 can be a predetermined maximum value that limits the current stacking height 510 of the placement location 430. In some embodiments, the robotic system 100 can deviate from the placement rule and follow the exception rule. For example, when the total height of the second placement location exceeds the stacking height limit, the robotic system 100 can stack the target package 410 at a first placement location that is closer to the robot unit than the second placement location 430, and the total height of the first placement location is greater than the current stacking height of the second placement location. The robotic system 100 can determine the stacking height limit 530 based on user input (e.g., customer-specified requirements), vehicle size, robot characteristics (e.g., the maximum reachable height of the robot unit 440 for one or more placement locations), etc.

[0079] When determining the target placement location 430 of the target package 410, the robot system 100 can determine whether the target package 410 can be placed at the selected placement location based on the current stack height and the sequence identifier 450 corresponding to the placement location 430. For example, the robot system 100 can determine the combined stack height of the selected placement location to compare with the current stack height of the placement location 430 available on the placement platform 308. In addition, the robot system 100 can determine the sequence identifier of the selected placement location to compare with the sequence identifier 450 of the placement location 430. Using the combined stack height, the current stack height, and the sequence identifier 450, the robot system 100 can iteratively compare the combined stack height and sequence identifier of the selected placement location with the current stack height and sequence identifier 450 of the placement location 430 to determine whether the selected placement location at least meets the placement rule or the exception rule. When the selected placement location meets the placement rule, the exception rule, or both, the robot system 100 can determine that the target package 410 can be placed at the selected placement location. When the placement location fails to meet one or more of the rules, the robot system 100 can eliminate the placement location.

[0080] In some embodiments, the robotic system 100 may iteratively compare the selected placement location to the placement locations 430, starting with the first placement location closest to the robotic unit (e.g., with the largest value in the sequence identifier) ​​and sequentially following the order of the sequence identifiers 450 until ending at the last placement location farthest from the robotic unit. The robotic system 100 may select the first valid selected placement location as the target placement location 430 for the target package 410. In other embodiments, the robotic system 100 may repeatedly select a previously selected target placement location of the target placement location for the target package 410 until a placement rule or exception rule is violated. When repeatedly selecting a previously selected target placement location, the robotic system 100 may skip the comparison between the selected placement location and the placement location 430.

[0081] As an illustrative example, Figure 5A 1 shows a time point at which a package has been placed on a pallet with three placement locations. Before placing the package, the heights of the three placement locations are zero. Thus, the robotic system 100 can select the farthest placement location (e.g., identifier 1, where the robotic unit 440 is located on the left side of the pallet) to place the package. After placing the target, the current height 501 at identifier 1 can correspond to the height (e.g., h1) of the placed package (e.g., package 1).

[0082] Figure 5B The placement of a second package (e.g., package 2a) having a height 502 greater than or equal to the height 501 of the first package is shown. When considering the placement location of the second package, the robot system 100 can calculate the combined stacking height of placement location 1 (e.g., h1+h2a) and placement location 2 (e.g., h2a). The robot system 100 can use the combined stacking height of the resulting candidate positions to compare with the rules. Since h2a≥h1, the combined height resulting from placing the second package at position 2 will be greater than the current height of position 1, thereby violating the placement rule. Alternatively, the combined height at position 1 will keep the height of the farther stack higher than the height of the closer position, and keep the combined height below the maximum stacking height, thereby complying with the placement rule. In addition, the robot system 100 can ignore the exception rule because the combined height of the two candidate positions remains below the maximum stacking height.

[0083] Figure 5C and Figure 5D Different placement possibilities for different second packages (eg, package 2b) having a height 503 that is less than the height 501 of the first package are shown. Figure 5C A second package 2b placed at position 2 may be shown, and Figure 5DA second package 2b placed at position 1 and stacked on top of the first package 1 may be shown. The robotic system 100 may calculate the combined height of the two candidate placements and evaluate them according to the rules. Since the height 2b of the second package 2b is less than the height h1 of the first package, subtracting the height 2b from the height 1 may provide a positive number 504. Therefore, the robotic system 100 may determine that placing the second package 2b at the second position satisfies the placement rule. In addition, the robotic system 100 may determine that placing the second package 2b at the first position also satisfies the placement rule and keeps the height below the maximum threshold. When the robotic system 100 identifies multiple valid placement possibilities, the robotic system 100 may use one or more predetermined processes to select between valid possible positions. For example, the robotic system 100 may have a predetermined rule to first select a farther position and increase the highest stack height or disperse the placement to reduce the height difference between the stacks. Figure 5E The figure shows the preference for reducing the height difference of the placement position when placing the third package 3a. Figure 5C transformation.

[0084] In other embodiments, the robotic system 100 can generate a packing plan using a separate subsystem (e.g., a planner). For example, when the total contents of a pallet are known, such as based on an order or shipping list, the robotic system 100 can calculate a packing plan (e.g., a designated location for each package in the contents) before placing the package. In this case, the robotic system 100 can consider different placement scenarios during the planning phase. During the placement phase, the robotic system 100 can follow a stacking plan instead of evaluating different placement possibilities. In addition, during the placement phase, the robotic system 100 can use various rules to consider the out-of-order arrival of packages at the starting position or packages with multiple placement locations within the stack. Additionally or alternatively, the robotic system 100 can use placement rules to generate a stacking sequence during the packaging planning phase.

[0085] Fig. 5F Exceptions to the rules are shown. Fig. 5F Can be expressed Figure 5Bstacking sequence thereafter. For example, after stacking the second package (identifier 2a) at position 1, the robot system 100 may place the third package (identifier 3) at position 2, because stacking the third package 3 at position 1 will produce a combined height that exceeds the maximum stacking height. In addition, the robot system 100 can eliminate the possibility of position 3 as the third package because the corresponding combined height will exceed the current height of position 2 and violate the placement rule. Subsequently, when evaluating the placement position of the fourth package (identifier 4), the robot system 100 can determine that the remaining height 506 at position 1 (e.g., the difference between the current height and the stacking height limit 530) is less than the minimum height dimension of the known or available object. In response, the robot system 100 can effectively eliminate the position by indicating that position 1 has been filled. Therefore, the robot system 100 can invoke the exception rule and select position 2, where the combined height at position 2 exceeds the current height of position 1. Otherwise, such as when the remaining height at position 1 is greater than one or more dimensions of the possible or expected package, the robot system 100 can select position 3 as the placement position for package 4. In other embodiments, the robotic system 100 may invoke an exception rule when the remaining height 506 is within the height threshold.

[0086] Example placement sequence

[0087] FIG. 6A to FIG. 6B 3 is a diagram of a first exemplary placement sequence and a second exemplary placement sequence according to one or more embodiments of the present technology. The robotic system 100 can use placement rules and / or exception rules to derive a first placement sequence 610 and a second placement sequence 620 for placing a set of objects on the derived placement positions 601-603 of the placement platform 308 (e.g., a pallet) of FIG. 3. The first placement position 601 can be located farthest from the robotic unit, and the third placement position 603 can be located closest to the robotic unit. The first placement sequence 610 and the second placement sequence 620 correspond to side views representing the vertical occupancy of objects placed on the placement platform 308.

[0088] for Fig. 6AIn the first exemplary placement sequence 610 shown, a first group of target objects 611-617 are designated to be placed on the placement platform 308 in placement locations 601-603. Therefore, the robotic system can derive the placement sequence 610 using the placement rules. The robotic system 100 can start the placement sequence 610 by placing the first target object 611 in the first placement location 601 farthest from the robotic unit according to the placement rules. Subsequently, the second target object 612 can be placed on the first placement location 601 directly above the first target object 611. In other embodiments, the robotic system 100 can place the second target object 612 in the second placement location 602 because the vertical height of the second target object 612 is less than the vertical height of the first target object 611. When the robotic system 100 identifies multiple valid placement possibilities, the robotic system 100 can use one or more predetermined processes to select between valid possible positions. For example, the sequence 610 shows that the robotic system 100 follows the predetermined rules to first select farther positions and increase the highest stacking height.

[0089] The third target object 613 can be placed on the second placement location 602 because the combined vertical height of the first three target objects will exceed the stacking height limit 630. The fourth target object 614 can be placed on the second placement location 602 because the combined vertical height of the third target object 613 and the fourth target object 614 will be within the stacking height limit 630. In other embodiments, the robotic system 100 can place the fourth target on the third placement location 603 because the height of the fourth target object is less than the stacking height on the placement location 602. Similar to the placement of the second target object 612, the robotic system 100 can follow a predetermined rule to first select a farther position and increase the highest stacking height. In other embodiments, the robotic system 100 can use one or more alternative predetermined processes to select between multiple valid possible positions for the fourth placement object.

[0090] The fifth target object 615 can be placed in the third placement position 603 because placing the fifth target object 615 in the first placement position 601 or the second placement position 602 will cause the combined stack height to exceed the stack height limit 630, thereby violating the placement rule. Similarly, the sixth target object 616 and the seventh target object 617 can be placed in the third placement position 603 because their object heights will be greater than the height difference between the stack height limit 630 and the current stack height at the placement positions 602 and 603.

[0091] for Figure 6BIn the second exemplary placement sequence 620 shown, a second set of target objects 621-627 are designated to be placed in the placement locations 601-603 on the placement platform 308. Thus, the robotic system can derive the placement sequence 620 using placement rules and exception rules. The robotic system 100 begins the placement sequence 620 by placing a first target object 621 at the first placement location 601, which is farthest from the robotic unit. Subsequently, a second target object 622 is also placed at the first placement location 601, directly above the first target object 621. In other embodiments, the robotic system 100 can place the second target object 622 in the second placement location 602 because the vertical height of the second target object 622 is equal to the vertical height of the first target object 621.

[0092] The third target object 623 must be placed on the second placement location 602 because the combined vertical height of the first three target objects will exceed the stacking height limit 630. The fourth target object 624 is placed on the second placement location 602. In other embodiments, the robotic system 100 can place the fourth target object 624 in the third placement location 603 because the vertical heights of the third target object 623 and the fourth target object 624 are equal. The fifth target object 625 is placed on the second placement location 602 above the fourth target object 624. Therefore, the combined stacking height of the second placement location 602 exceeds the combined stacking height of the first placement location 601 and violates the placement rule. Alternatively, the robotic system 100 can use an exception rule to allow the fifth target 625 to be placed on the second placement location 602. Subsequently, the sixth target object 616 and the seventh target object 617 are placed in the third placement location 603.

[0093] Exemplary System Operation

[0094] Figure 7 is an operation according to one or more embodiments of the present technology Figure 1 Flow chart of an exemplary method 700 for a robotic system of FIG. The method 700 can be used to derive a target placement position 430 for placing a target package 410 on a placement platform 308. The method 700 can be based on using Figure 2 One or more of the processors 202 execute the stored Figure 2 The processor 202 may control the Figure 3A The robot arm 302 and / or Figure 3A The end effector 304, such as Figure 1 The target object 112 from Figure 1 The starting position 114 is transferred to Figure 1When the pallet of the task position 116 is located. For example, the processor 202 can send commands, settings and / or other communications effective to control the robotic unit 440 to manipulate the parts / objects and place them at corresponding placement locations on the pallet.

[0095] At block 702, the robotic system 100 may identify one or more objects (e.g., Figure 4A For example, the robotic system 100 can identify objects that are available for packaging, within an inbound shipment, arriving at a specified location, located at a source, designated for placement, and / or listed in an order / request / manifest.

[0096] Furthermore, at block 702, the robotic system 100 may identify a pallet that may be used to receive and stack the identified object. For example, the robotic system 100 may identify a placement platform 308 that may be used to receive and stack the identified object. Additionally, the robotic system 100 may determine the characteristics / features (e.g., category, size, identifier, etc.) of the identified placement platform 308. The robotic system 100 may also determine a base surface of the placement platform 308 for placing and / or stacking the identified object. The robotic system 100 may interface with another system (e.g., a transport robotic system) to access information from the placement platform 308. Figure 2 information from the master data 252, and / or obtaining real-time information from the placement platform 308 (via, for example, sensors at predetermined locations) to identify the placement platform 308 and / or its characteristics.

[0097] At block 704, the robotic system 100 may obtain (e.g., by generating in real time and / or from Figure 2 For example, the robotic system 100 may obtain a physical property (e.g., size, shape, anomaly) of each identified object (such as Figure 4A The robot system 100 may also group the identified objects into one or more groups of identified objects based on similar dimensions (e.g., length and / or width). In addition, the robot system 100 may obtain information about the abnormal physical characteristics of each identified object. In some embodiments, the robot system 100 may determine the physical properties of the identified objects in real time (e.g., after receiving an order and / or before starting a placement operation, or offline) based on image data and / or depth metrics (e.g., cloud points) received from an imaging sensor. In some embodiments, the robot system 100 may assess whether the abnormal object properties prevent additional stacking on top of the target package 410. Therefore, the robot system 100 may adjust the stacking height limit 530 based on the assessment.

[0098] At box 706, the robot system 100 can determine the stacking posture and corresponding lateral substrate area (e.g., occupied space) of the expected packaging target. For example, the robot system 100 can determine the stacking posture of the target package 410 selected from the identified object. Using the recorded physical dimensions (e.g., length and / or width) of the target package 410, the robot system 100 can form the lateral substrate area of ​​the target package 410 (e.g., by forming the periphery of the edge around the substrate surface of the expected packaging target and / or by rotating the shape template included in the master data 252). In some embodiments, the robot system 100 can fix the lateral dimensions (e.g., length and width) of the expected packaging target to a constant value. Therefore, the robot system 100 can assign a fixed lateral substrate area to each expected packaging target after calculating the lateral substrate area once.

[0099] At block 708, the robotic system 100 may determine a placement location 430 for stacking the intended packaging object on the placement platform 308. For example, the robotic system 100 may use the lateral base area to divide the base surface of the placement platform 308 into one or more different placement areas (e.g., bounded areas on the bottom surface of the placement platform 308 for stacking packages) to define the boundary perimeters of the different placement areas. For each different placement area, the robotic system 100 may define a new placement location and determine the characteristics / features of the new placement location 430 (e.g., spatial location, size, height relative to the base surface of the placement platform 308). In other embodiments, the robotic system 100 may access one or more predetermined placement locations corresponding to an order, a pallet, a group of items, or a combination thereof for the current task.

[0100] When determining placement locations for target packaging objects, the robotic system 100 can determine a candidate sequence (e.g., a preferred stacking order) of the placement locations 430 by assigning a sequence identifier (e.g., an identification number) to each placement location 430 based on a distance measurement (e.g., proximity to a certain location and / or object). For example, the robotic system 100 can identify a lateral (e.g., xy plane) reference for calculating the distance measurement of the placement location 430. In some embodiments, the lateral reference can be a position of the robotic unit 440 (e.g., a base of a robotic arm) lateral to the base surface of the placement platform 308 and / or the placement location 430.

[0101] At block 722, the robotic system 100 may calculate the distance of the placement locations 430 from the robotic unit. For example, the robotic system 100 may determine the distance between each placement location 430 (e.g., the coordinate center of the placement area perimeter) and the robotic unit 440 (e.g., the unit center coordinates) transverse to the placement location 430 and / or the placement platform 308. In some embodiments, the robotic system 100 may calculate the separation distance between the robotic unit and the placement location 430 using a predetermined configuration of the robotic unit (e.g., pre-recorded position data).

[0102] At block 724, the robotic system 100 may generate sequence identifiers 450 corresponding to the placement locations 430 based on the distances of the placement locations 430 from the robotic unit. The sequence identifiers 450 (e.g., the order of the identification numbers) represent the ordering of the placement locations 430. For example, the robotic system may assign the sequence identifiers 450 in ascending order based on the distances of the placement locations 430 in descending order. Thus, the first / lowest sequence identifier is assigned to the placement location 430 that is farthest from the robotic unit, while the last / highest sequence identifier is assigned to the placement location 430 that is closest to the robotic unit.

[0103] At box 710, the robot system 100 can determine the target placement location 430 of the target package 410 selected from the expected packaging target. For example, the robot system 100 can assign the target package 410 to a serial identifier corresponding to the target placement location 430. The robot system 100 can use the feature information / characteristics (e.g., position, size, shape) of the placement location 430 on the placement platform 308 to select the target placement location 430. The robot system 100 can use the feature information to determine the current stack height and the combined stack height of the placement location 430. Using the current stack height and the combined stack height, the robot system 100 can determine the candidate positions including the placement location 430 that meets the placement rule or the exception rule. The robot system 100 can select the best candidate position from the candidate positions as the target placement location 430 by assigning the serial identifier of the best candidate position to the target package 410.

[0104] At box 726, the robot system 100 can calculate the current stack height of the placement location 430 (e.g., the height of the topmost package placed). For example, the robot system 100 can calculate the current stack height of the placement location 430 by measuring the vertical distance between the placement platform 308 and the top surface of the placed object at the placement location 430. The robot system 100 can be configured to analyze the depth measurement (e.g., cloud point data) from the imaging sensor in real time to determine the vertical distance between the base surface of the placement platform 308 and the top surface of the placed object. In other embodiments, the robot system 100 can determine the current stack height by retrieving the recorded vertical distance measurement from the master data 252. In addition or alternatively, the robot system 100 can use the characteristics / characteristics (e.g., height measurement) of the identified group of stacked packages at the placement location 430 to calculate the current stack height.

[0105] At box 732, the robot system 100 can identify a group of stacked packages at the placement location 430 (e.g., packages previously placed by the robot unit) to determine the physical dimensions of each package in the group of stacked packages. For example, the robot system 100 can use a record log (e.g., a sequence of motion plans) of the placement location 430 from the tracking history / log to identify the individual packages. The robot system 100 can retrieve the physical properties (e.g., height) of each individual package from the master data 252. Alternatively, in the absence of a record log, the robot system 100 can use a depth metric (e.g., cloud point) from an imaging sensor to determine the height of each individual package in real time. The robot system 100 can use the depth metric to determine the height as the distance between the top surface and the bottom surface of each individual package. In addition, the robot system 100 can use the robot arm 302 to displace the individual packages in the group of stacked packages to distinguish the top surface and the bottom surface of each individual package.

[0106] At block 734, the robotic system 100 may calculate a total height based on the heights of the individual packages (e.g., the current stack height of the packages stacked at the placement location 430). For example, the robotic system 100 may calculate the total height of the placement location 430 by combining the individual package heights of the stacked packages. The total height of the placement location 430 may represent the vertical distance from the placement platform 308 to the top surface of the stacked packages. In addition, the total height of the placement location 430 represents the current stack height of the placement location 430.

[0107] At block 728, the robotic system may use the current stack height and the sequence identifier 450 of the placement location 430 to select a target placement location 430. For example, the robotic system 100 may use the current stack height and the sequence identifier 450 to identify a candidate location that may be selected as the target placement location 430. The robotic system 100 may calculate a combined stack height representing the candidate location for placing the target package 410 on the placement platform 308 and / or the packages stacked at the candidate location. Using the combined stack height of the candidate locations and the current stack height of the remaining placement locations, the robotic system 100 may determine the candidate location with the highest cumulative stack height.

[0108] At block 736, the robotic system 100 may identify candidate positions including a placement position 430 that may be selected as the target placement position 430. For example, the robotic system 100 may identify candidate positions that each maintain a corresponding resulting height within a stack height limit when the target object is placed. Additionally or alternatively, the resulting heights of the candidate positions maintain a current stack height for placement positions closer to the robotic unit to be lower than or equal to a current stack height for placement positions farther from the robotic unit. In other embodiments, the robotic system 100 may identify candidate positions that meet a placement rule or an exception rule.

[0109] At block 738, the robotic system 100 may calculate the combined stack heights of the candidate locations. For example, the robotic system 100 may calculate the combined stack heights of the candidate locations by adding the height of the target package 410 to the current stack heights of the candidate locations.

[0110] At block 740, the robotic system 100 may determine the target placement location 430 using the combined stack heights of the candidate locations and the current stack heights of the remaining placement locations. For example, the robotic system 100 may determine the target placement location 430 by calculating the cumulative stack height of each candidate location by adding the corresponding combined stack height to the current stack height of the remaining placement locations. The robotic system 100 may select the candidate location with the highest cumulative stack height as the target placement location 430 for the target package 410. In other embodiments, the robotic system 100 may determine the target placement location 430 by comparing the combined stack heights of the candidate locations and selecting the candidate location with the highest combined stack height. The robotic system 100 may assign a sequence identifier of the target placement location 430 to the target package 410.

[0111] At block 712, the robotic system 100 may derive and implement a stacking motion plan for the target package 410 using the target placement location 430. For example, the robotic system 100 may retrieve the location information of the target placement location 430 using a sequence identifier assigned to the target placement location 430 of the target package 410. In some embodiments, the robotic system may use the location information to derive a stacking motion plan corresponding to the target placement location. Each stacking motion plan may correspond to an object and include a stacking motion path or a set of corresponding commands / settings for the object and / or the robotic unit 440 (e.g., a robotic arm and / or an end effector). The stacking motion plan may correspond to the following operations of the robotic unit 440: approaching the object at its starting position, grasping the object with an end effector, lifting and transferring the object to its placement position, and releasing / placing the object at the target placement position. In some embodiments, the robotic system 100 may iteratively and in reverse order (e.g., from the placement position to the starting position) derive the path of the target object using incremental changes in candidate positions. The robotic system 100 may maintain a sequence of positions that avoids collisions and minimizes travel distance, manipulation, resources, and / or other consumption parameters to derive the path. The robotic system 100 may use the resulting path to derive a motion plan.

[0112] The robotic system 100 may implement the stacking motion plan, such as by communicating one or more of the stacking motion plan and / or corresponding commands / settings to the target robotic unit 440. The robotic system may also implement the motion plan by executing the commands / settings at the target robotic unit 440. Thus, the robotic system may operate the robotic unit 440 to transfer the objects from the starting position to the corresponding target placement position according to the stacking motion plan.

[0113] The robotic system 100 may repeat the above process each time a new object is received or arrives at the starting location. For example, one or more sensors (e.g., a crossing sensor, a weight sensor, etc.) may indicate to the task station that a new object is present at one of the inbound conveyors. The robotic system 100 may obtain image data corresponding to the new object, detect or identify the new object, and then determine a target placement location for the new object. Once the placement location is determined, the robotic system 100 may derive and implement a corresponding motion plan to transfer the object to the target placement location. The robotic system 100 may repeat the process until the intended group of objects has been packaged and / or all placement locations have reached the maximum allowable height.

[0114] in conclusion

[0115] The above specific embodiments of the examples of the disclosed technology are not intended to be exhaustive or to limit the disclosed technology to the precise form disclosed above. Although the specific examples of the disclosed technology are described for illustrative purposes, as those skilled in the relevant art will recognize, multiple equivalent modifications are also possible within the scope of the disclosed technology. For example, although the process or frame is presented in a given order, alternative embodiments may perform routines with steps in different orders, or adopt systems with frames in different orders, and some processes or frames may be deleted, moved, added, subdivided, combined and / or modified to provide alternatives or sub-combinations. Each of these processes or frames may be implemented in a variety of different ways. In addition, although the process or frame is sometimes shown as being executed in series, these processes or frames may be alternatively executed or implemented in parallel, or may be executed at different times. In addition, any specific numbers mentioned herein are merely examples; alternative embodiments may adopt different values ​​or ranges.

[0116] According to the above specific embodiments, these and other changes can be made to the disclosed technology. Although the specific embodiments describe certain examples of the disclosed technology and the expected best mode, no matter how detailed the above description appears in the text, the disclosed technology can be practiced in many ways. The details of the system may vary greatly in its specific embodiment, but are still covered by the technology disclosed herein. As described above, the specific terms used when describing certain features or aspects of the disclosed technology should not be considered to imply that the terms are redefined herein to be limited to any specific characteristics, features or aspects of the disclosed technology associated with the terms. Therefore, the present invention is not limited except for the appended claims. Generally, unless the above specific embodiment section explicitly defines the terms used in the appended claims, such terms should not be interpreted as limiting the disclosed technology to the specific examples disclosed in the specification.

[0117] Although certain aspects of the invention are presented below in certain claim forms, applicants contemplate the various aspects of the invention in any number of claim forms. Accordingly, applicants reserve the right to seek additional claims after filing this application or in a subsequent application to seek such additional claim forms.

Claims

1. A method for operating a robotic system, the method comprising: identifying a package set representing available packages for placement on the platform; obtaining a set of object properties of one or more packages in the package set, wherein the object properties represent physical size, shape, or a combination thereof of the one or more packages in the package set; determining a placement location on the platform using the set of object properties, wherein the placement location represents a stacking location of the set of packages or a portion thereof; calculating a set of distances for the placement locations, wherein the distances are distances between a corresponding one of the placement locations and a robotic unit configured to transfer the set of packages to the platform; generating a sequence identifier for each of the placement locations based on the corresponding distance; A target placement position is determined for the target package from the placement positions, wherein the target placement position is determined by: calculating a current stack height of the placement location; and selecting the target placement location using the sequence identifier and the current set of stack heights, wherein a resulting height of the selected target placement location is within a stack height limit and maintains a height of a stack closer to a base robot lower than or equal to a height of a stack further from the base; as well as A motion plan is implemented for placing the target package on the platform at the target placement location.

2. The method according to claim 1, wherein: Calculating the current stack height includes: identifying a set of stacked packages currently placed at each of the placement locations; calculating a total height for each of the placement locations, wherein the total height represents a vertical distance from the platform to a top surface of a package placed at a corresponding one of the placement locations; and Determining the target placement position includes: Using the sequence identifier, identifying candidate positions that, when placing the target object, (1) maintain the corresponding resulting height within the stack height limit and (2) maintain a closer stack height lower than or equal to a farther stack height; For each candidate position, calculating a cumulative stack height by adding the corresponding resulting height to the height of the remaining stack; and One of the candidate positions having a highest cumulative stack height is determined as the target placement position for the target package.

3. The method according to claim 1, wherein: the generated sequence identifier is higher for the placement location closer to the robot unit and lower for the placement location farther from the robot unit; and The target placement position is based on the height-based placement rule h i ≤h j , where i>j is determined by, where i and j represent sequence identifiers, and h i and h j represents the height of the corresponding placement position, and The height-based placement rule is configured to keep the height of closer stacks lower than or equal to the height of farther stacks.

4. The method according to claim 3, wherein: The target placement location is determined based on an exception to the height-based placement rule, wherein the first candidate position has a combined stack height greater than the stack height limit, wherein the second candidate position has a combined stack height that is less than the stack height limit and greater than the current stack height of the first candidate position, wherein a first sequence identifier corresponding to the first candidate position is located before a second sequence identifier corresponding to the second candidate position, and The second candidate position is selected as the target placement position.

5. The method of claim 4, wherein the target placement location is determined according to the exception to the height-based placement rule when a height difference between a current stack height of the second candidate placement location and the stack height limit is within a height threshold.

6. The method according to claim 3, wherein: Determining the placement location includes obtaining a packing plan describing a pre-planned location of the set of packages above the platform; as well as When the target packages are received out of order, the target placement position is determined according to the height-based placement rule.

7. The method according to claim 3, further comprising: identifying a plurality of available objects for placement, wherein the plurality of available objects includes the target package; Determine a placement position corresponding to the available object outside the target package; as well as When the target placement location is farther than the placement locations corresponding to other available objects, the target package is selected for transfer.

8. The method according to claim 7, further comprising: calculating a remaining height at each of the placement locations, wherein the remaining height corresponds to a difference between the current stack height and the stack height limit at the corresponding one of the placement locations; and Wherein when the remaining height is less than the physical size of the package set, the target placement position is determined according to the exception to the height-based placement rule.

9. The method of claim 1, wherein determining the target placement location of the target package comprises: identifying the nature of the anomalous object associated with the target package, wherein the anomalous object properties correspond to physical deviations of the target package, including deformation, bending, misalignment and / or partial closure, and wherein properties of the anomalous object prevent additional stacking on top of the target package; as well as The identified unusual object properties are used to adjust the stack height limit.

10. The method of claim 1, wherein a subset of properties in the set of object properties includes one or more lateral dimensions that remain fixed across the one or more packages in the set of packages.

11. A tangible, non-transitory computer-readable medium having instructions stored thereon, which when executed by a processor causes the processor to perform a method comprising: identifying a package set representing available packages for placement on the platform; obtaining a set of object properties of one or more packages in the package set, wherein the object properties represent physical size, shape, or a combination thereof of the one or more packages in the package set; determining a placement location on the platform using the set of object properties, wherein the placement location represents a stacking location of the set of packages or a portion thereof; calculating a set of distances for the placement locations, wherein the distances are distances between a corresponding one of the placement locations and a robotic unit configured to transfer the set of packages to the platform; generating a sequence identifier for each of the placement locations based on the corresponding distance; A target placement position is determined for the target package from the placement positions, wherein the target placement position is determined by: calculating a current stack height of the placement location; and selecting the target placement location using the sequence identifier and the current set of stack heights, wherein a resulting height of the selected target placement location is within a stack height limit and maintains a height of a stack closer to a base robot lower than or equal to a height of a stack further from the base; as well as A motion plan is implemented for placing the target package on the platform at the target placement location.

12. The tangible, non-transitory computer-readable medium of claim 11, wherein: the generated sequence identifier is higher for the placement location closer to the robot unit and lower for the placement location farther from the robot unit; and The target placement position is based on the height-based placement rule h i ≤h j , where i>j is determined by, where i and j represent sequence identifiers, and h i and h j represents the height of the corresponding placement position, and The height-based placement rule is configured to keep the height of closer stacks lower than or equal to the height of farther stacks.

13. The tangible, non-transitory computer-readable medium of claim 12, wherein: The target placement location is determined based on an exception to the height-based placement rule, wherein the first candidate position has a combined stack height greater than the stack height limit, wherein the second candidate position has a combined stack height that is less than the stack height limit and greater than the current stack height of the first candidate position, wherein a first sequence identifier corresponding to the first candidate position is located before a second sequence identifier corresponding to the second candidate position, and The second candidate position is selected as the target placement position.

14. The tangible, non-transitory computer-readable medium of claim 12, wherein: Determining the placement location includes obtaining a packing plan describing a pre-planned location of the set of packages above the platform; as well as When the target packages are received out of order, the target placement position is determined according to the height-based placement rule.

15. The tangible, non-transitory computer-readable medium of claim 12, wherein the method further comprises: identifying a plurality of available objects for placement, wherein the plurality of available objects includes the target package; Determine a placement position corresponding to the available object outside the target package; as well as When the target placement location is farther than the placement locations corresponding to other available objects, the target package is selected for transfer.

16. A robot system comprising: at least one processor; as well as at least one memory device coupled to the at least one processor, the at least one memory device having instructions stored thereon, the instructions, when executed by the processor, causing the processor to: identifying a package set representing available packages for placement; obtaining a set of object properties of one or more packages in the package set; determining a placement location on a platform using the set of object properties, wherein the placement location represents a stacking location of the set of packages or a portion thereof; calculating a set of distances for the placement locations, wherein the distances are distances between a corresponding one of the placement locations and a robotic unit configured to transfer the set of packages to the platform; generating a sequence identifier for each of the placement locations based on the corresponding distances; determining a target placement location for the target package from the placement locations; as well as A motion plan is implemented for placing the target package on the platform at the target placement location.

17. The robot system according to claim 16, wherein: the generated sequence identifier is higher for the placement location closer to the robot unit and lower for the placement location farther from the robot unit; and The target placement position is based on the height-based placement rule h i ≤h j , where i>j is determined by, where i and j represent sequence identifiers, and h i and h j represents the height of the corresponding placement position, and The height-based placement rule is configured to keep the height of closer stacks lower than or equal to the height of farther stacks.

18. The robot system according to claim 17, wherein: The target placement location is determined based on an exception to the height-based placement rule, wherein the first candidate position has a combined stack height greater than the stack height limit, wherein the second candidate position has a combined stack height that is less than the stack height limit and greater than the current stack height of the first candidate position, wherein a first sequence identifier corresponding to the first candidate position is located before a second sequence identifier corresponding to the second candidate position, and The second candidate position is selected as the target placement position.

19. The robot system according to claim 17, wherein: Determining the placement location includes obtaining a packing plan describing a pre-planned location of the set of packages above the platform; as well as When the target packages are received out of order, the target placement position is determined according to the height-based placement rule.

20. The robotic system of claim 17, wherein the at least one memory includes instructions executable by the processor to: identifying a plurality of available objects for placement, wherein the plurality of available objects includes the target package; Determine a placement position corresponding to the available object outside the target package; as well as When the target placement location is farther than the placement locations corresponding to other available objects, the target package is selected for transfer.

Images