Robot system with dynamic packing mechanism

The robotic system dynamically adjusts object placement based on real-time conditions, addressing the limitations of fixed packing plans and reducing costs by enhancing adaptability and efficiency in handling complex tasks.

JP7589894B2Active Publication Date: 2024-11-26MUJIN INC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2020072970
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-05-31
Filing Date
2020-04-15
Publication Date
2024-11-26
Estimated Expiration
2039-09-19

AI Technical Summary

Technical Problem

Robots lack the sophistication and adaptability to handle complex tasks with real-world uncertainties, such as varying object sequences and orientations, leading to inefficiencies and high costs due to the need for fixed packing plans and human intervention.

Method used

A robotic system that dynamically derives optimal storage positions for objects using discretization mechanisms, allowing real-time adjustments based on actual conditions, eliminating the need for pre-defined sequences and reducing human intervention.

Benefits of technology

Improves packing efficiency, accuracy, and reduces costs by dynamically adapting to real-time conditions, enabling flexible object placement without the need for sequence buffers and human assistance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007589894000001
    Figure 0007589894000001
  • Figure 0007589894000002
    Figure 0007589894000002
  • Figure 0007589894000003
    Figure 0007589894000003
Patent Text Reader

Abstract

To solve the problem in which robots lack the granularity of control and the flexibility in executed actions to take into account deviations or uncertainties that may result from various real-world factors.SOLUTION: A method for operating a robotic system includes: determining a discretized object model representative of a target object; determining a discretized platform model representative of a task location; and determining height measures based on real-time sensor data representative of the task location. In addition, a placement location is dynamically derived based on (1) overlapping the discretized object model and the discretized platform model for stacking objects at the task location and (2) calculating a placement score associated with the overlapping based on the height measures.SELECTED DRAWING: Figure 7
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application includes subject matter related to a concurrently filed U.S. patent application entitled "A ROBOTIC SYSTEM WITH PACKING MECHANISM" by Rosen N. Diankov and Denys Kanunikov, which is assigned to Mujin, Inc. and identified by attorney docket number 131837-8005.US01, and which is incorporated herein by reference in its entirety.

[0002] This application includes subject matter related to a concurrently filed U.S. patent application entitled "A ROBOTIC SYSTEM WITH ERROR DETECTION AND DYNAMIC PACKING MECHANISM" by Rosen N. Diankov and Denys Kanunikov, which is assigned to Mujin, Inc. and identified by attorney docket number 131837-8007.US01, and which is incorporated herein by reference in its entirety.

[0003] This application includes subject matter related to a concurrently filed U.S. patent application entitled "ROBOTIC SYSTEM FOR PROCESSING PACKAGES ARRIVING OUT OF SEQUENCE" by Rosen N. Diankov and Denys Kanunikov, which is assigned to Mujin, Inc. and identified by attorney docket number 131837-8008.US01, and which is incorporated herein by reference in its entirety.

[0004] This application includes subject matter related to a concurrently filed U.S. patent application entitled "ROBOTIC SYSTEM FOR PALLETIZING PACKAGES USING REAL-TIME PLACEMENT SIMULATION" by Rosen N. Diankov and Denys Kanunikov, which is assigned to Mujin, Inc. and identified by attorney docket number 131837-8009.US01, and which is incorporated herein by reference in its entirety.

[0005] The present technology is directed generally to robotic systems, and more particularly to systems, processes, and techniques for packing objects based on dynamic computations. [Background technology]

[0006] With ever-improving performance and decreasing costs, many robots (e.g., machines configured to automatically / autonomously perform physical actions) are now widely used in many fields. Robots can be used to perform various tasks (e.g., manipulating or transposing objects through space), for example, in manufacturing and / or assembly, packing and / or packaging, transportation and / or shipping, etc. In performing tasks, robots can replicate human actions, thereby replacing or reducing human involvement that would otherwise be required to perform dangerous or repetitive tasks.

[0007] However, despite technological advances, robots often lack the sophistication necessary to replicate the human sensitivity and / or adaptability required to perform more complex tasks. For example, robots often lack the granularity and flexibility of control in the actions they perform to account for variability or uncertainty that may result from various real-world factors. Thus, there remains a need for improved techniques and systems for controlling and managing various aspects of robots to complete tasks despite various real-world factors. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an exemplary environment in which a robotic system having a dynamic packing mechanism may operate. [Diagram 2] FIG. 1 is a block diagram illustrating a robotic system in accordance with one or more embodiments of the present technology. [Figure 3A] FIG. 1 is a diagram of a discretized object in accordance with one or more embodiments of the present technique. [Figure 3B] FIG. 1 is an illustration of a discretized packing table in accordance with one or more embodiments of the present technique. [Figure 4A] FIG. 13 is an illustration of a calculation of support according to one or more embodiments of the present technology. [Figure 4B] FIG. 1 is a diagram of a support metric in accordance with one or more embodiments of the present technology. [Diagram 5] 1 is a top view illustrating an exemplary placement performed by a robotic system in accordance with one or more embodiments of the present disclosure. [Figure 6A] 1 is a side view illustrating a first exemplary approach according to one or more embodiments of the present disclosure. [Figure 6B] 11 is a side view illustrating a second exemplary approach according to one or more embodiments of the present disclosure. [Figure 7] 2 is a flow diagram for operating the robotic system of FIG. 1 in accordance with one or more embodiments of the present technique. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Described herein are systems and methods for dynamically packing objects (e.g., packages and / or boxes). A robotic system (e.g., an integrated system of devices performing one or more designated tasks) configured according to some embodiments improves packing and storage efficiency by dynamically deriving optimal storage locations for objects and stacking them accordingly.

[0010] Conventional systems use offline packing simulators to predetermine packing sequences / placements. Conventional packing simulators process object information (e.g., case shapes / sizes) for a given or estimated set of cases to generate a packing plan. Once determined, the packing plan specifies and / or requires specific placement positions / poses of objects at a destination (e.g., pallets, bins, cages, boxes, etc.), a predefined sequence of placements, and / or a given motion plan. From a given packing plan, conventional packing simulators may derive source requirements (e.g., sequences and / or placements for objects) that match or enable the packing plan. Because the packing plan is developed offline in conventional systems, the plan is independent of the actual packing operations / situations, object arrivals, and / or other system implementations. Thus, the overall operation / implementation requires that received packages (e.g., at a start / pick-up location) follow a fixed sequence that matches the given packing plan. Thus, conventional systems are unable to adapt to real-time conditions and / or variations in received packages (e.g., different sequences, positions, and / or orientations), unexpected errors (e.g., collisions and / or losses), real-time packing requirements (e.g., sales orders), and / or other real-time factors.

[0011] Furthermore, because conventional systems group and pack objects according to a strict predetermined plan / sequence, they require that all objects at a source location (1) have the same size / type and / or (2) arrive according to a known sequence. For example, conventional systems require that objects arrive at a pick location (e.g., via a conveyor) according to a fixed sequence. Also, for example, conventional systems require that objects at a pick location be placed at a specified location according to a predetermined pose. Thus, conventional systems require one or more operations to queue and / or place objects at the source (i.e., prior to the packing operation) according to a predetermined sequence / position. Often, conventional systems require a sequence buffer to queue and / or place objects at the source according to a predetermined sequence / pose, which can cost over $1 million.

[0012] In contrast to conventional systems, the robotic system described herein can dynamically derive a placement location for an object during system operation (e.g., when an object arrives or is identified and / or after initially starting one or more operations, such as an actual packing operation). In some embodiments, the robotic system can initiate / perform dynamic derivation of placement based on the occurrence of a trigger event, e.g., a reevaluation timing, a packing / operation error (e.g., a collision event or a lost event), an object that is not recognized (e.g., at the source and / or destination), a change in the position / orientation of a placed package, and / or other dynamic conditions. In some embodiments, a placement location can be dynamically derived when the robotic system does not have a priori information regarding the arriving object, e.g., to receive an object that is not known in advance and / or to receive a random / unknown sequence of objects. In dynamically deriving a placement location, the robotic system can utilize various real-time conditions (e.g., currently existing or ongoing conditions), including, for example, available / arriving objects, object characteristics and / or requirements, placement requirements, and / or other real-time factors.

[0013] The robotic system can derive the placement location based on a discretization mechanism (e.g., a process, a circuit, a function, and / or a routine). For example, the robotic system can use the discretization mechanism to describe the physical size / shape of the object and / or the target location according to a discretization unit (i.e., one discrete area / space). The robotic system can generate a discretized object profile that describes the envisioned object using the discretization unit, and / or a discretized destination profile that describes the target location (e.g., the top surface of a pallet and / or the space / bottom surface within a bin / case / box). Thus, the robotic system can convert a continuous real-world space / area into computer-readable digital information. Furthermore, the discretized data can enable reducing the computational complexity for describing package footprints and comparing various package placements. For example, package dimensions can correspond to integer numbers of discretized units instead of real-world decimal numbers, which facilitates mathematical calculations.

[0014] In some embodiments, the robotic system can check the discretized cells of the placement platform to determine the object placement probability. For example, the robotic system can use a depth measurement or height of the object placed on the placement platform. The robotic system can determine the depth dimension to determine the height in / accordance with the discretized cell. The robotic system can evaluate the depth dimension according to a group of discretized cells corresponding to the object to be placed. The robotic system can determine the maximum height in the group to evaluate the placement probability. In other words, the robotic system can determine whether the placement position being tested provides sufficient support such that the object to be placed can be placed relatively flat (e.g., according to a predetermined threshold and / or condition). More details regarding the derivation of dynamic placement are provided below.

[0015] Thus, the robotic system can improve the efficiency, speed, and accuracy of dynamically deriving object placement based on real-time conditions. For example, the systems described herein can derive placement locations given the uncertainties associated with and / or deviations from expected conditions due to real-world conditions. Thus, the robotic system can receive and pack unknown objects and / or objects that arrive randomly (i.e., not in a known / predetermined sequence).

[0016] Additionally, the robotic system can reduce overall costs by eliminating one or more actions, machine (e.g., sequence buffers), and / or human assistance that would be required in conventional systems to line up or place objects at the source and / or for packing operations (e.g., for error handling). By dynamically deriving placement locations as objects become available (e.g., based on object arrival and / or trigger events), the robotic system eliminates the need to reorganize or sequence packages along with associated machine / human actions.

[0017] In the following description, numerous specific details are set forth to provide a thorough understanding of the disclosed technology. In other embodiments, the technology presented herein may be practiced without these specific details. In other instances, well-known features, e.g., specific functions or routines, have not been described in detail so as not to unnecessarily obscure the present disclosure. References herein to "an embodiment," "an embodiment," or the like, mean that the particular feature, structure, material, or characteristic being described is included in at least one embodiment of the present disclosure. Thus, appearances of such language herein do not necessarily all refer to the same embodiment. On the other hand, such references are not necessarily mutually exclusive. Moreover, particular features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that the various embodiments shown in the drawings are merely illustrative representations and are not necessarily drawn to scale.

[0018] For the sake of clarity, some details describing structure or processing that are well known and often associated with robotic systems and subsystems, but that may unnecessarily obscure some important aspects of the disclosed technology, are not included in the following description. Also, while the following disclosure describes some embodiments of different aspects of the technology, some other embodiments may have different configurations or different components than those described in this section. Thus, the disclosed technology may have other embodiments that have additional elements and that do not have some of the elements described below.

[0019] Many embodiments or aspects of the disclosure described below may take the form of computer-executable or processor-executable instructions, including routines executed by a programmable computer or processor. Those skilled in the art will appreciate that the disclosed technology may be practiced on computer or processor systems other than those illustrated and described below. The technology described herein may be implemented in a special-purpose computer or data processor that is specifically programmed, configured, or constructed to execute one or more of the computer-executable instructions described below. Thus, the terms "computer" and "processor" as generally used herein refer to any data processor, including Internet appliances and handheld devices (e.g., palmtop computers, wearable computers, cellular or mobile telephones, multiprocessor systems, processor-based or programmable consumer electronics, network computers, minicomputers, etc.). Information manipulated by these computers and processors may be presented on any suitable display medium, such as a liquid crystal display (LCD). Instructions for performing computer-executable or processor-executable tasks may be stored in or on any suitable computer-readable medium, including hardware, firmware, or a combination of hardware and firmware. The instructions may be contained in any suitable memory device, including, for example, a flash drive and / or other suitable medium.

[0020] The terms "coupled" and "connected," as well as derivatives thereof, may be used herein to describe structural relationships between components. It should be understood that these terms are not intended to be synonyms for each other. Rather, in certain embodiments, "connected" may be used to indicate that two or more elements are in direct contact with each other. Unless otherwise indicated by the context, the term "coupled" may be used to indicate that two or more elements are in direct or indirect contact with each other (through other intervening elements therebetween), or that two or more elements cooperate or interact with each other (e.g., in a causal relationship for sending and receiving signals or calling functions, etc.), or both.

[0021] Favourable environment 1 is a diagram of an example environment in which a robotic system 100 having a dynamic packing mechanism may operate. The robotic system 100 may include and / or communicate with one or more units (e.g., robots) configured to perform one or more tasks. Aspects of the dynamic packing mechanism may be practiced or implemented by various units.

[0022] In the example shown in FIG. 1 , the robotic system 100 can include an unloading unit 102, a transposition unit 104 (e.g., a palletizing robot and / or a piece picker robot), a transport unit 106, a loading unit 108, or a combination thereof, in a warehouse or distribution / shipping depot. Each unit in the robotic system 100 can be configured to perform one or more tasks. Tasks can be combined in sequence to perform actions to achieve a goal, such as unloading an object from a truck or van and storing it in a warehouse, or unloading an object from a storage location and preparing it for shipment. In some embodiments, a task can include placing an object at a goal location (e.g., on a pallet and / or in a bin / cage / box / case). As described in more detail below, the robotic system 100 can derive individual placement positions / or orientations, calculate corresponding motion plans, or a combination thereof, to place and / or stack the objects. Each unit can be configured to perform a sequence of actions (e.g., operate one or more components therein) to perform the task.

[0023] In some embodiments, a task may include manipulating (e.g., moving and / or reorienting) a target object 112 (e.g., one of a package, box, case, cage, pallet, etc. corresponding to the task being performed) from a start / source location 114 to a task / destination location 116. For example, the unloading unit 102 (e.g., a devanning robot) may be configured to transpose a target object 112 from a position in a carrier (e.g., a truck) to a position on a conveyor belt. Also, the transposition unit 104 may be configured to transpose a target object 112 from one position (e.g., a conveyor belt, a pallet, or a bin) to another position (e.g., a pallet, a bin, etc.). In other examples, the transposition unit 104 (e.g., a palletizing robot) may be configured to transpose a target object 112 from a source location (e.g., a pallet, a pick-up area, and / or a conveyor) to a destination pallet. Upon completing an operation, the transport unit 106 may transpose the target object 112 from an area associated with the displacement unit 104 to an area associated with the loading unit 108, and the loading unit 108 may transpose the target object 112 from the displacement unit 104 (e.g., by moving a pallet carrying the target object 112) to a storage location (e.g., a location on a shelf). More details regarding the tasks and associated actions are provided below.

[0024] For illustrative purposes, the robotic system 100 is described in the context of a shipping center, however, it should be understood that the robotic system 100 can be configured to perform tasks in other environments / purposes, e.g., for manufacturing, assembly, packaging, healthcare, and / or other types of automation. It should also be understood that the robotic system 100 can include other units not shown in FIG. 1 , e.g., manipulators, service robots, modular robots, etc. For example, in some embodiments, the robotic system 100 can include a depalletizing unit for transposing objects from a cage cart or pallet onto a conveyor or other pallet, a container switching unit for transposing objects from one container to another, a packaging unit for packaging objects, a sorting unit for grouping objects according to their one or more characteristics, a piece picking unit for manipulating (e.g., to sort, group, and / or transpose) objects differently according to their one or more characteristics, or combinations thereof.

[0025] Preferred system 2 is a block diagram illustrating a robotic system 100 according to one or more embodiments of the present technology. In some embodiments, for example, the robotic system 100 (e.g., in one or more of the units and / or robots described above) can include electronic / electrical devices, such as one or more processors 202, one or more storage devices 204, one or more communication devices 206, one or more input / output devices 208, one or more actuation devices 212, one or more transport motors 214, one or more sensors 216, or combinations thereof. The various devices can be coupled to each other via wired and / or wireless connections. For example, the robotic system 100 can include a bus, such as, for example, a system bus, a Peripheral Component Interconnect (PCI) bus or PCI-Express bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a Small Computer System Interface (SCSI) bus, a Universal Serial Bus (USB), an IIC (I2C) bus, or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (also called "Firewire"). Also, for example, the robotic system 100 may include bridges, adapters, processors, or other signal-related devices to provide wired connections between devices. Wireless connections may be based, for example, on cellular communication protocols (e.g., 3G, 4G, LTE, 5G, etc.), wireless local area network (LAN) protocols (e.g., Wireless Fidelity (Wi-Fi)), peer-to-peer or device-to-device communication protocols (e.g., Bluetooth, Near Field Communication (NFC), etc.), Internet of Things (IoT) protocols (e.g., NB-IoT, LTE-M, etc.), and / or other wireless communication protocols.

[0026] The processor 202 may include a data processor (e.g., a central processing unit (CPU), a special purpose computer, and / or an on-board server) configured to execute instructions (e.g., software instructions) stored in a storage device 204 (e.g., a computer memory). In some embodiments, the processor 202 may be included in an independent / stand-alone controller that is operably coupled to other electronic / electrical devices shown in FIG. 2 and / or the robotic unit shown in FIG. 1. The processor 202 may control / interface other devices to implement program instructions that cause the robotic system 100 to perform actions, tasks, and / or operations.

[0027] The storage device 204 may include a non-transitory computer-readable medium on which program instructions (e.g., software) are stored. Some examples of the storage device 204 may include volatile memory (e.g., cache and / or random access memory (RAM)) and / or non-volatile memory (e.g., flash memory and / or magnetic disk drives). Other examples of the storage device 204 may include portable memory and / or cloud storage devices.

[0028] In some embodiments, the storage device 204 can be used to further store and provide access to processing results and / or predefined data / thresholds. For example, the storage device 204 can store master data 252 including descriptions of objects (e.g., boxes, cases, and / or products) that may be manipulated by the robotic system 100. In one or more embodiments, the master data 252 can include dimensions, shapes (e.g., templates for possible poses and / or computer-generated models for recognizing objects in different poses), color schemes, images, identification information (e.g., barcodes, Quick Response (QR) codes, logos, etc., and / or their expected locations), expected weights, other physical / visual characteristics, or combinations thereof, for the objects assumed to be manipulated by the robotic system 100. In some embodiments, the master data 252 can include operation-related information for the objects, such as the center of gravity (CoM) location of each object, expected sensor measurements (e.g., for force, torque, pressure, and / or contact measurements) corresponding to one or more actions / procedures, or combinations thereof. Also, for example, the storage device 204 may store object tracking data 254. In some embodiments, the object tracking data 254 may include a log of objects being scanned or manipulated. In some embodiments, the object tracking data 254 may include imaging data (e.g., photographs, point clouds, live video feeds, etc.) of an object at one or more locations (e.g., designated pick-up or drop-off locations and / or a conveyor belt). In some embodiments, the object tracking data 254 may include a position and / or orientation of an object at one or more locations.

[0029] The communication device 206 may include circuitry configured to communicate with external or remote devices over a network. For example, the communication device 206 may include a receiver, a transmitter, a modulator / demodulator (modem), a signal detector, a signal coder / decoder, a connector port, a network card, etc. The communication device 206 may be configured to transmit, receive, and / or process electrical signals according to one or more communication protocols (e.g., Internet Protocol (IP), wireless communication protocols, etc.). In some embodiments, the robotic system 100 may use the communication device 206 to exchange information between units of the robotic system 100 and / or to exchange information with systems or devices external to the robotic system 100 (e.g., for reporting, data collection, analysis, and / or troubleshooting purposes).

[0030] The input / output devices 208 can include user interface devices configured to communicate and / or receive information to a human operator. For example, the input / output devices 208 can include a display 210 and / or other output devices (e.g., speakers, haptic circuitry, or tactile feedback devices, etc.) for communicating information to a human operator. The input / output devices 208 can also include control or receiving devices, such as a keyboard, mouse, touch screen, microphone, user interface (UI) sensors (e.g., cameras for receiving motion commands), wearable input devices, etc. In some embodiments, the robotic system 100 can use the input / output devices 208 to interact with a human operator in performing actions, tasks, operations, or combinations thereof.

[0031] The robotic system 100 may include physical or structural members (e.g., robotic manipulator arms) connected at joints for movement (e.g., rotational and / or translational displacement). The structural members and joints may form a kinematic chain configured to manipulate an end effector (e.g., gripper) configured to perform one or more tasks (e.g., grasping, rotating, joining, etc.) depending on the application / operation of the robotic system 100. The robotic system 100 may include actuation devices 212 (e.g., motors, actuators, wires, artificial muscles, electroactive polymers, etc.) configured to drive or manipulate (e.g., displace and / or reorient) the structural members around or at the corresponding joints. In some embodiments, the robotic system 100 may include a transport motor 214 configured to transport a corresponding unit / chassis from one location to another.

[0032] The robotic system 100 may include sensors 216 configured to obtain information used to perform tasks, for example, to manipulate structural members and / or transport the robotic unit. The sensors 216 may include devices configured to detect or measure one or more physical characteristics of the robotic system 100 (e.g., the state, condition, and / or position of one or more structural members / joints thereof) and / or the surrounding environment. Some examples of the sensors 216 may include accelerometers, gyroscopes, force sensors, strain gauges, tactile sensors, torque sensors, position encoders, etc.

[0033] In some embodiments, for example, the sensor 216 can include one or more imaging devices 222 (e.g., visual and / or infrared cameras, 2D and / or 3D imaging cameras, distance measuring devices, e.g., lidar or radar, etc.) configured to detect the surrounding environment. The imaging devices 222 can generate a representation of the detected environment, such as a digital image and / or a point cloud, that can be processed via machine / computer vision (e.g., for automated inspection, robotic guidance, or other robotic applications). As described in more detail below, the robotic system 100 (e.g., via the processor 202) can process the digital image and / or point cloud to identify the target object 112 of FIG. 1, the start position 114 of FIG. 1, the task position 116 of FIG. 1, the pose of the target object 112, a confidence metric for the start position 114 and / or pose, or a combination thereof.

[0034] To manipulate the target object 112, the robotic system 100 (e.g., via various circuits / devices described above) can capture and analyze image data of a designated area (e.g., a pick-up location, such as in a truck or on a conveyor belt) to identify the target object 112 and its start location 114. Similarly, the robotic system 100 can capture and analyze image data of other designated areas (e.g., a drop-off location for placing the object on a conveyor, a location for placing the object in a container, or a location on a pallet for stacking purposes) to identify a task location 116. For example, the imaging device 222 can include one or more cameras configured to generate image data of the pick-up area and / or one or more cameras configured to generate image data of the task area (e.g., a drop-off area). Based on the image data, as described below, the robotic system 100 can determine the start location 114, the task location 116, associated poses, packing / placing locations, and / or other processing results. More details regarding the dynamic packing algorithm are provided below.

[0035] In some embodiments, for example, the sensors 216 may include position sensors 224 (e.g., position encoders, potentiometers, etc.) configured to detect the position of structural members (e.g., robotic arms and / or end effectors) and / or corresponding joints of the robotic system 100. The robotic system 100 may use the position sensors 224 to track the positions and / or orientations of the structural members and / or joints during the performance of a task.

[0036] Discretized Model 3A and 3B are diagrams of discretized data used to plan and pack an object in accordance with one or more embodiments of the present technology, with FIG 3A showing a discretized object and FIG 3B showing a discretized packing table for packing the object.

[0037] In some embodiments, the robotic system 100 of FIG. 1 may include a predefined discretized model / representation of the intended object stored in the master data 252 of FIG. 2. In some embodiments, the robotic system 100 (e.g., via the processor 202 of FIG. 2) may dynamically generate the discretized model by mapping continuous faces / edges of real-world objects (e.g., packages, pallets, and / or other objects relevant to the task) to discrete counterparts (e.g., unit length and / or unit area). For example, the robotic system 100 may discretize image data (e.g., top view images and / or point cloud data) of the target object 112 and / or the pallet top surface captured by one or more imaging devices 222 of FIG. 2. In other words, the robotic system 100 may discretize image data of the start position 114 of FIG. 1, a position on the conveyor prior to the start position 114, and / or the task position 116 of FIG. 1. The robotic system 100 may discretize based on identifying a perimeter of the object / palette in the image data and then dividing the area within the perimeter according to a unit dimension / area. In some embodiments, the unit dimension / area may be scaled or mapped to the image data based on the size and / or location of the object / palette relative to the imaging device 222 according to a coordinate array and / or a predefined adjustment factor / formula.

[0038] 3A, some embodiments of the robotic system 100 can use a discretized object model 302 to plan / derive a placement location for an object (e.g., target object 112). The discretized object model 302 (shown in dotted lines) can represent the physical dimensions, shapes, edges, faces, or combinations thereof (shown in dashed lines) of an arriving or incoming object (e.g., package, box, case, etc.) according to discretized units (e.g., unit length). The discretized object model 302 can represent expected / known objects and / or unexpected / unknown objects that have been imaged and discretized as described above.

[0039] As shown in FIG. 3B , some embodiments of the robotic system 100 can use one or more discretized bed models 304 (e.g., discretized representations of the task location 116 in FIG. 1 ) to plan / derive stacking placement of objects. The discretized bed model 304 can represent a placement area 340 (e.g., physical dimensions, shape, or combinations thereof, of the task location 116, such as the top surface of the task location 116, the top surface of a package placed thereon, or combinations thereof) according to a discretized unit. In one or more embodiments, the discretized bed model 304 can represent a real-time situation of the placement area 340, such as through real-time updates. For example, with respect to a top view, the discretized bed model 304 can initially represent the top surface of a pallet, the interior bottom surface of a bin or box, etc., that receives and directly contacts the object. As the robotic system 100 places the object, the placement area 340 can change to include the top surface of the placed package (e.g., to stack the package), and the discretized bed model 304 can be updated to reflect the change.

[0040] In some embodiments, the discretized bed model 304 can be based on a top view of one or more standard size pallets (e.g., 1.1 m by 1.1 m pallets). The discretized bed model 304 can thus correspond to a pixelated 2D representation of a placement area along a horizontal plane (e.g., xy plane) that conforms to a grid scheme utilized by the robotic system 100. In some embodiments, the discretized object model 302 can include a top view (e.g., xy plane) of an expected or arriving object. The discretized object model 302 can thus correspond to a pixelated 2D representation of an object.

[0041] The discretization units used to generate the discretized model may include lengths set by a system operator, a system designer, a predetermined input / setting, an order, or a combination thereof. In some embodiments, the robotic system 100 may use unit pixels 310 (e.g., polygons, e.g., quadrilaterals, having one or more dimensions according to the discretization units) to describe the area / surface of the target object (e.g., via the discretized object model 302) and the platform / surface (e.g., via the discretized platform model 304). Thus, the robotic system 100 may pixelate the object and platform in 2D along the xy axis. In some embodiments, the size of the unit pixels 310 (e.g., the discretization units) may vary depending on the object dimensions and / or the platform dimensions. Also, the size of the unit pixels 310 may be adjusted (e.g., via pre-set rules / formulas and / or operator selection) to balance the required resources (e.g., computation time, required memory, etc.) with the packing accuracy. For example, as the size of the unit pixels 310 decreases, the computation time and packing accuracy may increase. Thus, by discretizing the packing task (e.g., target package and packing platform) using adjustable unit pixels 310, the flexibility for palletizing packages is increased. The robotic system 100 can control the balance between computational resources / time and packing accuracy depending on real-time requirements, patterns, and / or environment.

[0042] In some embodiments, the robotic system 100 can include instances of unit pixels 310 that only partially overlap the object for the discretized object model 302, such that the unit pixels 310 extend beyond the actual perimeter of the object. In other embodiments, the robotic system 100 can exclude overlapping instances of unit pixels 310 from the discretized base model 304, such that the unit pixels 310 in the discretized object model 302 are covered by and / or contained within the actual perimeter of the base surface.

[0043] As an illustrative example, FIG. 3A shows a first model orientation 332 and a second model orientation 334 of a discretized object model representing the target object 112. In some embodiments, the robotic system 100 can rotate one of the discretized models (i.e., the one captured / stored as the first model orientation 332) a predetermined amount along the imaged plane. As shown in FIG. 3A, the robotic system 100 can rotate the discretized object model 302 90 degrees around a vertical axis (an axis extending in and out of or perpendicular to the plane of the figure) along a horizontal plane (e.g., a plane represented along the x-axis and y-axis) to obtain the second model orientation 334. The robotic system 100 can use the different orientations to test / evaluate the corresponding placement of the object.

[0044] Based on the discretized data / representation, the robotic system 100 can dynamically derive a placement location 350 for the target object 112. As shown in FIG. 3B, the robotic system 100 can dynamically derive the placement location 350 even after one or more objects (e.g., shown as objects with cross-hatched areas in FIG. 3B) are placed in the placement area 340. Also, the dynamic derivation of the placement location 350 can occur after / while the target object 112 is unloaded / shelfed, registered, scanned, imaged, or a combination thereof. For example, the robotic system 100 can dynamically derive the placement location 350 when the target object 112 is being transported (e.g., via a conveyor), after the imaging device 222 of FIG. 2 generates image data of the target object 112, or a combination thereof.

[0045] Dynamically deriving the placement location 350 of the object increases flexibility and reduces human labor associated with the shipping / packaging environment. Using discretized real-time image / depth maps of the object and pallet (i.e., including the placed object), the robotic system 100 can test and evaluate various placement locations and / or orientations. Thus, the robotic system 100 can still pack objects without human operator intervention even when the object is not recognizable (e.g., in the case of new / unexpected objects and / or computer vision errors), when the arrival sequence / order of the objects is unknown, and / or when unexpected events occur (e.g., lost and / or collision events).

[0046] For illustrative purposes, the placement location 350 is shown in FIG. 3B as adjacent to the placed object (i.e., placed on the same horizontal layer / at the same height), e.g., directly above / in contact with the pallet. However, it should be understood that the placement location 350 may be above the placed object. In other words, the robotic system 100 may derive a placement location 350 for stacking the target object 112 above and / or on one or more objects already on the pallet. As described in more detail below, the robotic system 100 may evaluate the height of the placed object when deriving the placement location 350 so that the object is adequately supported when stacked on top of the placed object.

[0047] In some embodiments, the robotic system 100 can identify object edges 362 when deriving the placement location 350. The object edges 362 can include lines in the image data that represent edges and / or sides of the object that has been placed on the pallet. In some embodiments, the object edges 362 can define the perimeter of one or a group of objects (e.g., a layer of objects) placed on the task location 116 by corresponding to edges that are exposed (e.g., not directly in contact with / adjacent to other objects / edges).

[0048] As described in further detail below, the robotic system 100 can derive the placement location 350 according to a set of placement rules, conditions, parameters, requirements, etc. In some embodiments, the robotic system 100 can derive the placement location 350 based on evaluating / testing one or more candidate locations 360. The candidate locations 360 can correspond to the discretized object model 302 overlaid on the discretized bed model 304 in various positions and / or orientations. Thus, the candidate locations 360 can include placing the target object 112 adjacent to one or more of the object edges 362 and / or stacking the target object 112 on one or more of the placed objects. The robotic system 100 can evaluate each candidate location 360 according to various parameters / conditions, such as support degree / condition, support weight versus fragility rating of the supporting object (e.g., maximum support weight for packages stacked on top, etc.), space / packing concerns, or a combination thereof. The robotic system 100 can further evaluate the candidate locations 360 using one or more placement rules, such as collision-free requirements, stack stability, customer-specified rules / priorities, package spacing requirements or lack thereof, maximizing the total loaded packages, or a combination thereof.

[0049] Real-time surface updates 4A and 4B illustrate various aspects of support calculations and support metrics according to one or more embodiments of the present technology. In some embodiments, as shown in FIG. 4A, the robotic system 100 of FIG. 1 can generate a candidate position 360 of FIG. 3B based on overlaying the discretized object model 302 of FIG. 3A of the target object 112 of FIG. 1 on the discretized bed model 304 of the task location 116 of FIG. 1. Furthermore, the robotic system 100 can iteratively move the discretized object model 302 throughout the discretized bed model 304 in generating the candidate position 360. For example, the robotic system 100 can generate an initial instance of the candidate position 360 by placing the discretized object model 302 corresponding to a predetermined initial position (e.g., a corner) of the discretized bed model 304 according to one or more orientations (e.g., the first model orientation of FIG. 3A and / or the second model orientation 334 of FIG. 3A). For the next instance of the candidate position 360, the robotic system 100 may move the discretized object model 302 corresponding to the other / next object a predetermined distance (e.g., one or more unit pixels 310 in FIG. 3B) according to a predetermined direction / pattern.

[0050] If the candidate position 360 overlaps one or more objects already placed at the task location 116, the robotic system 100 can calculate and evaluate the degree of support provided by the placed objects. To calculate and evaluate the degree of support, the robotic system 100 can determine the height / contour for the placement area 340 of FIG. 3B in real time using one or more of the imaging devices 222 of FIG. 2. In some embodiments, the robotic system 100 can use a depth dimension (e.g., point cloud value) from one or more of the imaging devices 222 positioned above the task location 116. In some embodiments, the robotic system 100 can have a predefined height / position value that corresponds to the vertical position of the ground and / or platform (e.g., pallet) surface, e.g., the height of the platform surface above the facility ground. In some embodiments, the robotic system 100 can use the depth dimension to calculate the height / contour of the exposed top surface(s) of the platform, the placed object, or a combination thereof. In some embodiments, the robotic system 100 can image the task location 116 and update the height of the exposed top surface(s) in real time, for example, after transposing an object to and / or placing it on the platform.

[0051] 4A, the robotic system 100 can update the discretized bed model 304 to include a height dimension 402. The robotic system 100 can determine the height dimension 402 according to each discretized pixel (e.g., unit pixel 310) in the discretized bed model 304. For example, the robotic system 100 can determine the height dimension as the maximum height for the surface portion of the mounting area 340 represented by the corresponding unit pixel 310.

[0052] For each candidate location 360 that overlaps one or more of the placed objects, the robotic system 100 can evaluate the placement probability based on the height dimension 402. In some embodiments, the robotic system 100 can evaluate the placement probability based on identifying the maximum of the overlapping height dimensions 402 at each candidate location 360. The robotic system 100 can further identify other height dimensions 402 located at each candidate location 360 that have a height dimension 402 within a difference threshold limit to the maximum one of the height dimensions 402. The eligible cells / pixels can represent locations that can provide support for the stacked objects such that the stacked objects are essentially flat / level.

[0053] 4A, for the first of the candidate locations 360 (top left corner of the discretization mount model 304), the maximum height dimension may be 0.3 (i.e., 300 millimeters (mm) tall). With a predetermined difference threshold as 0.02 (e.g., representing 20 mm), the robotic system 100 may identify the top four discretized cells / pixels as meeting the difference threshold. The robotic system 100 may use the identified / qualified cells / pixels to evaluate / represent the degree of support.

[0054] 4B illustrates a further example of support calculation. FIG. 4B illustrates one of the candidate locations 360 of FIG. 3 with the discretized object model 302 (shown in solid bold outline) overlaid on the top left corner of the discretized base model 304. The robotic system 100 can calculate / utilize various support parameters 410, which are parameters used to evaluate the candidate locations 360. For example, the support parameters 410 can include a discretization dimension 412, an overlap area 414, a height difference threshold 416, a support threshold 418, a maximum height 420, a height lower limit 422, a qualifying number 424, a set of support area contours 426, a support area size 428, a support ratio 430, a center of mass (CoM) location 432, or a combination thereof.

[0055] The discretized dimensions 412 may describe the physical dimensions (e.g., length, width, height, perimeter, etc.) of the target object 112 of FIG. 1 according to the unit pixels 310 of FIG. 3A. For example, the discretized dimensions 412 may include a quantity of unit pixels 310 that form the perimeter of the discretized object model 302. The overlap area 414 may describe the area (e.g., footprint size along a horizontal plane) occupied by the target object 112, which may be similarly expressed according to the unit pixels 310. In other words, the overlap area 414 may correspond to the quantity of unit pixels 310 in the discretized object model 302. In the example shown in FIG. 4B, the target object 112 may have a discretized dimension 412 of 6 pixels by 7 pixels, which corresponds to an overlap area 414 of 42 pixels.

[0056] The height difference threshold 416 and the support threshold 418 may correspond to limits used to process and / or validate the candidate locations 360. The height difference threshold 416, which may be pre-determined and / or adjusted by the operator and / or order, may represent an acceptable deviation from the height of another reference (e.g., a maximum height 420 corresponding to the largest instance of the height dimension 402 within the overlapping area of ​​the discretized object models 302) to contact and / or support a package placed thereon. In other words, the height difference threshold 416 may be used to define a range of heights of a surface that may contact and / or support a package placed thereon. Thus, relative to the maximum height 420, the lower height limit 422 may correspond to a lower limit of a height within the overlapping area 414 that may provide support for a stacked package. In the example shown in FIG. 4B, the height difference threshold 416 may be 0.02. If the maximum height 420 is 0.2, the lower height limit may be 0.18. Thus, when placing the target object 112 at the candidate location 360, the robotic system 100 may estimate that faces / pixels having a height greater than 0.18 will contact and / or provide support to the target object 112.

[0057] Thus, in one or more embodiments, the robotic system 100 can categorize the unit pixels 310 in the overlap area 414 according to the height difference threshold 416. For example, the robotic system 100 can categorize the unit pixels 310 having heights that meet the height difference threshold 416 (i.e., values ​​equal to or greater than the lower height limit 422) as support positions 422 (e.g., a group of unit pixels 310 representing surfaces on which objects can be stacked, represented by the shaded pixels in FIG. 4B). The robotic system 100 can categorize the other unit pixels 310 as ineligible positions 444 (e.g., pixels having heights less than the lower height limit 422).

[0058] The support threshold 418 may represent a limit for evaluating the candidate locations 360 based on the sufficiency of the support locations 442. For example, the support threshold 418 may be for evaluating a quantity, a ratio, an area, a location, or a combination thereof, associated with the support locations 442. In some embodiments, the support threshold 418 may be used to determine whether the qualification number 424 for the candidate locations 360 (e.g., the sum of the support locations 442) is sufficient to support the target object 112.

[0059] In one or more embodiments, the support threshold 418 can be used to evaluate a support area (e.g., a unit pixel 360 that can provide support to an object stacked above, which can be determined by a height threshold) associated with the support location 442. For example, the robotic system 100 can determine the support area contour 426 based on connecting corners of the outermost / perimeter instances of the support location 442 by extending edges and / or determining lines that span or extend around the ineligible locations 444. In this manner, the support area contour 426 can exclude the ineligible locations 444. Thus, the support area contour 426 can define a perimeter of the support area based on the perimeter instances of the support location 442. Because the support area contour 426 can span and / or include the ineligible locations 444, the support area size 428 (e.g., a quantity of unit pixels 310 within the support area) can be greater than the eligible number 424. Thus, the support area size 428 effectively represents the spacing between the outermost edges / corners where support is provided. Because wider support is preferred (e.g., when a portion of the support area contour 426 is larger than the object overlap area 414 to reduce protrusion and / or improve stability), the support threshold 418 may correspond to a minimum number of unit pixels 310 within the support area (e.g., for evaluating the support area contour 426), thereby effectively evaluating the spacing between the outermost edges / corners where support is provided.

[0060] In some embodiments, the support threshold 418 can be for evaluating a support ratio 430, which can be calculated based on comparing the eligible number 424 and / or the support area size 428 to the overlap area 414. For example, the support ratio 430 can include a ratio of the eligible number 424 to the overlap area 414 to represent horizontal stability, support weight concentration, or a combination thereof. The support ratio 430 can also include a ratio of the support area size 428 to the overlap area 414 to represent the relative width between the support edges / corners under the target object 112.

[0061] Additionally, the robotic system 100 can further evaluate the candidate positions 360 based on the CoM position 432 of the target object 112. In some embodiments, the robotic system 100 can access the CoM position 432 of the target object 112 from the master data 252 of FIG. 2 and / or dynamically estimate the CoM position 432 based on grasping and / or lifting the target object 112. Once accessed / estimated, the robotic system 100 can compare the CoM position 432 to the support area contour 426. The robotic system 100 can require the candidate positions 360 to include the CoM position 432 within the support area contour 426 and eliminate / disqualify candidate positions 360 that do not meet such requirements. In one or more embodiments, the robotic system 100 can calculate and evaluate a placement score based on a separation distance (e.g., along the x-axis and / or y-axis) between the CoM position 432 and the support area contour 426.

[0062] The robotic system 100 can use the support parameters 410 to evaluate the constraints / requirements. For example, the robotic system 100 can eliminate / disqualify candidate locations that do not satisfy the support threshold 418, the CoM position threshold (e.g., the requirement to include the CoM position 432 within the support area contour 426), and / or other stacking rules. The robotic system 100 can also use the support parameters 410 to calculate placement scores for the candidate locations 360 (e.g., locations that satisfy the constraints) according to predefined weights and / or formulas. As described in more detail below, the robotic system 100 can use the calculated placement scores to rank the candidate locations 360 according to predefined settings (e.g., as reflected by the weights / formulas).

[0063] Object placement action 5 is a top view illustrating an exemplary placement performed by the robotic system 100 in accordance with one or more embodiments of the present disclosure. In some embodiments, the robotic system 100 can include and / or communicate with a robotic arm 502 (e.g., part of the transposition unit 104 of FIG. 1 , such as a palletizing robot) configured to transpose the target object 112 from the start location 114 and place it at a derived placement location 350 at the task location 116. For example, the robotic system 100 can operate the robotic arm 502 to grasp and pick up the target object 112 from a designated location / portion on a conveyor and place the target object 112 on a pallet.

[0064] The robotic system 100 can dynamically derive the placement location 350, for example, when the target object 112 arrives at the facility and / or start location 114 and / or after initially starting one or more operations, such as a packing operation. The robotic system 100 can dynamically derive the placement location 350 based on or taking into account one or more uncertainty factors, for example, the absence of a packing plan (e.g., a plan to represent the placement location 350 derived for a set of objects including the target object 112 at the task location 116), an error in the arriving object (e.g., if the object does not match an expected / known object or sequence), or a combination thereof. The robotic system 100 can also dynamically derive the placement location 350 based on or taking into account one or more uncertainties in the destination, for example, due to the placed objects 508 (e.g., unrecognized and / or unexpected packages on a pallet) and / or misalignment of one or more of the placed objects 508.

[0065] In some embodiments, the robotic system 100 can dynamically derive the placement location 350 based on data (e.g., image data and / or measurement data) dynamically collected via one or more of the sensors 216 (e.g., the imaging device 222 of FIG. 2 ) of FIG. 2 . For example, the robotic system 100 can include and / or communicate with a source sensor 504 (e.g., a 3D camera) located above the start location 114 and / or the incoming path (e.g., a conveyor). The robotic system 100 can use data from the source sensor 504 to generate and / or access the discretized object model 302 of FIG. 3A . In one or more embodiments, the robotic system 100 can use the source sensor 504 to image an object and / or measure one or more dimensions of the object. The robotic system 100 can compare the image and / or measurement to the master data 252 of FIG. 2 to identify the incoming object. Based on the identification, the robotic system 100 can access the discretized object model 302 associated with the object. In one or more embodiments, the robotic system 100 can dynamically generate the discretized object model 302 based on dividing the image / dimension according to unit pixels 310 as described above.

[0066] The robotic system 100 may also include and / or communicate with a destination sensor 506 (e.g., a 3D camera) positioned above the task location 116. The robotic system 100 may use data from the destination sensor 506 to determine and dynamically update the discretized bed model 304 of FIG. 3B. In one or more embodiments, the robotic system 100 may image the placement area (e.g., the task location 116, such as a pallet) and / or measure one or more dimensions thereof. The robotic system 100 may use the images and / or measurements to identify, access, and / or generate the discretized bed model 304, similar to as described above for the discretized object model 302. Additionally, the robotic system 100 may use data (e.g., a depth map) from the destination sensor 506 to determine the height dimension 402 of FIG. 4A. Thus, the robotic system 100 can use the height dimension 402 to update the placement area 340 and the discretized bed model 304 in real time. For example, the robotic system 100 can update the height dimension 402 in response to the placed object 508, e.g., after placing the target object 112 at the placement location 350.

[0067] The robotic system 100 can derive an entry path 510 for displacing the target object 112 to the placement location 350 and / or each candidate location 360 of FIG. 3B. The entry path 510 can correspond to a motion plan for manipulating / displacing the target object 112 across space from the starting location 114 to the corresponding candidate location 360. The entry path 510 can be 3D, extending horizontally and / or vertically.

[0068] Approach route evaluation 6A and 6B are side views illustrating an example approach for placing the target object 112 of FIG. 1 in accordance with one or more embodiments of the present disclosure. 6A and 6B show the approach path 510 of FIG. 5 for placing the target object 112 on one of the placed objects 508 on the task location 116 (e.g., a pallet) at the corresponding candidate location 360 of FIG. 3B.

[0069] The robotic system 100 of FIG. 1 can derive an entry path 510 based on entry units 602 illustrated by dashed boxes F-1 to F-5. The entry units 602 can include sequential positions of the target object 112 in 3D space along the corresponding entry path 510. In other words, the entry units 602 can correspond to sample positions of the target object 112 for following the corresponding entry path 510. The entry units 602 can be positioned according to path segments 604 of the corresponding entry path 510. The path segments 604 can correspond to straight line segments / directions in the entry path 510. The path segments 604 can include a final segment 606 for placing the target object 112 at the corresponding candidate position 360. The final segment 606 can include a vertical (e.g., downward) direction.

[0070] To derive the entry path 510, the robotic system 100 can identify any of the placed objects 508 that may be obstacles 610 (e.g., potential obstacles when placing the target object 112 at the candidate location 360, etc.). In one or more embodiments, the robotic system 100 can identify the potential obstacle(s) 610 as instance(s) of the placed object 508 that overlap with a horizontal line 611 (e.g., a straight line along the xy plane) connecting the start location 114 and the corresponding candidate location 360. The robotic system 100 can further identify the potential obstacle(s) 610 as instance(s) of the placed object 508 that overlap with a derived lane 613 near the horizontal line, for example, based on deriving a lane that is parallel to and overlaps with the horizontal line, the lane having a width based on one or more dimensions (e.g., width, length, and / or height) of the target object 112. 6A and 6B, the starting location 114 may be to the right of the candidate location 360. Thus, the robotic system 100 may identify the placed object to the right as a potential obstacle 610.

[0071] In some embodiments, the robotic system 100 can verify / identify a potential obstacle 610 based on the height dimension 402 of FIG. 4A. For example, the robotic system 100 can verify / identify a potential obstacle 610 by one or more of the height dimensions 402 equal to or greater than the candidate location 360. The robotic system 100 can eliminate a placed object 508 having a height dimension 402 less than the candidate location 360 as a potential obstacle 610. In one or more embodiments, the robotic system 100 can identify / eliminate a potential obstacle 610 based on ambiguity associated with the height of the candidate location 360 and / or the height of the potential obstacle 610.

[0072] In some embodiments, the robotic system 100 can derive the entry path 510 in reverse order, e.g., starting from the candidate position 360 and ending at the starting position 114 in FIG. 5 . Thus, the robotic system 100 can derive the final segment 606 first (e.g., before other segments) to avoid the potential obstacle 610. For example, the robotic system 100 can determine the entry unit 602 based on iteratively increasing the height of the entry unit 602 by a predetermined distance (e.g., first "F-1", then "F-2", etc.). For each iteration, the robotic system 100 can calculate and analyze the vector 612 between the determined entry unit 602 (e.g., bottom surface / its edge) and the potential obstacle 610 (e.g., top surface / its edge). The robotic system 100 may continue to increase the height of the entry unit 602 until the vector 612 indicates that the determined entry unit 602 is above the potential obstacle 610 and / or jumps over the potential obstacle 610 by a gap threshold 614 (e.g., a requirement for a minimum vertical spacing for the target object 112 above the highest point of the potential obstacle 610 to avoid contact or collision between the target object 112 and the potential obstacle 610). If the determined entry unit 602 meets the gap threshold 614, or for the next iteration, the robotic system 100 may adjust the corresponding entry unit 602 by a predetermined distance along the horizontal direction (e.g., toward the starting position 114). Thus, the robotic system 100 may derive the final segment 606 and / or subsequent path segment 604 based on the candidate position 360 and the entry unit 602 that met the gap threshold 614 to derive the entry path 510.

[0073] Once derived, the robotic system 100 can use the entry path 510 to evaluate the corresponding candidate location 360. In some embodiments, the robotic system 100 can calculate a placement score as a function of the entry path 510. For example, the robotic system 100 can calculate a placement score as a function of a setting for a shorter length / distance for the final / vertical segment 606 (e.g., as a function of one or more weights corresponding to a given placement setting). Thus, in comparing the entry paths 510 of FIG. 6A and FIG. 6B, the robotic system 100 can prioritize the path shown in FIG. 6B, in which the length of the final / vertical segment 606 is shorter. In one or more embodiments, the robotic system 100 can include constraints, such as upper limits, associated with the entry path 510 (e.g., for the final / vertical segment 606) used to eliminate or disqualify the candidate location 360.

[0074] In some embodiments, the robotic system 100 may further evaluate the corresponding candidate location 360 according to other collision / obstacle related parameters. For example, the robotic system 100 may evaluate the candidate location 360 according to the horizontal spacing 616 between the candidate location 360 and one or more of the placed objects 508. Each horizontal spacing 616 may be a distance (e.g., a shortest distance) along a horizontal direction (e.g., an xy plane) between the corresponding candidate location 360 and an adjacent instance of the placed object 508. The robotic system 100 may calculate a placement score for the candidate location 360 based on the horizontal spacing 616, similar to that described above for the entry path 510. The robotic system 100 may also eliminate or disqualify the candidate location 360 based on the horizontal spacing 616, such as when the horizontal spacing 616 does not meet a minimum requirement. More details regarding the calculation of placement scores and / or constraints for eliminating the candidate location 360 are provided below.

[0075] Operational flow 7 is a flow diagram of a method 700 of operating the robotic system 100 of FIG. 1 in accordance with one or more embodiments of the present technology. The method 700 can be for dynamically deriving a placement location 350 of FIG. 3B on the task location 116 of FIG. 1 for placing the target object 112 of FIG. 1. The method 700 can be implemented based on instructions stored in one or more of the storage devices 204 of FIG. 2 being executed by one or more of the processors 202 of FIG. 2.

[0076] At block 702, the robotic system 100 can identify a real-time packaging situation. For example, as shown in block 732, the robotic system 100 can analyze the incoming object and / or the task location 116 of FIG. 1 in real time. The robotic system 100 can receive and analyze sensor data from the sensor 216 of FIG. 2. In some embodiments, the robotic system 100 can receive and analyze source sensor data (e.g., from the source sensor 504 of FIG. 5) representing the target object 112 of FIG. 1 at or approaching the start location 114 of FIG. 1. The robotic system 100 can also receive and analyze destination sensor data (e.g., from the destination sensor 506 of FIG. 5) representing a placement area associated with the task location 116 and / or the placed object 508 of FIG. 5 thereon (e.g., the placement area 340 of FIG. 3B thereon).

[0077] In some embodiments, the robotic system 100 can analyze the sensor data to determine one or more uncertainty factors, as shown in block 734. For example, the robotic system 100 can compare the sensor data to a pre-derived packing plan (e.g., via offline and / or real-time calculations at the facility) that specifies placement locations for incoming objects, including the target object 112. Thus, some instances of uncertainty can be based on a discrepancy between the sensor data and the packing plan or associated arrival sequence.

[0078] In analyzing the sensor data for uncertainty, the robotic system 100 may process the sensor data (e.g., images and / or depth maps) to identify / estimate edges, as shown in block 736. For example, the robotic system 100 may recognize edges of the target object 112, the task location 116, the placed object 508, or combinations thereof, by processing the sensor data using a Sobel filter, or the like. The robotic system 100 may use the edges to identify areas representing separate objects and / or their dimensions.

[0079] In some cases, the mismatch may include a source matching error resulting from comparing the source sensor data to the master data 252 of FIG. 2, the access / arrival sequence associated with the packing plan, or a combination thereof. A source matching error may result from, for example, a misidentification of an incoming object (e.g., the source sensor data does not correspond to any object in the packing plan and / or the master data 252) and / or an incoming object being out of order and not matching the expected arrival / access sequence for the packing plan. A mismatch may also include a destination matching error resulting from comparing the destination sensor data to the packing plan. A destination matching error may result from, for example, one or more of the placed objects 508 being in an unexpected (i.e., not matching the packing plan) location due to a misaligned package, etc. Other examples may include a container at the task location 116 not being open enough to receive the package and / or containing an unexpected item upon arrival.

[0080] In one or more embodiments, the robotic system 100 can determine the uncertainty based on additional triggers. For example, the robotic system 100 can determine the uncertainty based on the absence of a packing plan. Also, for example, the robotic system 100 can determine the uncertainty based on an operation status or event, such as a collision event (e.g., if the robotic unit and / or the object collide), an object loss event (e.g., if the object falls during transportation / manipulation), an object displacement event (e.g., if the object displaces after placement), or a combination thereof. As described in more detail below, the robotic system 100 can dynamically derive the placement position 350 of FIG. 3 for the target object 112 in response to and / or to compensate for the uncertainty.

[0081] In block 704, the robotic system 100 may generate and / or access a discretized model (e.g., the discretized object model 302 of FIG. 3A and / or the discretized platform model 304 of FIG. 3B) that represents the incoming package (including the target object, etc.) and / or the task location 116, e.g., a pallet and / or a cage.

[0082] The robotic system 100 can determine (e.g., generate and / or access) a discretized model (e.g., discretized object model 302 and / or discretized table model 304) based on real-time sensor data (e.g., source sensor data and / or destination sensor data). In some embodiments, the robotic system 100 can identify an object type (e.g., an identity or category of an incoming object) of an object, such as the target object 112, based on the source sensor data. The robotic system 100 can search the master data 252 to match the imaged surface with a surface image corresponding to the object type when checking for uncertainty as described above. In some embodiments, the robotic system 100 can also estimate one or more dimensions or lengths of a sensed object (e.g., an incoming object, a target object 112, a pallet, a cage, etc.) based on the sensor data (e.g., source sensor data) when checking for uncertainty. The robotic system 100 can use the identification information to access discretized models stored in the storage device of Figure 2 and / or other devices (e.g., a package supplier's storage device, database, and / or server accessed via the communication device 206 of Figure 2). For example, the robotic system 100 can find and access a matching discretized model by searching the master data 252 using the identification information (e.g., a surface image and / or estimated dimensions).

[0083] In some embodiments, the robotic system 100 can generate the discretized model in real time, e.g., immediately in response to receiving source sensor data and / or determining uncertainties. To dynamically generate the discretized model, the robotic system 100 can divide the sensor data and / or corresponding physical dimensions (e.g., of an incoming object, a pallet top, etc.) according to the unit pixels 310 of FIG. 3B. In other words, the robotic system 100 can generate the discretized model based on overlaying the unit pixels 310 on areas representing the target object 112 and / or the task location 116 according to the corresponding sensor data. The unit pixels 310 can be predetermined (e.g., by the manufacturer, the ordering customer, and / or the operator), e.g., 1 mm or 1 / 16 inch (in) or more (e.g., 5 mm or 20 mm). In some embodiments, the unit pixels 310 can be based on the dimensions or size of one or more of the packages and / or the platform (e.g., a percentage or fraction).

[0084] In block 706, the robotic system 100 may derive a set of candidate positions (e.g., the candidate positions 360 in FIG. 3B ) for placing the target object 112 at / above the task location 116. The robotic system 100 may derive the candidate positions 360 based on overlaying the discretized object model 302 of the target object 112 onto the discretized bed model 304 at corresponding positions in / above the task location 116. The candidate positions 360 may correspond to positions of the discretized object model 302 above / in the discretized bed model 304 along a horizontal plane. The robotic system 100 may derive the candidate positions 360 that overlap and / or are adjacent to the placed object 508.

[0085] In some embodiments, the robotic system 100 can iteratively determine the position of the discretized object model 302 based on determining the initial placement position (e.g., a predefined position of an instance of the candidate position 360, e.g., a specified corner of the placement area). The robotic system 100 can determine the subsequent candidate positions 360 according to a predefined direction for deriving the next candidate position 360, a spacing requirement between the candidate positions 360 over the iterations, rules / conditions governing placement, a limit on the total number of candidate positions 360, one or more patterns thereof, or a combination thereof. Furthermore, the robotic system 100 can include a set of settings and / or rules for determining the candidate positions 360 for the placed object 508. For example, the robotic system 100 can be configured with a setting (e.g., to perform a function sooner than most other types / categories of candidate positions 360) for determining the candidate positions 360 where the discretized object model 302 is adjacent to or abuts one or more edges of the placed object 508 and / or a peripheral boundary / perimeter of the placement area 340. The robotic system 100 can also be configured with settings to determine candidate positions 360 where the discretized object model 302 is above the placed object 508, fits within one of the objects, and / or overlaps with one or more edges of the object.

[0086] The robotic system 100 can derive the candidate positions 360 according to a predefined rule, pattern, constraint, and / or sequence for placing the discretized object model 302. For example, the robotic system 100 can derive the candidate positions 360 based on a preference for an object edge that is adjacent to and / or within a predefined distance constraint, for example, an outermost edge of the placed object 508. The robotic system 100 can also derive the candidate positions 360 based on a preference for an outer edge / boundary of the placement area 340 where the discretized object model 302 is closest to or abuts the boundary / edge of a pallet, cage, etc. The robotic system 100 can also derive the candidate positions 360 that overlap the placed object 508.

[0087] In block 708, the robotic system 100 can determine / update a real-time status of the placement area 340 of FIG. 3B, such as for evaluation of 3D stacking. For example, the robotic system 100 can use the destination sensor data to determine the height dimension 402 of FIG. 4. The robotic system 100 can use the depth dimension derived from the destination sensor data and the known height of the task location 116 and / or the sensor to calculate the height of the top surface(s) at the task location 116. The robotic system 100 can match the calculated height with the unit pixels 310 in the discretized bed model 304 and assign the maximum calculated height in the unit pixel 310 as the corresponding height dimension 402. In some embodiments, the robotic system 100 can determine the height dimension 402 for the unit pixels 310 that the discretized object model 302 in the candidate location 360 overlaps.

[0088] In block 710, the robotic system 100 may evaluate the candidate locations 360. In some embodiments, the robotic system 100 may evaluate the candidate locations 360 according to real-time conditions, processing results, predefined rules and / or parameters, or a combination thereof. For example, the robotic system 100 may evaluate the candidate locations 360 based on calculating a corresponding placement score, validating / qualifying the candidate locations 360, or a combination thereof.

[0089] At block 742, the robotic system 100 may calculate a placement score for each candidate location 360. The robotic system 100 may calculate the placement score according to one or more of the placement conditions. For example, the robotic system 100 may use placement settings (e.g., via multiplier weights) and / or formulas to describe settings for package-to-package separation distance, package dimension / fragility rating / package weight difference for horizontally adjacent packages, collision probability (e.g., based on the approach path 510 in FIG. 5 or its characteristics and / or the horizontal spacing 616 in FIG. 6), consecutive / adjacent surfaces at the same height, their statistical results (e.g., average, maximum, minimum, standard deviation, etc.), or combinations thereof. Other examples of placement settings may include result height, proximity, edge-placed status, maximum supportable weight, object type, support weight ratio, or combinations thereof. Thus, in some embodiments, the robotic system 100 may include processing weights / multipliers that represent settings for a lower maximum height, for placing the target object 112 near the boundary of an already placed object or the edge of a placement platform, for minimizing the difference in height and / or maximum supportable weight of adjacent objects, for lowering the ratio of support weight to maximum supportable weight for objects that the target object 112 overlaps, for matching object types of adjacent objects, or combinations thereof. Each placement location may be scored according to preference coefficients and / or formulas predefined by the system manufacturer, order, and / or system operator.

[0090] In some embodiments, for example, the robotic system 100 can calculate a placement score based on the degree of support for the candidate locations 360. The robotic system 100 can calculate the amount of support (e.g., in stacking objects) for one or more of the candidate locations 360 based at least in part on the height dimension 402. As an illustrative example, the robotic system 100 can calculate the amount of support based on identifying the maximum height 420 of FIG. 4B for each candidate location 360. Based on the maximum height 420 of FIG. 4B and the height difference threshold 416, the robotic system 100 can calculate the lower height limit 422 of FIG. 4B for each candidate location 360. The robotic system 100 can compare the height dimension 402 of the candidate location 360 to the corresponding lower height limit 422 to identify the support location 442 of FIG. 4B for each candidate location 360. The robotic system 100 can calculate a placement score for each candidate location 360 based on the eligible number 424 of FIG. 4B of the corresponding support location 442.

[0091] In one or more embodiments, the robotic system 100 can calculate a placement score based on deriving the support area contours 426 of FIG. 4B for the candidate locations 360. As described above, the robotic system 100 can derive a set of support area contours 426 for each candidate location 360 based on extending the outer edges and / or joining corners of the outermost / perimeter instances of the support locations 442 in the corresponding locations. Based on the support locations 442, the robotic system 100 can determine the support area size 428 of FIG. 4B and / or the support ratio 430 of FIG. 4B to calculate the placement score. The robotic system 100 can also calculate a minimum separation distance between the CoM location 432 and the support area contours 426. The robotic system 100 can use the support area size 428, the support ratio 430, the minimum separation distance, the corresponding preference weight, or a combination thereof to calculate a placement score for the corresponding candidate location.

[0092] In one or more embodiments, the robotic system 100 can calculate a placement score based on deriving the entry path 510 for the candidate location 360 as described above. The robotic system 100 can calculate a placement score for each candidate location 360 depending on the final segment 606 in Figure 6 (e.g., its length), the quantity / length of one or more path segments 604 in Figure 6, or a combination thereof. In some embodiments, the robotic system 100 can calculate a placement score based on the horizontal spacing 616 in Figure 6 for the candidate location 360.

[0093] In some embodiments, as shown in block 744, the robotic system 100 can qualify the candidate positions 360. The robotic system 100 can qualify the candidate positions 360 based on dynamically deriving a verified set of the candidate positions 360 according to one or more placement constraints. In deriving the verified set, the robotic system 100 can eliminate or disqualify instances of the candidate positions 360 that violate or do not satisfy one or more of the placement constraints related at least in part to the height dimension 402. In one or more embodiments, the robotic system 100 can first derive the verified set and then calculate a placement score for the verified set. In one or more embodiments, the robotic system 100 can derive the verified set simultaneously with the calculation of the placement score.

[0094] In one or more embodiments, the placement constraints may involve comparing the eligible numbers 424, the set of support area contours 426, the support area size 428, the support ratio 430, the CoM location 432, the approach path 510, the horizontal spacing 616, or a combination thereof, to a threshold (e.g., the support threshold 418 in FIG. 4B ) or requirement. For example, the robotic system 100 may derive the verified set to include locations where the eligible numbers 424, the support area size 428, and / or the support ratio 430 meet / exceed the corresponding thresholds. The robotic system 100 may also derive the verified set to include locations where the CoM location 432 is within / enclosed by the support area contours 426 and / or meets a minimum separation distance from the support area contours 426. The robotic system 100 may also derive the verified set to include locations where the approach path 510 (e.g., the final segment 606 therein) meets a maximum length threshold and / or the horizontal spacing 616 meets a minimum threshold.

[0095] In block 712, the robotic system 100 can dynamically derive a placement location 350 for placing the target object 112 above / on the task location 116. The robotic system 100 can dynamically derive the placement location 350 based on selecting one of the positions in the validated set or the candidate positions 360 according to the placement score. In some embodiments, the robotic system 100 can track the candidate positions 360 using a heap structure. Thus, the robotic system 100 can remove a position from the heap structure if it violates the above-mentioned constraints. Furthermore, the robotic system 100 can order or rank the tracked positions according to the corresponding placement score. In some embodiments, the robotic system 100 can continuously order the tracked positions when the placement score or iterative updates of the score are being calculated. As a result, the robotic system 100 can select a position at a designated position (e.g., the first slot) in the heap structure as the placement position 350 when the score calculation is finished.

[0096] At block 714, the robotic system 100 may place the target object 112 at the derived placement location 350. In placing the target object 112 at the placement location 350, one or more components / devices of the robotic system 100 may communicate with and / or operate other components / devices. For example, the processor 202 and / or one or more of a stand-alone controller (e.g., a control device of a warehouse / shipping center, etc.) may send information to the other components / devices, such as, for example, the placement location 350, a corresponding motion plan, a set of commands and / or settings for operating the actuation devices 212 of FIG. 2 and / or the transport motors 214 of FIG. 2, or a combination thereof. Other components / devices, such as other instances of the processor 202, and / or the robotic arm 502, actuation device 212, transport motor 214, and / or other external devices / systems of FIG. 5, can receive the information and perform corresponding functions to manipulate (e.g., grasp and pick up, transpose and / or reorient through space, place at a destination, and / or release) the target object 112 and place it in a placement position.

[0097] In some embodiments, the robotic system 100 may update or re-identify the real-time packaging status after placing the target object 112. In other words, following block 714, the control flow may move to block 702. Thus, the robotic system 100 may update / identify the next incoming object as the target object 112. The robotic system 100 may also update information about the placement area 340 and / or the placed objects 508 thereon to include the recently placed object. In other embodiments, the robotic system 100 may recalculate or adjust the packing plan and / or resume following the packing plan after placing the target object 112.

[0098] The above-described task discretization and 2D / 3D layering increases the efficiency, speed, and accuracy of packing objects. Thus, the human effort for the automated packing process can be further reduced by reducing operator input and increasing accuracy. In some environments, the above-described robotic system 100 can eliminate the need for sequence buffers, which can cost approximately US$1 million or more.

[0099] Furthermore, by dynamically calculating the placement position 350 according to real-time conditions (e.g., as represented by sensor data and other status / data), motion errors are reduced. As described above, the robotic system can consider and resolve uncertainties introduced by unexpected conditions / events without requiring human intervention. Also, the above-described embodiments can stack objects in 3D without a pre-existing packing plan, such as by dynamically deriving the placement position 350 when the object arrives at the starting position 114. By considering height, the above-described embodiments can stack objects on top of each other and improve packing density, as compared to conventional systems that are limited to 2D dynamic packing (i.e., placing objects directly on the pedestal as a single layer).

[0100] conclusion The above detailed description of the embodiments of the disclosed technology is not exhaustive and is not intended to limit the disclosed technology to the precise form disclosed above. Although specific examples of the disclosed technology are described above for illustrative purposes, those skilled in the art will recognize that various equivalent modifications are possible within the scope of the disclosed technology. For example, although processes or blocks are presented in a given order, alternative embodiments may perform routines having steps in a different order or employ systems having blocks in a different order, and some processes or blocks may be removed, moved, added, subdivided, combined, and / or modified to provide alternative combinations or subcombinations. Each of these processes or blocks may be implemented in a variety of different ways. Also, although processes or blocks may be shown to be performed in a sequence, these processes or blocks may instead be performed or implemented in parallel, or may be performed at different times. Furthermore, any specific numbers described herein are merely examples, and alternative embodiments may employ different values ​​or ranges.

[0101] These and other changes can be made to the disclosed technology in light of the above detailed description. Although the detailed description describes the best mode contemplated as well as specific examples of the disclosed technology, no matter how detailed the above description may appear in text, the disclosed technology can be practiced in many ways. The details of the system may vary widely in its particular embodiment and still be encompassed by the technology disclosed herein. As mentioned above, a particular term used in describing a particular feature or aspect of the disclosed technology should not be construed to mean that the term is redefined herein to be limited to any particular characteristic, feature, or aspect of the disclosed technology to which the term relates. Thus, the present invention is not limited except as set forth in the appended claims. In general, the terms used in the following claims should not be construed to limit the disclosed technology to the specific examples disclosed herein, unless the above detailed description section expressly defines such terms.

[0102] Although certain aspects of the invention are presented below in particular claim forms, Applicant contemplates various aspects of the invention in any number of claim forms, and accordingly, Applicant reserves the right to pursue additional claims after filing this application to pursue such additional claim forms in this application or any continuing application.

Claims

1. 1. A method of operating a robotic system, comprising: determining a two-dimensional discretized object model representing physical dimensions of the target object according to a discretization unit, the determining the two-dimensional discretized object model comprising overlaying the discretization unit on an area representing the target object obtained from image data of the target object; determining a two-dimensional discretization bed model representing a physical dimension of a placement area for the target object according to the discretization unit, wherein determining the two-dimensional discretization bed model comprises overlaying the discretization unit on an area representing the placement area obtained from image data of the placement area; deriving candidate locations for placing the target object by overlaying the two-dimensional discretized object model on the two-dimensional discretized base model; transmitting, to the robot system, information for placing the target object at a placement position within the placement area, the placement position being based on at least the candidate positions; A method comprising:

2. The method of claim 1 , further comprising dynamically deriving the placement position in response to one or more uncertainty factors.

3. the one or more uncertainty factors relate to a lack of a packing plan; The method of claim 2 , wherein the packing plan is for representing placement positions derived for a set of objects, including the target object, in the placement area.

4. the one or more uncertainty factors relate to one or more deviations from a packing plan; The method of claim 2 , wherein the packing plan is for representing placement positions derived for a set of objects, including the target object, in the placement area.

5. 5. The method of claim 4, wherein the one or more uncertainty factors are related to a source matching error resulting from comparing at least one feature of the target object obtained from source sensor data of the target object with corresponding information of at least one object included in master data, a sequence associated with the packing plan, or a combination thereof.

6. The method of claim 4 , wherein the one or more uncertainty factors relate to a destination matching error resulting from comparing destination sensor data for the loading area to the packing plan.

7. The method of claim 4 , wherein the one or more uncertainty factors relate to a collision event, a lost object event, a misaligned object event, or a combination thereof.

8. Determining the two-dimensional discretized object model comprises: identifying an object type that identifies the target object based on source sensor data of the target object; accessing the two-dimensional discretized object model based on retrieving master data according to the object type; The method of claim 1 , comprising:

9. Determining the two-dimensional discretized object model comprises: estimating one or more lengths representative of one or more dimensions of the target object based on source sensor data of the target object; accessing the two-dimensional discretized object model based on retrieving master data according to the one or more lengths; The method of claim 1 , comprising:

10. 2. The method of claim 1 , wherein determining the two-dimensional discretized object model includes generating the two-dimensional discretized object model in real-time in response to capturing or receiving source sensor data of the target object.

11. determining a height dimension representing a height of the portion of the placing area according to the discretization unit; The method of claim 1 , wherein determining the height dimension comprises determining the height dimension for the discretized unit over which the two-dimensional discretized object model overlaps for one or more candidate locations.

12. determining a height dimension representing a height of the portion of the placing area according to the discretization unit; identifying a maximum height for the candidate location, the maximum height representing a largest instance of the height dimension for the candidate location; calculating a lower height limit as the maximum height minus a height difference threshold representing an acceptable deviation, the lower height limit corresponding to the lowest allowed height dimension for a position within the placement area that is considered to provide support for the target object; and identifying support locations based on comparing the height dimension to the lower height limit, the support locations representing surfaces within the placement area onto which the target object can be stacked, the support locations including instances of the discretization units whose height dimension meets and / or exceeds the lower height limit. The method of claim 1.

13. dynamically validating the candidate locations according to one or more placement constraints associated with the height dimension; and calculating at least one placement score for the verified candidate locations; The method of claim 12 , wherein dynamically validating the candidate locations and / or calculating the at least one placement score comprises evaluating the candidate locations as a function of the support location.

14. Dynamically validating the candidate locations includes: calculating a qualifying number representative of a quantity of said support locations; dynamically validating the candidate locations based on comparing the eligible number for the candidate locations to a threshold; The method of claim 13, comprising:

15. Dynamically validating the candidate locations includes: deriving a support area contour based on edges and / or corners of the outermost instances of the support locations; calculating a support area size based on the support area contour, the support area size representing a quantity of the discretization units within the support area contour; dynamically validating the candidate locations based on comparing the support area size for the candidate locations to a threshold; The method of claim 13, comprising:

16. Dynamically validating the candidate locations includes: deriving a support area contour based on edges and / or corners of the outermost instances of the support locations; comparing a center of mass (CoM) location associated with the two-dimensional discretized object model to the support area contour; The method of claim 13, comprising:

17. Dynamically validating the candidate locations includes: The method of claim 13 , further comprising deriving an entry path for placing the target object at the candidate location.

18. Calculating the at least one placement score comprises: calculating a first placement score for the candidate location based on a qualifying number representing a quantity of the support locations, a support area size representing a quantity of the discretized units within a support area contour that is based on edges and / or corners of the outermost instances of the support locations, a center of mass (CoM) location associated with the two-dimensional discretized object model, an approach path for placing the target object at the candidate location, or a combination thereof; and / or calculating a second placement score for the candidate location based on a resultant height representing a height dimension at the candidate location after stacking the target object at the candidate location, a proximity representing a dimension between a boundary of the target object and a placed object, an edge-placed status indicating a relationship between the target object at the candidate location and a placement platform on which the object is stacked, a maximum supportable weight of placed objects adjacent to the candidate location, an object type representing a category of objects adjacent to the candidate location, a support weight ratio representing a ratio of a weight supported by objects overlapping the target object at the candidate location to a maximum support weight for those objects, or a combination thereof; The method of claim 13, comprising:

19. At least one processor; at least one memory device coupled to the at least one processor and storing instructions executable by the at least one processor; The instruction: determining a two-dimensional discretized object model representing physical dimensions of the target object according to a discretization unit, the determining the two-dimensional discretized object model comprising overlaying the discretization unit on an area representing the target object obtained from image data of the target object; determining a two-dimensional discretized mount model representing physical dimensions of a destination area for the target object according to the discretized units, wherein determining the two-dimensional discretized mount model includes overlaying the discretized units on an area representing the destination area obtained from image data of the destination area; deriving candidate locations for placing the target object in the destination area by overlaying the two-dimensional discretized object model on the two-dimensional discretized platform model; A robot system comprising:

20. 1. A tangible, non-transitory computer readable medium having stored thereon processor instructions that, when executed by a robotic system via one or more processors thereof, cause the robotic system to perform a method, comprising: The processor instructions are: instructions for determining a two-dimensional discretized object model representing physical dimensions of a target object according to a discretization unit, the determining the two-dimensional discretized object model comprising overlaying the discretization unit onto an area representing the target object obtained from image data of the target object; instructions for determining a two-dimensional discretized mount model representing a physical dimension of a task position according to the discretized unit, the determining the two-dimensional discretized mount model comprising overlaying the discretized unit on an area representative of the task position obtained from image data of the task position; instructions for deriving candidate locations for placing the target object by overlaying the two-dimensional discretized object model on the two-dimensional discretized base model; an instruction to place the target object at a placement location within the task location, the placement location being based on at least the candidate locations; 4. A tangible, non-transitory computer readable medium comprising:

Citation Information

Patent Citations

  • Three-dimensional object recognizing apparatus based on image data

    JP1996114416A

  • Control device for articulated robot and its control method

    JP1998080881A

  • Hand form recognition device and monochromatic object form recognition device

    JP1999167455A

  • Full automatic mixed-stowing method, and system terefor

    JP2000168958A

  • Method and apparatus for palletizing package of random size and weight

    JP2003335417A