Robot system having robot arm suction control mechanism and operation method thereof

Abstract

To provide an improved method for controlling and managing a robot's ability to grip and handle objects.SOLUTION: A system and method of operation of a robotic system including: receiving a sensor reading associated with a target object; generating a base plan for performing a task on the target object, wherein generating the base plan includes determining a grip point for gripping the target object and a grip pattern based on a location of the grip point relative to a designated area, a task location, and another target object; implementing the base plan by operating an actuation unit and a suction gripper according to a grip pattern rank, to generate an established grip on the target object, wherein the established grip is at a grip pattern location; measuring the established grip; comparing the established grip with a force threshold; and re-gripping the target object on the basis of the established grip falling below the force threshold.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001]

[0001] Embodiments of the present invention relate generally to robotic systems, and more specifically to robotic systems having a robotic arm suction control mechanism. [Background technology]

[0002]

[0002] Robots are being used extensively in many fields due to their ever-improving performance and decreasing cost. For example, robots can be used to perform a variety of tasks, including maneuvering or transporting objects from one place to another. Such tasks are particularly useful in manufacturing, assembly, packing, packaging, warehousing, and transportation. In performing these tasks, robots can replicate human actions, thereby replacing or reducing human involvement that would otherwise require a human to perform dangerous or repetitive tasks. However, despite technological advances, robots still lack the sophistication required to replicate the human sensitivity, adaptability, and dexterity required to perform more complex tasks. For example, robotic hands or grippers often have difficulty grasping objects with relatively soft or irregular surfaces due to insufficient sensitivity of contact sensors or insufficient granularity of force control.

[0003]

[0003] Thus, there is room for improvement in the technology for controlling and managing the performance of robots to grip and handle objects. Finding answers to these problems is becoming increasingly important, given the ever-increasing competitive commercial pressures, coupled with rising consumer expectations and fewer opportunities for meaningful product differentiation in the marketplace. Furthermore, the need to reduce costs, improve efficiency and performance, and meet competitive pressures makes the critical need to find answers to these problems even more urgent.

[0004]

[0004] Solutions to these problems have long been sought, but prior developments have not taught or suggested solutions, and have long eluded those skilled in the art. Summary of the Invention

[0005]

[0005] One embodiment of the present invention provides a method of operation of a robotic system, the method including receiving sensor readings associated with a target object; generating a base plan for performing a task on the target object, where generating the base plan includes determining a grip point and one or more grip patterns associated with the grip point for gripping the target object based on a designated area, a work location, and a location of the grip point relative to another target object; implementing the base plan for performing the task by operating an actuation unit and one or more suction grippers in accordance with a grip pattern rank to generate an established grip on the target object, where the established grip is at a grip pattern location associated with the grip pattern; measuring the established grip; comparing the established grip to a force threshold; and re-gripping the target object based on the established grip falling below the threshold.

[0006]

[0006] One embodiment of the present invention provides a robotic system comprising: a communications unit configured to receive sensor readings associated with a target object; and a control unit connected to the communications unit, wherein the control unit is configured to generate a base plan for performing an operation on the target object, wherein generating the base plan includes determining a grip point and one or more grip patterns associated with the grip point for gripping the target object based on a designated area, a work location, and a location of the grip point relative to another target object, wherein the control unit is configured to implement the base plan for performing the operation by operating the actuation unit and one or more suction grippers in accordance with a grip pattern rank to generate an established grip on the target object, wherein the established grip is at a grip pattern location associated with the grip pattern, and wherein the control unit is configured to measure the established grip, compare the established grip to a force threshold, and re-grip the target object based on the established grip falling below the threshold.

[0007]

[0007] One embodiment of the present invention provides a non-transitory computer readable medium including instructions for a robotic system comprising: receiving sensor readings associated with a target object; generating a base plan for performing a task on the target object, where generating the base plan includes determining a grip point and one or more grip patterns associated with the grip point for gripping the target object based on a designated area, a work location, and a location of the grip point relative to another target object; implementing the base plan for performing the task by operating an actuation unit and one or more suction grippers in accordance with a grip pattern rank to generate an established grip on the target object, where the established grip is at a grip pattern location associated with the grip pattern; measuring the established grip; comparing the established grip to a force threshold; and re-gripping the target object based on the established grip falling below the threshold.

[0008]

[0008] Certain embodiments of the invention have other steps or elements in addition to, or instead of, those steps or elements described above, which will become apparent to those skilled in the art from a reading of the following detailed description in conjunction with the accompanying drawings. [Brief description of the drawings]

[0009] [Figure 1]

[0009] An exemplary environment in which a robotic system having a robotic arm suction control mechanism can operate. [Diagram 2]

[0010] FIG. 1 is an exemplary block diagram of components of a robotic system. [Diagram 3]

[0011] 1 is an example of an arm unit according to one or more embodiments of the present invention. [Figure 4]

[0012] 4 is an exemplary control flow of a motion stage of a robotic system in one embodiment of the present invention. [Diagram 5]

[0013] 13 is an exemplary control flow of a suction control pattern generation mechanism in one embodiment of the present invention. [Figure 6]

[0014] 1 is an example of mapping grip points to grip patterns in accordance with one or more embodiments of the present invention. [Figure 7]

[0015] 1 is an exemplary control flow of an online state of a robot system in one embodiment of the present invention. [Figure 8]

[0016] 1 is a flowchart of a method of operating a robotic system in one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010]

[0017] The following embodiments are described in sufficient detail to enable those skilled in the art to make and use the invention, but it is understood that other embodiments will be apparent based on this disclosure, and that system, process, or mechanical changes may be made without departing from the scope of the embodiments of the invention.

[0011]

[0018] In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent that the present invention can be practiced without some of these specific details. Detailed disclosure of some well-known structures, circuits, system configurations, and process steps is omitted to avoid obscuring the embodiments of the present invention.

[0012]

[0019] The drawings showing the embodiments of the system and method are semi-schematic and not to scale. Some dimensions are exaggerated in the drawings for clarity of presentation. Similarly, some figures in the drawings are typically shown in similar orientations for ease of explanation, but the depictions in the drawings are largely arbitrary. In general, the invention can operate in any orientation. The embodiments are numbered for convenience of explanation, such as one embodiment, second embodiment, etc., but are not intended to have any other meaning or to impose limitations on the embodiments of the invention.

[0013]

[0020] The term "module" or "unit" as used herein may include software, hardware, mechanical mechanisms, or combinations thereof in embodiments of the invention, depending on the context in which the term is used. For example, software may be machine code, firmware, embedded code, or application software. For example, hardware may be a circuit, a processor, a special-purpose computer, an integrated circuit, an integrated circuit core, a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), a passive element, or combinations thereof. Furthermore, mechanical mechanisms may include actuators, motors, arms, joints, handles, end effectors, guides, mirrors, anchor bases, vacuum lines, vacuum generators, light source lines, or stoppers. Furthermore, when a "module" or a "unit" is described in the system claims section below, such a "module" or "unit" is deemed to include a hardware circuit for the purpose and scope of the system claims.

[0014]

[0021] In the following description of the embodiments, modules or units may be connected or attached to each other as described or illustrated. Such connection or attachment may be indirect, including an intermediary between the connected or attached modules or units, or may be direct, including no intermediary. Also, such connection or attachment may be by physical contact or by communication between the modules or units.

[0015]

[0022] 1, an exemplary environment in which a robotic system 100 having a robotic arm suction control mechanism may operate is shown. The robotic system 100 may include one or more structures or units, such as robots or robotic devices, configured to perform one or more operations 118. Aspects of the robotic arm suction control mechanism may be realized or implemented by a variety of structures. In one embodiment, the robotic system 100 may include an arm unit 102, a transfer unit 104, a transport unit 106, a loading unit 108, or a combination thereof, within a warehouse, distribution center, or shipping hub.

[0016]

[0023] The robotic system 100 or a portion of the robotic system 100 may be configured to perform operations 118. An operation 118 is a function performed or implemented by the robotic system 100 due to a physical transformation on the arm unit 102, the transfer unit 104, the transport unit 106, the loading unit 108, or a combination thereof. For example, an operation 118 may include moving a target object 112 from a location, such as a container, bin, cage, basket, shelf, platform, pallet, or conveyor belt, to another location based on a physical transformation on the arm unit 102, the transfer unit 104, the transport unit 106, the loading unit 108, or a combination thereof. Multiple operations 118 may be sequentially combined to perform actions to achieve a goal, including loading or unloading a target object 112.

[0017]

[0024] The target object 112 is an item that is to be or has been handled by the robotic system 100. For example, the target object 112 may include a box, a case, a rigid body, a semi-rigid body, an item with a flexible surface, or a combination thereof. As another example, the operation 118 may cause the robotic system 100 to load or unload the target object 112 onto or from a vehicle, such as a truck, trailer, van, or train car, for storage in a warehouse, or to unload the target object 112 from a storage location and load it onto a vehicle for shipping. The parts of the robotic system 100 may be configured to perform a series of actions, such as operating one or more components of the robotic system 100, to perform the operation 118. As an example, the parts of the robotic system 100 may be configured independently, individually, or separately with respect to each other. Also, as an example, the parts of the robotic system 100 may be configured together as a group in a coordinated manner, in a sequential manner, or a combination thereof. FIG. 1 shows examples of possible functions and operations that may be performed by various units of the robotic system 100 when handling a target object 112, it being understood that the environments and conditions may differ from those described below.

[0018]

[0025] The arm unit 102 may be a robotic arm configured to handle the target object 112. The arm unit 102 may be used as a vehicle unloading robot configured to transfer the target object 112 from one location to another. As an example, the arm unit 102 may be a piece-picking robot configured to transfer the target object 112 from one container to another. In another example, the arm unit 102 may be, for example, a palletizing robot. As an example, the arm unit 102 may transfer the target object 112 from one location in a carrier, such as a truck or container, to a location on a conveyor belt. Also, for example, the arm unit 102 may be used to load the target object 112 onto a carrier. Further details of the arm unit 102 are described below.

[0019]

[0026] The transport unit 104 may be a stationary part of a mechanical handling device that moves the target object 112 from one location to another (e.g., a conveyor belt). As an example, the transport unit 104 may move the target object 112 on a conveyor belt to a location on the transport unit 106, such as for loading the target object 112 onto a pallet on the transport unit 106.

[0020]

[0027] The transport unit 106 may be a mobile robot used to transport the target object 112 from an area associated with the transfer unit 104 to an area associated with the loading unit 108. The transport unit 106 may be, for example, a transport robot, a delivery drone, a delivery robot, a forklift, or a combination thereof.

[0021]

[0028] The loading unit 108 may be configured to transport the target objects 112, such as by moving a pallet holding the target objects 112 from the transport unit 106 to an additional storage location, such as a location on one or more shelves. The loading unit 108 may be, for example, a freight elevator, a warehouse elevator, a cargo lift, or a combination thereof.

[0022]

[0029] As a further example, the operation 118 includes transporting the target object 112 from one or more designated areas 114 to the work location 116. For example, the designated area 114 may include a container for storing the target object 112, such as a cage, bin, box, pallet, or combinations thereof. The designated area 114 may include numerous configurations and forms. For example, the designated area 114 may be a platform with or without walls, such as a pallet, shelf, or conveyor belt, on which the target object 112 may be placed or stacked. As another example, the designated area 114 may be a partially or fully enclosed container with walls or a lid, such as a bin, cage, or basket, within which the target object 112 may be placed. In some embodiments, the walls of the designated area 114, which is a partially or fully enclosed container, may be transparent or have openings or gaps of various sizes such that portions of the target object 112 contained therein may be visible or partially visible through the walls. The work location 116 may be an area in which a target object 112 is placed to have an operation 118 performed, or an area designated as an end or start point for an operation 118 to be performed on a target object 112 .

[0023]

[0030] For illustrative purposes, the robotic system 100 is described in the context of a shipping center, but it should be understood that the robotic system 100 may be configured to perform the operations 118 in other environments or for other purposes. By way of example, the robotic system 100 may operate in environments for manufacturing, assembly, packaging, healthcare, or other types of automation. It should also be understood that the robotic system 100 may include other units, such as manipulators, service robots, modular robots, etc., not shown in FIG. 1 . For example, in some embodiments, the robotic system 100 may include a depalletizing unit for transferring the target objects 112 from a cage cart or pallet to a conveyor or other pallet, a container switching unit for transferring the target objects 112 from one container to another, a packaging unit for wrapping the target objects 112, a sorting unit for grouping the target objects 112 according to one or more characteristics of the target objects 112, a piece picking unit for maneuvering the target objects 112 in various ways (e.g., for sorting, grouping, or transport) according to one or more characteristics of the target objects 112, or a combination thereof.

[0024]

[0031] 2, an exemplary block diagram of components of the robotic system 100 is shown. In one embodiment, the robotic system 100 may include a control unit 202, a storage unit 206, a communication unit 212, a user interface 216, an actuation unit 220, and a sensor unit 230. In one embodiment, one or more of these components may be combined within an enclosure 234.

[0025]

[0032] The enclosure 234 may be a housing in which a portion of the robotic system 100 is housed. The enclosure 234 may separate the portion of the robotic system 100 housed therein from other portions that are outside the enclosure 234. For example, the enclosure 234 may be a case, a chassis, a box, a console, a computer tower, a computer motherboard. In one embodiment, for example, the control unit 202, the storage unit 206, the communication unit 212, or a combination thereof may be housed within the enclosure 234. In another embodiment, the control unit 202, the storage unit 206, the communication unit 212, or a combination thereof may be housed within the enclosure 234, and the user interface 216 may be accessible from outside the enclosure 234.

[0026]

[0033] One or more components of the robotic system 100 may be housed in or on the enclosure 234, while other components of the robotic system 100 may be external to the enclosure 234. For example, in one embodiment, the control unit 202, the storage unit 206, and the communication unit 212 may be housed within the enclosure 234, while the user interface 216, the actuation unit 220, the sensor unit 230, or a combination thereof are external to the enclosure 234. The above are merely examples of components that may be housed in or on the enclosure 234 and are not intended to be limiting. Other combinations of components may be housed within the enclosure 234.

[0027]

[0034] The control unit 202 can execute software 210 to provide the intelligence of the robotic system 100. The control unit 202 can also execute software 210 for other functions of the robotic system 100. The control unit 202 can be implemented in a variety of ways. For example, the control unit 202 can be a processor, an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), a digital signal processor (DSP), or a combination thereof.

[0028]

[0035] The control unit 202 may include a control interface 204. The control interface 204 may be used for communication between the control unit 202 and other functional units of the robotic system 100. The control interface 204 may also be used for communication outside the robotic system 100. The control interface 204 may receive information from other functional units or external sources, or may transmit information to other functional units or external destinations. External sources and external destinations refer to sources and destinations that are outside the robotic system 100.

[0029]

[0036] The control interface 204 may be implemented in different ways and may include different implementations depending on the functional or external units that it interfaces with. For example, the control interface 204 may be implemented with pressure sensors, inertial sensors, microelectromechanical systems (MEMS), optical circuits, waveguides, wireless circuits, wired circuits, application program interfaces, or combinations thereof.

[0030]

[0037] The storage unit 206 may store the software 210, the master data 226, the object tracking data 228, the configuration data 248, or a combination thereof. For purposes of illustration, the storage unit 206 is illustrated as a single element, but it should be understood that the storage unit 206 may be multiple storage elements distributed. Also, for purposes of illustration, the robotic system 100 is illustrated having the storage unit 206 as a single hierarchical storage system, but it should be understood that the robotic system 100 may have a storage unit 206 in a different configuration. For example, the storage unit 206 may be formed by different storage technologies forming a memory hierarchical system including different tiers of caching, main memory, rotating media, or offline storage.

[0031]

[0038] The storage unit 206 may be a volatile memory, a non-volatile memory, an internal memory, an external memory, or a combination thereof. For example, the storage unit 206 may be a non-volatile storage, such as a non-volatile random access memory (NVRAM), a flash memory, a disk storage, or a volatile storage, such as a static random access memory (SRAM).

[0032]

[0039] The storage unit 206 may include a storage interface 208. The storage interface 208 may be used for communication between the storage unit 206 and other functional units of the robotic system 100. The storage interface 208 may also be used for communication outside the robotic system 100. The storage interface 208 may receive information from other functional units of the robotic system 100 or from external sources, or may transmit information to other functional units of the robotic system 100 or to external destinations. External sources and external destinations refer to sources and destinations that are outside the robotic system 100.

[0033]

[0040] The storage interface 208 may include different implementations depending on the functional or external units that the storage unit 206 interfaces with. The storage interface 208 may be implemented in similar technologies and manners as the implementation of the control interface 204.

[0034]

[0041] In one embodiment, the storage unit 206 can be used to further store and provide access to the process results, data, thresholds, or combinations thereof. The process results, data, thresholds, or combinations thereof may constitute the master data 226. The master data 226 may include a description of the target object 112 of FIG. 1, such as, for example, a box, a box type, a case, a case type, a product, or combinations thereof. In one embodiment, the master data 226 may include the target object 112's dimensions, shape (e.g., a template for potential orientations of the target object 112 or a computer-generated model for recognizing the target object 112 in various orientations), color scheme, image, identification information (e.g., a barcode, a Quick Response (QR) code, a logo), predicted dimensions, predicted weight, or combinations thereof. In one embodiment, the master data 226 may further include maneuver-related information of the target object 112, such as a location of a center of mass 236 on the target object 112 and predicted sensor measurements (e.g., force, torque, pressure, or contact measurements) corresponding to one or more actions, maneuvers, or combinations thereof.

[0035]

[0042] In one embodiment, the storage unit 206 may further store configuration data 248. The configuration data 248 refers to parameter and initial setting information for one or more components of the robotic system 100 or one or more external components required to operate the one or more components of the robotic system 100 or one or more components external to the robotic system 100. For example, such configuration information may include information regarding which components to turn ON and / or OFF, when to activate the components (e.g., when to turn one or more components ON and / or OFF), or other setting variables, thresholds, or operational data required to operate the one or more components. In one embodiment, the configuration data 248 may be stored in a configuration file 250. The configuration file 250 refers to a computer file, such as a text file, that includes parameter and initial setting information.

[0036]

[0043] In one embodiment, the storage unit 206 may further store object tracking data 228. The object tracking data 228 may be data indicative of the location, position, status, or combinations thereof, of the target object 112. The object tracking data 228 may include a log of the target object 112 that has been scanned or steered. In some embodiments, the object tracking data 228 may include imaging data (e.g., photographs, point clouds / depth maps, live video feeds) of the target object 112 at one or more locations (e.g., designated pick-up or drop locations, or a conveyor belt). In some embodiments, the object tracking data 228 may include the position and orientation of the target object 112 at one or more locations.

[0037]

[0044] The communications unit 212 may enable communications to and from the robotic system 100, including communications between functional units of the robotic system 100, communications between external devices, or combinations thereof. For example, the communications unit 212 may enable the robotic system 100 to communicate with external devices, such as an external computer, an external database, an external machine, an external peripheral device, or combinations thereof, via the communications path 238.

[0038]

[0045] The communication paths 238 may span and represent a variety of networks and network topologies. For example, the communication paths 238 may include wireless communications, wired communications, optical communications, ultrasonic communications, or combinations thereof. For example, satellite communications, cellular communications, Bluetooth®, Infrared Data Association Standard, (IrDA)®, Wireless Fidelity (WiFi)®, and Worldwide Interoperability for Microwave (WiMAX)® are examples of wireless communications that may be included in the communication paths 238. Cable, Ethernet, Digital Subscriber Line (DSL), Fiber Optic Line, Fiber to the Home (FTTH), Plain Old Telephone Service (POTS) are examples of wired communications that may be included in the communication paths 238. Furthermore, the communication paths 238 may span a multitude of communication topologies and distances. For example, the communication paths 238 may include a direct connection, a personal area network (PAN), a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), or combinations thereof. The control unit 202 may further execute software 210 for interaction with a communication path 238 via a communication unit 212 .

[0039]

[0046] The communications unit 212 can also function as a communications hub, enabling the robotic system 100 to function as part of the communications path 238, and is not limited to being an endpoint or terminal device to the communications path 238. The communications unit 212 can include active and passive components, such as microelectronics or antennas, for interaction with the communications path 238.

[0040]

[0047] The communication unit 212 may include a communication interface 214. The communication interface 214 may be used for communication between the communication unit 212 and other functional units of the robotic system 100. The communication interface 214 may receive information from other functional units of the robotic system 100 or an external source, or may transmit information to other functional units of the robotic system 100 or an external destination. The communication interface 214 may include different implementations depending on the functional units that the communication unit 212 interfaces with. The communication interface 214 may be implemented in similar technologies and manners as the implementation of the control interface 204.

[0041]

[0048] The control unit 202 can operate the user interface 216 to present or receive information generated by the robotic system 100. The user interface 216 may include input devices and output devices. Examples of input devices of the user interface 216 may include a keypad, touchpad, soft keys, keyboard, microphone for providing data and communication input, a sensor for receiving remote signals, a camera for receiving motion commands, or any combination thereof. Examples of output devices may include a display interface 218 and an audio interface 232.

[0042]

[0049] The display interface 218 may be any graphical user interface, such as a display, a projector, a video screen, or any combination thereof. The audio interface 232 may include a speaker, a microphone, headphones, a subwoofer, a sound component, a transducer, or any combination thereof. The display interface 218 and the audio interface 232 enable a user of the robotic system 100 to interact with the robotic system 100. The display interface 218 and the audio interface 232 may be optional.

[0043]

[0050] The robotic system 100 may also include an actuation unit 220. The actuation unit 220 may include, for example, devices (e.g., motors, springs, gears, pulleys, chains, rails, wires, artificial muscles, electroactive polymers, or combinations thereof) configured to drive, steer, displace, orient, reorient, or combinations thereof, structural members or mechanical components of the robotic system 100 about or at corresponding mechanical joints. The control unit 202 may operate the actuation unit 220 to control or steer the actuation unit 220.

[0044]

[0051] The actuation unit 220 may include an actuation interface 222. The actuation interface 222 may be used for communication between the actuation unit 220 and other functional units of the robotic system 100. The actuation interface 222 may also be used for communication outside the robotic system 100. The actuation interface 222 may receive information from other functional units of the robotic system 100 or from an external source, or may transmit information to other functional units or to an external destination.

[0045]

[0052] The actuation interface 222 may include different implementations depending on the functional units or external units of the robotic system 100 that the actuation unit 222 interfaces with. The actuation interface 222 may be implemented in similar technologies and manners as the implementation of the control interface 204.

[0046]

[0053] The robotic system 100 may include a sensor unit 230 configured to obtain sensor readings 246 used to perform tasks 118 and operations, such as to steer a structural member. The sensor readings 246 may be information or data obtained by the sensor unit 230, whose purpose is to detect events or changes in the environment of the robotic system 100 and transmit the information to components of the robotic system 100, external devices, or a combination thereof to facilitate the tasks 118. The sensor readings 246 may include image readings, such as, for example, digital images or point clouds / depth maps. The sensor readings 246 may also include quantified measurements, such as measurements of force, torque, rotation, velocity, distance, or a combination thereof.

[0047]

[0054] The sensor unit 230 can comprise devices configured for detecting or measuring sensor readings 246. For example, the sensor unit 230 is configured to detect or measure one or more physical characteristics of the robotic system 100, such as a state, a condition, a location of one or more structural members or joints, information about an object or the surrounding environment, or a combination thereof. By way of example, the sensor unit 230 can comprise a variety of sensors including an imaging device 240, a system sensor 242, a contact sensor 244, or a combination thereof.

[0048]

[0055] In some embodiments, the sensor unit 230 may include one or more imaging devices 240. The imaging devices 240 are devices configured to capture, recognize, detect, or a combination thereof, the surrounding environment of the robotic system 100. For example, the imaging devices 240 may include two-dimensional (2D) and three-dimensional (3D) cameras, both of which may have visible and infrared capabilities, lidar, radar, other distance measurement, and other imaging devices. The imaging devices 240 may generate representations of the environment of the robotic system 100, such as digital images or point clouds / depth maps, used to perform machine / computer vision for, for example, automated inspection, robotic guidance, or other robotic applications.

[0049]

[0056] In some embodiments, the sensor unit 230 may comprise a system sensor 242. The system sensor 242 is a device configured to monitor the robotic system 100. For example, the system sensor 242 may include a unit or device that detects and monitors the position of the structural members (such as robot components and end effectors), corresponding joints, or a combination thereof, of the robotic system 100. As yet another example, the robotic system 100 may use the system sensor 242 to track the location, orientation, or a combination thereof, of the structural members and joints during the performance of the operation 118. Examples of the system sensor 242 may include an accelerometer, a gyroscope, or a position encoder.

[0050]

[0057] In some embodiments, the sensor unit 230 may include a contact sensor 244, such as a pressure sensor, force sensor, strain gauge, piezoresistive / piezoelectric sensor, capacitive sensor, elasto-resistive sensor, torque sensor, linear force sensor or other tactile sensor configured to measure characteristics associated with direct contact between multiple physical structures or surfaces.

[0051]

[0058] The sensor unit 230 may include a sensor unit interface 224. The sensor unit interface 224 may be used for communication between the sensor unit 230 and other functional units of the robotic system 100. The sensor unit interface 224 may also be used for communication outside the robotic system 100. The sensor unit interface 224 may receive information from other functional units of the robotic system 100 or from an external source, or may transmit information to other functional units of the robotic system 100 or to an external destination.

[0052]

[0059] The sensor unit interface 224 may include different implementations depending on the functional unit or external unit of the robotic system 100 that the sensor unit 230 interfaces with. The sensor unit interface 224 may be implemented in similar technologies and manners as the implementation of the control interface 204.

[0053]

[0060] 3, an example of an arm unit 102 is shown in accordance with one or more embodiments of the present invention. The arm unit 102 may include a robotic arm 310. The arm unit 102 may further include a sensor unit 230, a gripping unit 306, and one or more suction grippers 308.

[0054]

[0061] In one embodiment, the sensor unit 230 may be attached to the robot arm 310, the gripping unit 306, the suction gripper 308, or a combination thereof. The gripping unit 306 may also be attached to the robot arm 310. The suction gripper 308 may also be attached to the gripping unit 306 and configured to grip the target object 112.

[0055]

[0062] The robot arm 310 functions to enable manipulation of the target object 112 via the grip unit 306, the suction gripper 308, the sensor unit 230, or a combination thereof. The robot arm 310 can include one or more arm sections 314 that are mechanical subsections of the robot arm 310, which together constitute the robot arm 310. In one embodiment, the robot arm 310 can also include one or more mechanical structural joints 312 that connect to the arm sections 314. The structural joints 312 can connect the arm sections 314 together and act as pivot points that can enable rotational and translational movement of the arm sections 314, the grip unit 306, the sensor unit, or a combination thereof. In one embodiment, the structural joints 312 can form a kinematic chain configured to steer the arm sections 314, the grip unit 306, the sensor unit 230, or a combination thereof.

[0056]

[0063] In one embodiment, the arm unit 102 may also include an actuation unit 220 configured to drive or steer the structural joint 312, the arm portion 314, the grip unit 306, the sensor unit 230, or a combination thereof about, at, or connected to the structural joint 312. The actuation unit 220 functions to cause the structural joint 312, the arm portion 314, the grip unit 306, the suction gripper 308, the sensor unit 230, or a combination thereof to perform or undergo rotational and translational movements. In one embodiment, the actuation unit 220 may be integrated into the structural joint 312, the arm portion 314, the grip unit 306, the suction gripper 308, or a combination thereof. In another embodiment, the actuation unit 220 may be external to the structural joint 312, the arm portion 314, the grip unit 306, the suction gripper 308, or a combination thereof.

[0057]

[0064] The arm unit 102 may also include a sensor unit 230. In one embodiment, the sensor unit 230 may include a device configured to detect or measure a sensor reading 246. For example, the arm unit 102 may include a device configured to detect and measure one or more physical characteristics of the robot system 100, such as the system sensors 242 of FIG. 2. Examples of physical characteristics may include a state, condition, or location of the structural joint 312, a location of the arm portion 314, a location of the grip unit 306, a state of the suction gripper 308, or a combination thereof. The system sensors 242 may be used to detect the location of the structural joint 312, the location of the arm portion 314, the location of the grip unit 306, or a combination thereof. The robot system 100 may use the system sensors 242 to track the location and orientation of the structural joint 312, the arm portion 314, the grip unit 306, or a combination thereof during the performance of the task 118, and store the sensor readings 246 as part of the object tracking data 228.

[0058]

[0065] Also, for example, the sensor unit 230 may further comprise one or more imaging devices 240 of FIG. 2. The imaging device 240 may generate a representation of the detected environment, such as a digital image or a point cloud / depth map, which may be further used to perform machine or computer vision for, for example, automated inspection, robotic guidance, or other robotic applications. The robotic system 100 may store and process the digital image or point cloud / depth map to identify, understand, and facilitate the navigation or transportation of the target object 112. In one embodiment, the digital image or point cloud / depth map may be stored in the storage unit 206.

[0059]

[0066] For example, in one embodiment, the sensor unit 230 can capture an image of the designated area 114, such as the interior of a truck, the interior of a container, or a pick-up location for the target object 112, and the robotic system 100 can store the image. Similarly, the sensor unit 230 can capture an image of other examples or locations of the designated area 114 or work location 116, such as a drop location for placing the target object 112 on a conveyor belt, a location for placing the target object 112 in a container, or a location on a pallet for stacking the target object 112, and the robotic system 100 can store the image.

[0060]

[0067] In another embodiment, the sensor unit 230 can also be used to identify the target object 112. For example, the sensor unit 230 can capture an image of the target object 112, an image of a container housing the target object 112, or a combination thereof, and the robotic system 100 can store the image to determine what type of target object 112 it is, so that the robotic system 100 can determine how to use the suction gripper 308 to manipulate, grip, and transport the target object 112.

[0061]

[0068] 2. The contact sensor 244 may measure a characteristic corresponding to the grip of the suction gripper 308 on the target object 112. As such, the contact sensor 244 may output a contact measurement 320 that represents a quantified measurement, such as a measured force, torque, or position, corresponding to adhesion between the suction gripper 308 and the target object 112. For example, the contact measurement 320 may include one or more force, torque, or pressure readings associated with the force applied by the suction gripper 308 to the target object 112.

[0062]

[0069] In one embodiment, the robotic system 100 can generate instructions to perform different actions to accomplish the task 118 based on the contact measurements 320, including gripping, re-gripping, moving, or a combination thereof, of the target object 112. For example, the instructions can include instructions to have the arm unit 102 grip or re-grip the target object 112 if the initial contact measurements 320 are above or below the force threshold 322. The instructions can also include instructions to have the arm unit 102 intentionally drop the target object 112, adjust the location of the arm portion 314, gripping unit 306, suction gripper 308, or a combination thereof, during the task 118, and further adjust the velocity, acceleration, or a combination thereof of the arm portion 314 during the task 118 if the contact measurements 320 are below or above the force threshold 322 during the task 118.

[0063]

[0070] The force threshold 322 refers to a condition against which the robotic system 100 or one or more components of the robotic system 100 compare one or more force, torque, or pressure readings on the target object 112 to determine whether a grip on the target object 112 can be maintained to accomplish a task 118. The force threshold 322 is predetermined by a user of the robotic system and can vary based on the size and material composition of the suction gripper 308 used by the robotic system 100 to grip the target object 112.

[0064]

[0071] It has been found that using suction grippers 308 that cover a larger surface area can hold the target object 112 with greater force, and therefore the force threshold 322 associated with such suction grippers 308 can be greater than suction grippers that cover a smaller surface area. Further details regarding the suction grippers 308 and how the force threshold 322 is used are provided below.

[0065]

[0072] The arm unit 102 may also include a gripping unit 306. The gripping unit 306, together with a suction gripper 308, may be configured to facilitate gripping of the target object 112 by an attractive force, which is achieved by forming and maintaining a vacuum between the gripping unit 306, the suction gripper 308, or a combination thereof, and the target object 112. For example, the gripping unit 306 may include a suction gripper 308 configured to contact a surface of the target object 112 and form a vacuum in a space between the surface of the target object 112. The configuration of the gripping unit 306 and the suction gripper 308 are described in further detail below.

[0066]

[0073] The vacuum is created by lowering the gripping unit 306 via the robotic arm 310, which presses the suction gripper 308 against the surface of the target object 112, displacing any air or gas between the suction gripper 308 and the opposing surface of the target object 112. In one embodiment, the suction gripper 308 may be pressed against the surface of the target object 112 until the robotic system 100 determines that a grip on the target object 112 has been established. For example, to determine when to stop pressing against the surface of the target object 112, in one embodiment, the contact sensor 244 may generate a contact measurement 320 indicative of the pressure between the suction gripper 308 and the surface of the target object 112. The robotic system 100 may compare the contact measurement 320 to a force threshold 322 to determine whether the contact measurement 320 is equal to or greater than the force threshold 322. If the contact measurement 320 is greater than or equal to the force threshold 322 , the robotic system 100 can determine that the suction gripper 308 is pressing sufficiently against the target object 112 to maintain a grip on the target object 112 .

[0067]

[0074] Once a grip is established, the arm unit 102 may attempt to lift the target object 112. As the robotic arm 310 lifts the gripping units 306, the contact sensors 320 may further measure the pressure difference between the interior space of the suction grippers 308 and the surrounding environment to determine whether the pressure is sufficient to keep the target object 112 adhered to the suction grippers 308. Thus, the degree of grip or adhesion of the gripping units 306 and suction grippers 308 to the target object 112 may be based on the number of suction grippers 308 that can adequately generate and maintain a vacuum.

[0068]

[0075] In one embodiment, the gripping unit 306 can include a suction gripper 308. The suction gripper 308 is a mechanical device that uses negative air or water fluid pressure to attach to a non-porous surface and create a vacuum. The suction gripper 308 is configured to hold or secure the target object 112 via an attractive force. The suction gripper 308 can include one or more suction cups 324 attached to a distal end, which can be configured to contact the surface of the target object 112 and maintain a vacuum in a space between the suction cups 324 and the surface of the target object 112.

[0069]

[0076] The suction cup 324 can contact the surface of the target object 112 along a face, an edge, or an angle 338. Additionally, the ability to contact the surface of the target object 112 at an angle 338 allows the suction gripper 308 to grip a target object 112 that is at rest with one side at an angle 338 relative to a horizontal plane (e.g., along a plane parallel to the bottom of a container in or on which the target object 112 is placed).

[0070]

[0077] It has been found that the ability of the suction cup 324 to maintain a grip on the target object 112 at the angle 338 depends on the size of the suction cup 324. For example, a larger size suction cup 324 can grip the target object 112 at a larger angle 338.

[0071]

[0078] The suction cup 324 may be implemented according to a variety of shapes and sizes. The suction cup 324 may further be made from a variety of materials. For example, in one embodiment, the suction cup 324 may be implemented as a circle and may be made from a flexible material including plastic, silicone, nitrile, viton, vinyl, urethane, rubber, which allows the suction cup 324 to flex or bend when picking up the target object 112.

[0072]

[0079] It has been found that the force exerted by the suction cup 324 is a function of the size and shape of the suction cup 324, such that the larger the suction cup 324, the greater the force exerted on the target object 112 by the suction gripper 308, the suction cup 324, or a combination thereof. Furthermore, it has been found that the force of the suction cup 324 is directly correlated to the effective surface area covered by the suction cup 324, and this correlation is characterized by the following equation:

[0080] F = (ΔP)(A) (1)

[0073]

[0081] In the above formula, "F" represents the force exerted by the suction cup 324 via the vacuum, "ΔP" represents the difference between atmospheric pressure and the vacuum pressure between the suction cup 324 and the target object 112, and "A" represents the effective surface area covered by the suction cup 324. Thus, for example, in an embodiment in which the suction cup 324 is implemented as, for example, a circle, the larger the diameter of the suction cup 324, the more force it can exert on the target object 112.

[0074]

[0082] Additionally, it has been found that the size and shape of the suction cup 324 affects the ability of the suction cup 324 to grip the target object 112. For example, a larger suction cup 324 is less suitable for gripping a target object 112 having a surface that includes brittle materials or brittle films that can easily bend, crack, or break when a small force is applied to the suction cup 324. This is because a suction cup 324 with a larger diameter can apply a larger force to the target object 112, which can result in damage to the surface of the target object 112.

[0075]

[0083] It has also been found that the material composition of the suction cup 324 affects the ability of the suction cup 324 to grip the target object 112. For example, a suction cup 324 made from a material that makes the suction cup 324 flexible or soft may be better suited for use in gripping a malleable object, such as a bag, since this flexible or soft characteristic allows the suction cup 324 to flex or bend relative to the surface of the target object 112 to provide a tighter grip on the target object 112.

[0076]

[0084] In one embodiment, the arm unit 102 may further include a vacuum hose (not shown) and a vacuum generator (not shown) attached to the grip unit 306, the suction gripper 308, or a combination thereof to generate a vacuum. For example, in one embodiment, when the sensor unit 230 detects contact between the suction cup 324 and the surface of the target object 112, the vacuum generator may be activated by the robot system 100 to evacuate air from between the suction cup 324 and the surface of the target object 112 to generate a vacuum. As a result, the air pressure between the suction gripper 308 and the surface of the target object 112 is lower than the outside environment between the suction gripper 308 and the surface of the target object 112, so that the target object 112 can be held against the suction gripper 308 by atmospheric pressure. The vacuum generator may include, for example, a vacuum ejector, a blower, or a pump that may evacuate air between the suction cup 324 and the surface of the target object 112 to generate a vacuum.

[0077]

[0085] In one embodiment, the vacuum generator may adjust or vary the rate at which air is extracted to vary the strength of the vacuum. Thus, the faster the air is extracted, the greater the difference between the vacuum pressure between the suction cup 324 and the surface of the target object 112 and atmospheric pressure, creating a stronger vacuum. Additionally, the vacuum may also vary depending on the size of the suction cup 324. For example, the greater the surface area covered by the suction cup 324, the stronger the vacuum will be when air is extracted by the vacuum generator at a faster rate.

[0078]

[0086] In one embodiment, the grip unit 306 can include a sensor unit 230. For example, the grip unit 306 can include a contact sensor 244 configured to determine a contact measurement 320. The contact sensor 244 can generate the contact measurement 320 as a representation of adhesion of the grip unit 306, the suction gripper 308, or a combination thereof, to the target object 112. In one embodiment, the contact sensor 244 can include a touch sensor or tactile sensor configured to indicate whether a surface is in contact with another surface and can be configured to determine the size of the surface in contact with another surface. The contact sensor 244 can also include a pressure sensor configured to measure a pressure, e.g., a vacuum between the suction gripper 308 and the surface of the target object 112. The contact sensor 244 can also include a linear force sensor configured to measure a weight of the target object 112 being carried or supported by the suction gripper 308.

[0079]

[0087] Additionally, the contact sensor 244 may comprise a torque sensor configured to measure a torque or force moment on the suction gripper 308, the gripping unit 306, the robot arm 310, or a combination thereof. The torque or force moment measurement may change (e.g., increase or decrease) when, for example, some of the suction grippers 308 (e.g., peripherally located suction grippers 308) are unable to maintain suction as compared to a fully gripped state.

[0080]

[0088] The torque measurements may further be used in determining the velocity 340 and acceleration 342 of the arm unit 102 while performing the operation 118. For example, in one embodiment, when the suction gripper 308 grips the gripped target object 112 at a point (e.g., the center of mass 236 or other point of the gripped target object 112) and begins performing the operation 118, the torque sensor may measure the torque or moment of force on the target object 112 using the following equation:

[0089] T = (F) (d) (2)

[0081]

[0090] In the above equation, "T" represents the torque or moment of force on the target object 112, "F" represents the force applied to the target object 112 as a result of the rotational movement of the target object 112, and "d" represents the distance from a point (e.g., the center of mass 236 or other point of the target object 112 being gripped) to the center of rotation (e.g., the structural joint 312 about which the target object 112 rotates).

[0082]

[0091] The robotic system 100 can compare the torque measurement "T" to a force threshold 322, which may represent, for example, the maximum amount of force with which the suction cup 324 holds the target object 112. Based on this comparison, the robotic system 100 can determine whether the task 118 is being performed successfully without dropping the target object 112. Furthermore, based on the torque measurement, the robotic system 100 can compensate for the torque applied to the target object 112 by adjusting the velocity 340 and acceleration 342 of the arm unit 102 to counteract the effect of the torque on the target object 112. As an example, in one embodiment, if the torque measurement is greater than the force threshold 322, the suction gripper 308 will lose its grip on the target object 112, so the robotic system 100 adjusts the velocity 340 and acceleration 342 of the arm unit 102 to reduce the torque, for example by slowing down the velocity 340 and acceleration 342 of the arm unit 102, so that the gripper 308, the suction cup 324, or a combination thereof, can maintain its grip on the target object 112 and transport the target object 112 without dropping it.

[0083]

[0092] Depending on the type and location of the contact sensors 244, the contact measurements 320 may correspond to a sum or average of measurements of, for example, internal pressure, linear force, torque, or a combination thereof, for each suction gripper 308. In one embodiment, the contact sensors 244 attached to the gripping unit 306, the suction gripper 308, or a combination thereof, can determine a non-zero reading related to a weight being carried by the suction gripper 308. Such weight corresponds to a linear force or torque and can be used to determine whether the suction gripper 308 has a sufficient grip on the target object 112 to perform an operation 118 including transferring the target object 112.

[0084]

[0093] In another embodiment, the contact sensor 244 can further determine a vacuum force corresponding to the suction gripper 308. If the vacuum force is greater than or equal to the force threshold 322, the contact sensor 244 can register a non-zero reading associated with the suction gripper 308 and determine that the gripping unit 306 and suction gripper 308 have a grip on the target object 112.

[0085]

[0094] In one embodiment, if the suction gripper 308 loses its grip, such as when the vacuum force, linear force, or a combination thereof falls below the force threshold 322 as determined by the contact sensor 244, the contact sensor 244 may determine a zero reading or a non-zero reading due to the failure of the grip. Additionally, due to an uneven distribution of force, a torque sensor associated with the gripping unit 306, the suction gripper 308, or a combination thereof may determine a non-zero reading.

[0086]

[0095] In one embodiment, if the robotic system 100 determines that the suction gripper 308 has failed to grip, the control unit 202, the communication unit 212, the sensor unit 230, or a combination thereof, notifies a user of the robotic system 100, another functional unit of the robotic system 100, or a combination thereof that there has been a failure to grip the target object 112. The user or other functional unit of the robotic system 100 can then address the issue by determining the cause of the failure or attempt to recalibrate, reconfigure, or re-grip the target object 112.

[0087]

[0096] In one embodiment, when all of the suction grippers 308 are establishing and maintaining a grip or vacuum on the target object 112, the linear force, vacuum force, or combination thereof will have a non-zero magnitude at all of the suction grippers 308, and the deviation between the linear force, vacuum force, or combination thereof will be within a relatively small range. Furthermore, because the weight is substantially evenly distributed across all of the suction grippers 308, the torque measured at the gripping units 306, suction grippers 308, or combination thereof will be close to a zero value. Thus, deviations in the linear force, vacuum force, torque readings, or combination thereof may inversely represent the strength of the grip. Thus, the robotic system 100 may use the above examples of contact measurements 320 as representative of the grip of the gripping units 306, suction grippers 308, suction cups 324, or combinations thereof, on the target object 112.

[0088]

[0097] In one embodiment, if some of the suction grippers 308, suction cups 324, or combinations thereof fail to grip the target object 112 while other suction grippers 308, suction cups 324, or combinations thereof establish and maintain a grip or vacuum on the target object 112, the contact sensor 244 can determine whether the grip from those instances of suction grippers 308 that are maintaining a grip or vacuum is sufficient to maintain stability of the target object 112 to manipulate, transport, or perform other operations 118 on the target object 112, taking into account the linear force, vacuum force, or combinations thereof. For example, while the torque measurement is at or near a zero reading, the contact sensor 244 can compare the weight of the object to which the linear force, vacuum force, measured torque, or combinations thereof are applied to a force threshold 322 to determine whether the linear force and vacuum force are equal to or greater than the force threshold 322 so that the target object 112 can be safely manipulated or transported.

[0089]

[0098] Also, for example, the robotic system 100 can further use a lookup or conversion table, database, formula, process, or combination thereof to convert or translate the expected readings as a function of different orientations of the grip unit 306 and the target object 112. In some embodiments, the master data 226, the configuration data 248, or a combination thereof can include expected readings for each of the different orientations of the grip unit 306 and the target object 112. The robotic system 100 can use the expected readings to evaluate or process the contact measurements 320 as a function of the orientation of the grip unit 306 and the target object 112.

[0090]

[0099] The suction grippers 308 may be implemented according to a layout 302 along the distal end of the gripping unit 306. The layout 302 refers to the manner in which the suction grippers 308, suction cups 324, or combinations thereof are arranged. The layout 302 may be implemented according to a variety of shapes and orientations, including linear, rectangular, circular, triangular, square, or combinations thereof. For example, in one embodiment, the layout 302 may be implemented as a rectangular grid at the distal end of the gripping unit 306 with x suction grippers 308 along a first direction 328 and y suction grippers 308 along a vertical direction 322 perpendicular to the first direction 328. As a result, the suction grippers may form an x ​​by y rectangular grid. As an example, in the embodiment shown in FIG. 3, the layout 302 is shown as a 2 by 2 square grid with two suction cups 324 along the first direction 328 and two suction cups 324 along the vertical direction 332. In other embodiments, the number of suction cups 324 can be increased to form a square grid of different sizes or even a rectangular grid.

[0091]

[0100] In another embodiment, the layout 302 may be implemented in a circular configuration with an equal number of suction grippers 308 positioned equidistant from the center of the gripping unit 306. In another embodiment, the layout 302 may be implemented in a triangular configuration with the suction grippers 308 positioned along three lines at the distal end of the gripping unit 306, with each line meeting the other two lines at an angle 338. The following description is merely an example, and other layout 302 configurations may be used.

[0092]

[0101] The suction grippers 308 can be controlled individually, in groups or subgroups, or all together based on the layout 302 as described above and the number of suction grippers 308 used to grip the target object 112. The robotic system 100 can control the suction grippers 308 by controlling the actuation units 220, vacuum hoses, vacuum generators, or combinations thereof. In one embodiment, the actuation units 220, vacuum hoses, vacuum generators, or combinations thereof can be attached to each suction gripper 308 individually or to a group of multiple suction grippers 308. The actuation units 220, vacuum hoses, vacuum generators, or combinations thereof can cause the suction grippers 308 to perform their functions, such as causing mechanical movement of the suction grippers 308 to press down on the target object 112 to establish a grip, and generate the suction and vacuum conditions necessary to maintain or establish a grip on the target object 112.

[0093]

[0102] For example, in one embodiment, the robotic system 100 can control the actuation unit 220, the vacuum hose, the vacuum generator, or a combination thereof, by turning the actuation unit 220, the vacuum hose, the vacuum generator, or a combination thereof, ON or OFF so that the actuation unit 220, the vacuum hose, the vacuum generator, or a combination thereof can be pressed down on the target object 112 to establish a grip and generate the suction and vacuum conditions necessary to maintain or establish a grip of the target object 112. As a result, the robotic system 100 can control the precision with which the suction gripper 308 grips the target object 112.

[0094]

[0103] The suction grippers 308 may further be controlled based on the number and layout 302 of the suction grippers 308. For example, in one embodiment, the robotic system 100 may generate one or more gripping patterns 330 for controlling the suction grippers 308 individually, as groups or subgroups, or all together based on the number and layout 302 of the suction grippers 308. The gripping patterns 330 refer to the configurations of the suction grippers 308 that can be used to grip the target object 112. While the robotic system 100 is online, the gripping patterns 330 may be used to grip the target object 112 and perform the operation 118 of FIG. 1 .

[0095]

[0104] For example, in one embodiment, the grip pattern 330 may be represented as a binary representation of the respective ON / OFF states of the actuation units 220, vacuum hoses, vacuum generators, or combinations thereof. As a result, each state of the actuation units 220, vacuum hoses, vacuum generators, or combinations thereof may be represented as a "1" or a "0" representing "ON" or "OFF". Based on the number and layout 302 of the suction grippers 308, the robotic system 100 may calculate the number of combinations of "ON" and "OFF" states of the suction grippers 308 using the following formula:

[0105] C=(S) n (3)

[0096]

[0106] In the above formula, "C" represents the number of combinations for the suction grippers 308, "S" represents the number of states of each of the actuation units 220, vacuum hoses, vacuum generators, or combinations thereof, and "n" represents the number of suction grippers 308. As an example, if the suction grippers 308 are arranged as a 2x2 square, the robotic system 100 may calculate C=(2) for the suction grippers 308. 4Thus, 16 different grip patterns 330 can be computed. In one embodiment, the grip patterns 330 can be represented as a lookup table 334. The lookup table 334 can be an array or matrix representation of the grip patterns 330. For example, the lookup table 334 can have the values ​​of the grip patterns 330 represented as binary codes 336, such as, for example, 1001, 0000, and 0001. The binary codes 336 can represent which suction grippers 308 are turned ON or OFF. For example, in one embodiment, one value of the binary code 336, 0000, can indicate that all suction grippers 308 are OFF, while another value of the binary code 336, 1111, can indicate that all suction grippers 308 are ON.

[0097]

[0107] In one embodiment, the grip patterns 330 may be pre-computed and stored as part of the configuration data 248 such that the robotic system 100 may be configured to have any number of grip patterns 330 available for a particular instance of a layout 302 of suction grippers 308. The grip patterns 330 may be used to grip the target object 112 during the performance of an operation 118. The manner in which the grip patterns 330 are used to grip the target object 112 is described in further detail below.

[0098]

[0108] 4, an exemplary control flow of the operational phases of the robotic system 100 in one embodiment of the present invention is illustrated. In one embodiment, the robotic system 100 may be operated in two phases, including a pre-configured state 402 and an online state 406. Although the embodiment illustrated in FIG. 4 assumes that the pre-configured state 402 is executed prior to operation of the robotic system 100 in the online state 406, this order of operations is merely an example, and in other embodiments, the pre-configured state 402 may be executed in parallel or in real-time with the online state 406. Real-time refers to instances in which the robotic system 100 is used such that the parameters, variables, configuration data 248 of FIG. 2, or combinations thereof of the robotic system 100 are determined during manufacturing, assembly, packing, packaging, warehousing, and transportation scenarios.

[0099]

[0109] The pre-configuration state 402 is an operational mode in which parameters, variables, configuration data 248, or combinations thereof are determined for the robotic system 100. The parameters, variables, configurations, or combinations thereof may include any thresholds or settings (e.g., settings associated with the arm unit 102, gripping unit 306, suction gripper 308, sensor unit 230, or combinations thereof of FIG. 1 necessary to perform the operation 118 of FIG. 1 on the target object 112 of FIG. 1).

[0100]

[0110] For example, in one embodiment, the pre-configured setting state 402 may include a suction control pattern generator 404. The suction control pattern generator 404 may determine parameters, variables, configuration data 248, or combinations thereof, associated with, for example, the grip unit 306 of FIG. 3, the suction gripper 308 of FIG. 3, the sensor unit 230 of FIG. 2, or combinations thereof. The robotic system 100 may implement the suction control pattern generator 404 using various functional units of the robotic system 100 shown in FIGS. 2 and 3, one or more components external to the robotic system 100, or a combination thereof. An external component refers to a component that is external to the robotic system 100. The implementation of the suction control pattern generator 404 is described in further detail below.

[0101]

[0111] The online state 406 is an operational mode in which the robotic system 100 is employed when the robotic system 100 is performing operations 118 on target objects 112 during manufacturing, assembly, packing, packaging, warehousing, and transportation scenarios. During the online state 406, the robotic system 100 can perform operations 118 on target objects 112 using the parameters, variables, configuration data 248, or combinations thereof, of the robotic system 100 determined during the pre-configuration state 402. The implementation of the online state 406 is described in further detail below.

[0102]

[0112] The pre-configured state 402 and the online state 406 may be implemented based on a set of instructions stored in the software 202 of FIG. 2 or the storage unit 206 of FIG. 2, which may be executed by the control unit 202 of FIG. 2, other functional units of the robotic system 100, or a combination thereof.

[0103]

[0113] 5, an exemplary control flow for the suction control pattern generator 404 is shown according to one embodiment of the present invention. The robotic system 100 can implement the suction control pattern generator 404 using various functional units shown in Figures 2 and 3 of the robotic system 100 of Figure 1, one or more components external to the robotic system 100, or a combination thereof.

[0104]

[0114] In one embodiment, the suction control pattern generator 404 may be configured to generate the grip pattern 330. The suction control pattern generator 404 may be further configured to recognize the target object 112 of FIG. 1 and test one or more locations of grip points 518 associated with the target object 112 before the robotic system 100 encounters the target object 112 in the online state 406 of FIG. 4. By testing one or more locations of grip points 518, the suction control pattern generator 404 may determine how to handle the target object 112 during the online state 406. The suction control pattern generator 404 may be further configured to determine which grip pattern 330 to use for the target object 112 based on the grip points 518. The grip points 518 refer to areas on a surface of the target object 112 that can be gripped by the suction gripper 308 of FIG. 3, the suction cups 324 of FIG. 3, or a combination thereof.

[0105]

[0115] In one embodiment, the suction control pattern generation mechanism 404 can be implemented using a grip pattern generation module 510, a scan module 502, a grip point mapping module 508, an external unit 514, an evaluation module 512, and a storage unit 206. In one embodiment, the scan module 502 can be connected to the grip point mapping module 508. The grip point mapping module 508 can be connected to the evaluation module 512 and, optionally, to the storage unit 206 and the external unit 514. The evaluation module 512 can be connected to the grip pattern generation module 510 and the storage unit 206. The grip pattern generation module 510 can be connected to the storage unit 206 and, optionally, to the external unit 514.

[0106]

[0116] The grip pattern generation module 510 may enable generation of the grip pattern 330 in the manner described with reference to FIG. 3. The grip pattern generation module 510 may further enable generation of the grip pattern location 530. The grip pattern location 530 refers to a location in the layout 302 of FIG. 3 where the suction gripper 308 can apply a uniform force distribution to the target object 112. The target object 112 may be gripped using the grip pattern location 530. The grip pattern location 530 may vary based on a number of variables including the grip pattern 330, physical characteristics of the suction cup 324, or a combination thereof. In one embodiment, the grip pattern location 530 may be determined based on the layout 302 of the grip pattern 330 and the suction cup data 534.

[0107]

[0117] Suction cup data 534 refers to variables, parameters, thresholds, or combinations thereof that may characterize the physical properties of the suction cups 324. For example, suction cup data 534 may include data regarding the maximum weight that can be held by one or more suction cups 324, the degree of flexibility of the suction cups 324, the tensile strength, coefficient of friction, size, or combinations thereof. Suction cup data 534 may be known by the robotic system 100 and stored in the storage unit 206, the external unit 514, or a combination thereof.

[0108]

[0118] The external unit 514 refers to storage that is external to the robotic system 100. Similar to the storage unit 206, the external unit 514 may be volatile memory, non-volatile memory, internal memory, external memory, or a combination thereof. For example, the external unit 514 may be non-volatile memory such as non-volatile random access memory (NVRAM), flash memory, disk storage, or volatile storage such as static random access memory (SRAM). In another embodiment, the external unit 514 may be a database or a lookup table.

[0109]

[0119] By way of further example, in an embodiment in which the grip pattern location 530 is determined based on the grip pattern 330 and suction cup data 534, the grip pattern location 530 may be specifically determined based on the amount of force that the suction cup 324 can hold and the location of the suction cup 324 in the layout 302. The grip pattern generation module 510 may determine the grip pattern location 530 by obtaining a variable, parameter, or combination thereof indicative of a maximum weight that can be held by one or more suction cups 324. Based on this maximum weight, the grip pattern generation module 510 may set the geometric center 542 of the suction cup 324 as a default location where the suction cup 324 can most evenly distribute the maximum weight. In a configuration of the grip pattern 330 in which only one suction cup 324 is turned on to grip the target object 112, the grip pattern generation module 510 may assign the geometric center 542 of this suction cup 324 to the grip pattern location 530. The geometric center 542 refers to the average location of all points of the suction cup 324 in all coordinate axis directions.

[0110]

[0120] In an embodiment in which multiple instances of the suction gripper 308 are used to grip the target object 112, the grip pattern generation module 510 may determine the grip pattern location 530 based on a weighted average that takes into account the geometric center 542 of the suction cup 324 and a weighting factor 536. The weighting factor 536 refers to a parameter assigned by the grip pattern generation module 510 to the suction cup 324 based on the amount of force the suction cup 324 can exert on the target object 112. The weighting factor 536 may be predetermined by a user of the robotic system 100.

[0111]

[0121] For example, in one embodiment, the more weight a suction cup 324 can hold, the higher the weighting factor 536 assigned to the suction cup 324. In another embodiment, the more surface area a suction cup 324 can cover, the higher the weighting factor 536 assigned to the suction cup 324. As a result, the grip pattern generation module 510 can calculate grip pattern locations 530 based on which suction cup 324 applies the most force to the target object 112, and position the grip pattern locations 530 near the suction cups 324 in scenarios having multiple instances of the suction cups 324.

[0112]

[0122] As an example, in an embodiment in which the layout 302 is a 2×2 square grid and all of the suction cups 324 are capable of holding the same maximum weight and are the same size, the grip pattern location 530 may be determined to be at a point located in the center of the square grid that is the same distance from each of the four sides of the square. As another example, if the layout 302 is a 2×2 square grid and the suction cups 324 are different sizes but capable of holding the same maximum weight, the grip pattern location 530 may be assigned to a point closer to the larger sized suction cup 324. As a result, the robotic system 100 may distribute forces on the target object 112 in a uniform or near uniform manner.

[0113]

[0123] It has been found that gripping the target object 112 based on the grip pattern locations 530 provides the most stable means of gripping the target object 112 because gripping the target object 112 based on the grip pattern locations 530 provides the most even distribution of force across the target object 112. Furthermore, it has been found that gripping the target object 112 based on the grip pattern locations 530 while performing an operation 118 on the target object 112 results in less dropping of the target object 112.

[0114]

[0124] In one embodiment, the grip pattern generation module 510 can generate grip pattern locations 530 for each of the grip patterns 330. As a result, the grip pattern generation module 510 can generate an index 538 of the grip pattern 330 and the associated grip pattern location 530 instance. The index 538 points to an array, lookup table, or combination of the grip pattern 330 and the associated grip pattern location 530 instance. In one embodiment, once the grip patterns 330, grip pattern locations 530, and index 538 are generated, they can be sent to an evaluation module 512 for further processing. The evaluation module 512 is described in more detail below.

[0115]

[0125] The scanning module 502 may enable scanning or imaging of features by the sensor unit 230 of FIG. 2 including one or more of imaging devices, such as a two-dimensional (2D) camera, a three-dimensional (3D) camera, an infrared camera, a lidar, a radar, other distance measuring or imaging devices, or a combination thereof. For example, the scanning module 502 may be configured to detect the surrounding environment of the target object 112 and take a scanned image 504 including a digital image, a point cloud / depth map, or a combination thereof, to identify the target object 112, a grip point 518 of the target object 112, or a combination thereof. The scanned image 504 may be used to generate a digital model 540 of the target object 112 that may be used in determining where to grip the target object 112.

[0116]

[0126] The digital model 540 is a computer representation of the physical features of the scanned environment, including the physical features of the target object 112. The digital model 540 can simulate the target object 112. The digital model 540 can be represented as a computer file, format, data structure, or combination thereof.

[0117]

[0127] In one embodiment, the target object 112 may have physical characteristics that are unknown to the robotic system 100, for example, because it may be an object that has not been encountered or is not known to the robotic system 100. The physical characteristics may include shape, size, dimensions, thickness 522, composition 506, surface contour 520, or a combination thereof.

[0118]

[0128] The scan module 502 may enable imaging of the target object 112, thereby enabling the sensor unit 230 to generate sensor readings 246, FIG. 2, related to physical characteristics of the target object 112, including the target object's physical dimensions, shape, height, width, depth, or combinations thereof. These sensor readings 246 may be shown in the scan image 504.

[0119]

[0129] In yet another embodiment, the scan module 502, in conjunction with the sensor unit 230 and other external devices including an x-ray machine or spectrometer, may generate a scan image 504 further indicative of the composition 506 of the target object 112, such that the scan image 504 may include details regarding the composition 506 of the target object 112. The composition 506 refers to the physical makeup, chemical makeup, or a combination thereof, of the target object 112. The robotic system 100 may determine the force or grip that may be applied by the suction gripper 308 at the identified grip point 518 based on the composition 506 of the target object 112, such that a balanced amount of force is applied at the grip point 518 so that the target object 112 is not damaged during the operation 118, or to select a grip point 518 at which the target object 112 can be gripped. The identification of the composition 506 may also be used to determine where the target object 112 cannot be gripped.

[0120]

[0130] For example, if the composition 506 of the surface of the target object 112 is identified as being capable of withstanding only forces equal to or less than the force threshold 322 in FIG. 3 before the material making up the surface is irreversibly deformed or destroyed, the robotic system 100 can determine whether or not to use that location of the grip point 518, or can identify another location of the grip point 518 that will not damage the target object 112. The robotic system 100 can also adjust the force or vacuum applied to the grip point 518 based on the identification of the composition 506. For example, in one embodiment, if the composition 506 of the surface of the target object 112 is identified as being capable of withstanding only forces equal to or less than the force threshold 322 before the material making up the surface is irreversibly deformed or destroyed, the robotic system 100 can determine to adjust the force or vacuum applied by the actuation unit 220 to ensure that the force is less than the force threshold 322. In one embodiment, the scanning module 502 may pass the scanned image 504, the digital model 540, or a combination thereof, once generated, to a grip point mapping module 508 for further processing.

[0121]

[0131] The grip point mapping module 508 may enable identification of grip points 518 of the target object 112 based on the scanned image 504, the digital model 540, or a combination thereof. The grip point mapping module 508 may identify the grip points 518 by processing the scanned image 504 and performing an analysis based on the object data 516. The object data 516 refers to data representing known physical properties of an object or shape, known properties of a material, or a combination thereof. For example, the object data 516 may include a formula associated with the object or shape, known properties such as a known atomic structure, known nanostructure, known microstructure, known macrostructure, known bonding properties, known kinetics, known crystal structure, known mechanical properties, etc., for the object or shape, or a combination thereof. In one embodiment, the object data 516 may be stored in the external unit 514. The grip point mapping module 508 may communicate with the external unit 514 via the communication unit 212 to retrieve the object data 516. In another embodiment, the object data 516 may be stored in the storage unit 206 as part of the master data 226 of the robotic system 100 .

[0122]

[0132] In one embodiment, the grip point mapping module 508 can perform the analysis, for example, by determining the geometric shape exhibited by the scanned image 504, the digital model 540, or a combination thereof by comparing shapes in the scanned image 504, the digital model 540, or a combination thereof to a set of known shapes that may be part of the object data 516. If, based on this comparison, the grip point mapping module 508 recognizes that the geometric shape matches or closely matches the shape of the target object 112, the grip point mapping module 508 can perform the analysis based on the object data 516 associated with the matched geometric shape or object.

[0123]

[0133] The grip point mapping module 508 may determine physical characteristics of the geometric shape, such as calculating a surface area for the geometric shape, a center of mass 236 of FIG. 2, or a combination thereof, based on a formula for determining a surface area for an object or shape. Based on the physical characteristics of the geometric shape, the grip point mapping module 508 may determine grip points 518 for the target object 112.

[0124]

[0134] In one embodiment, the grip point mapping module 508 may determine the grip points 518 based on the center of mass 236 (which may be an estimated center of mass 236). The grip point mapping module 508 may identify the grip points 518 for the target object 112 as points on the surface that are close to the center of mass 236 of the target object 112. As a result, the grip point mapping module 508 may ensure that the target object 112 is gripped with a uniform distribution of mass.

[0125]

[0135] In one embodiment, the grip point mapping module 508 may further consider other factors in addition to the center of mass 236 to identify the grip points 518. For example, in one embodiment, the grip point mapping module 508 may further consider a surface contour 520, including the flatness or curvature of the surface of the target object 112, a thickness 522 of the surface of the target object 112, a surface composition 506 of the countermeasure object 112, a surface area of ​​the target object 112, or a combination thereof to identify the grip points 518.

[0126]

[0136] For example, in one embodiment, the grip point mapping module 508 may identify the center of mass 236 of the target object 112. Additionally, the grip point mapping module 508 may determine that a particular surface closest to the center of mass 236 is not a sufficiently flat surface (e.g., the surface has curvature or unevenness) such that the suction gripper 308 cannot grip the grip point 518 associated with that surface, or the surface area is too small for gripping by the suction gripper 308. The grip point mapping module 508 may then further identify one or more other locations for the grip point 518 on the surface near the center of mass 236 at which the suction gripper 308 can grip the target object 112. Such other locations may be, for example, at a corner of the target object 112 (or, alternatively, a "corner grip"), on a face of the target object 112, or along any other surface along the target object 112. As a result, the grip point mapping module 508 attempts to identify the point closest to the center of mass 236 where a grip can be achieved, but if a grip cannot be achieved at that point, it can determine a different location for the grip point 518, such as a location along a corner of the target object 112, to grip the target object 112.

[0127]

[0137] In another embodiment, the grip point mapping module 508 may further consider the composition 506 of the surface of the target object 112, as determined by the scanned image 504, the digital model 540, or a combination thereof, and the known material properties of the object. For example, in one embodiment, if a grip point 518 is identified to be on a surface of the target object 112 that is made of a weak material and on which a force greater than the force threshold 322 cannot be applied without being damaged, the grip point mapping module 508 may determine whether to use the grip point 518 or may further identify another location for the grip point 518 closer to the center of mass 236 where the suction gripper 308 can grip the target object 112. Such another location may be, for example, a location at a corner of the target object 112 (or a "corner grip"), a location on the face of the target object 112, or a location along any other surface along the target object 112.

[0128]

[0138] In one embodiment, if the grip point mapping module 708 is unable to find a match for the target object 112 based on a comparison of the scanned image 504, the digital model 540, or a combination thereof to a set of known shapes or objects, a user of the robotic system 100 can further assist in identifying the grip points 518 by setting grip points 518 on the target object 112. The user of the robotic system 100 can specify the grip points 518 by analyzing the scanned image 504, the digital model 540, or a combination thereof, for example, using the user interface 216 on the display interface 218 to determine what is the optimal position for the suction gripper 308 to grip the target object 112. In one embodiment, the user of the robotic system 100 can set the grip points 518 using an input device of the user interface 216.

[0129]

[0139] In one embodiment, the grip point mapping module 508 can generate a ranking order 524 of the grip points 518. The ranking order 524 can be an array, a look-up table, or a combination thereof, that indicates the locations of the grip points 518 that are most suitable for gripping the target object 112. The grip points 518 can be ranked from most suitable to least suitable based on a number of factors, including distance from the center of mass 236, the composition 506 of the surface of the target object 112, the surface area available for gripping by the grip unit 306, or a combination thereof.

[0130]

[0140] For example, in one embodiment, grip points 518 closer to the center of mass 236 are ranked higher in the ranking order 524 because grip points 518 are most stable at gripping the target object 112 when the mass of the target object 112 is uniformly distributed. However, if the grip point mapping module 508 determines that it cannot use an identified grip point 518 along the surface of the target object 112 closer to the center of mass 236 because, for example, the surface area is too small, the suction gripper 308 cannot grip at the angle 338, the surface composition 506 is too brittle to be gripped by the suction gripper 308 because the suction gripper 308 would apply a force greater than the force threshold 322 resulting in damage to the surface of the target object 112, or a combination thereof, the grip point mapping module 508 can assign such grip point 518 as having a lower ranking in the ranking order 524. As a more specific example, the grip point mapping module 508 may determine other locations of grip points 518 that may be identified for gripping the target object 112 and give the other locations of grip points 518 a higher ranking in the ranking order 524. Such other locations may be, for example, locations at corners of the target object 112 (or "corner grips"), locations on a face of the target object 112, or locations along any other surface along the target object 112.

[0131]

[0141] As a result, the grip point mapping module 508 can determine a most suitable example of a grip point 518 for a given example of a target object 112. In another embodiment, a user of the robotic system 100 can override one or more rankings determined by the grip point mapping module 508 and determine a ranking order 524. In another embodiment, a user can set the ranking order 524 of the grip points 518.

[0132]

[0142] It has been discovered that identifying grip points 518 as described above, generating a ranking order 524, or a combination thereof, can provide the robotic system 100 with a list of preferred locations for grip points 518 at which to grip the target object 112. Additionally, the ranking order 524 can enable the robotic system 100 to have multiple fallback grip points for gripping the target object 112 such that the target object 112 can be gripped regardless of the target object's orientation or environmental conditions.

[0133]

[0143] In one embodiment, once the grip points 518 are determined, the grip points 518, the ranking order 524, or a combination thereof may be passed to an evaluation module 512. The evaluation module 512 may map the grip points 518 to grip patterns 330 and determine a grip pattern rank 528. The grip pattern rank 528 may be an array, a lookup table, or a combination thereof that indicates which configuration of the grip patterns 330 should be used to grip the target object 112.

[0134]

[0144] The evaluation module 512 allows for mapping of grip points 518 to grip patterns 330, determining grip pattern ranks 528, or combinations thereof, in a variety of ways. For example, in one embodiment, the evaluation module 512 can determine, for each of the grip patterns 330, whether the grip points 518 can be gripped by a particular configuration of the grip patterns 330 by aligning the grip points 518 with the grip pattern locations 530. In another example, the evaluation module 512 can determine whether there are instances of the grip patterns 330 associated with the grip pattern locations 530 for gripping the target object 112 using all or most of the suction grippers 308, suction cups 324, or a combination thereof. For example, if the evaluation module 512 determines that not all suction grippers 308 can be used to grip the target object 112 at a particular instance of a grip point 518, the evaluation module 512 may assign a low grip pattern rank 528 value to the particular configuration of grip patterns 330 for the particular location of the grip point 518 and search for other instances of grip patterns 330 that can be used to grip the target object 112 at the particular location of the grip point 518. However, if the evaluation module 512 determines that one or more instances of grip patterns 330 can be used to grip the target object 112 at the particular location of the grip point 518, the evaluation module 512 may assign a grip pattern 330 having a higher grip pattern rank 528 value for gripping at the particular location of the grip point 518.

[0135]

[0145] In another embodiment, the evaluation module 512 can determine the grip pattern rank 528 based on the suction grippers 308 having suction cups 324 that are capable of gripping the target object 112. For example, in one embodiment, multiple instances of the suction grippers 308 can be used to grip the target object 112. In another example, the suction gripper 308 has more than one instance of the suction cups 324. Specifically, by way of example, the multiple suction cups 324 may be formed from different materials, different sizes, or combinations thereof, which may affect the ability of the suction cups 324 to grip the target object 112. The evaluation module 512 can assign a lower value of the grip pattern rank 528 to grip patterns 330 that use suction cups 324 that are not suitable for gripping the target object 112 (e.g., cannot support the weight of the target object 112).

[0136]

[0146] In another embodiment, the evaluation module 512 can determine the grip pattern rank 528 according to a user's preference. For example, a user of the robotic system 100 can configure examples of grip patterns 330, examples of suction grippers 308, or a combination thereof that can be used in gripping a target object 112. The evaluation module 512 can assign higher values ​​of the grip pattern rank 528 to examples of grip patterns 330 that the user prefers.

[0137]

[0147] Based on the mapping, the assessment module 512 can determine a grip pattern rank 528 for all of the grip points 518 and associated grip pattern 330 instances. In one embodiment, once the grip pattern rank 528 is generated, the grip pattern rank 528 can be stored in the storage unit 206. The grip pattern rank 528 can be used by the online 406 robotic system 100 to grip the target object 112.

[0138]

[0148] It has been found that a robotic system 100 implementing the suction control pattern generation mechanism 404 described herein can provide enhanced control when gripping a target object 112 because the generated grip pattern ranks 528 provide a list of optimal examples of grip points 518 and grip patterns 330 by which the target object 112 should be gripped. Furthermore, it has been found that the enhanced control also results in fewer drops when gripping the target object 112.

[0139]

[0149] Furthermore, it has been found that the robotic system 100 implementing the suction control pattern generation mechanism 404 described herein provides greater configurability and customization based on the capabilities of the robotic system 100 to generate an individually configured configuration of the grip pattern 330 for each instance of the target object 112 such that the identified target object 112 can be gripped according to its individual physical characteristics.

[0140]

[0150] Furthermore, it has been found that a robotic system 100 implementing the suction control pattern generation mechanism 404 described herein can grip the target object 112 with a higher probability based on its ability to generate customized grip pattern 330 configurations such that the robotic system 100 has several options for gripping the target object 112.

[0141]

[0151] The suction control pattern generator 404 has been described with exemplary module functions and ordering. The robotic system 100 may separate or order the modules differently. For example, the software 210 may include a module for the suction control pattern generator 404. As a specific example, the software 210 may include a grip pattern generation module 510, a scan module 502, a grip point mapping module 508, an evaluation module 512, and associated sub-modules contained therein.

[0142]

[0152] 2 may execute software 210 to operate the modules. For example, the control unit 202 may execute software 210 to implement grip pattern generation module 510, scanning module 502, grip point mapping module 508, evaluation module 512, and associated sub-modules contained therein.

[0143]

[0153] The modules described in this application may be implemented as instructions stored on a non-transitory computer-readable medium that are executed by the control unit 202. The non-transitory computer-readable medium may include the storage unit 206. The non-transitory computer-readable medium may include non-volatile memory, such as a hard disk drive, a non-volatile random access memory (NVRAM), a solid-state storage device (SSD), a compact disk (CD), a digital video disk (DVD), or a universal serial bus (USB) flash memory device. The non-transitory computer-readable medium may be integrated as part of the robotic system 100 or may be installed as a removable part from the robotic system 100.

[0144]

[0154] 6, an example of mapping grip points 518 to grip patterns 330 is shown in accordance with one or more embodiments of the present invention. FIG. 6 illustrates an embodiment in which the evaluation module 512 of FIG. 5 determines whether a grip point 518 can be gripped by a particular configuration of grip patterns 330 by aligning the grip point 518 with the grip pattern location 530 for each of the grip patterns 330 and determining whether there is a grip pattern 330 associated with the grip pattern location 530 where all or most of the suction grippers 308, suction cups 324, or a combination thereof, can be used to grip the target object 112. FIG. 6 also illustrates an embodiment in which the layout 302 is a 2×2 square and the suction cups 324 are of various sizes. FIG. 6 is shown for illustrative purposes only. Other examples of layouts 302 and grip patterns 330 may be used.

[0145]

[0155] As shown in FIG. 6, the mapping can begin by attempting to grip the target object 112 at the center of mass 236 of the target object 112. If the center of mass 236 is located in a forbidden region 602 (e.g., a point or area on the surface of the target object 112 that cannot be gripped because the composition 506 of the material that makes up the surface of the target object 112 is too brittle to grip compared to the baseline, or for any other reason described with reference to FIG. 5), the evaluation module 512 can determine a grip point 518 along the surface of the target object 112 away from the center of mass 236 such that the target object 112 can be gripped according to the layout 302 and grip pattern 330. The evaluation module 512 can determine an offset 604 to align the grip pattern location 530 away from the center of mass 236 of the target object 112 with the potentially grippable grip point 518. The offset 604 refers to the distance the evaluation module 512 moves the grip pattern location 530 away from the center of mass 236. 6 , the evaluation module 512 may determine a grip pattern location 530 at location "A" determined as a potential location for one or more grip points 518 capable of being gripped. The evaluation module 512 may test the one or more grip points 518 to determine whether or not these grip points 518 can be gripped according to the grip pattern 330. If the one or more grip points 518 can be gripped according to the grip pattern 330, the evaluation module 512 may assign location "A" as a grip pattern rank 528 as one of the preferred instances of the grip pattern 330 for gripping the target object 112.

[0146]

[0156] In one embodiment, the evaluation module 512 can further determine other examples of grip points 518 that can be gripped. For example, the evaluation module 512 can again determine an offset 604 to move the grip pattern location 530 to another location of the grip point 518 and determine whether the grip point 518 is grippable. For example, in FIG. 6, the grip point 518 is illustrated as position "B". As above, if the grip point 518 at position "B" is grippable, the evaluation module 512 can assign position "B" to the grip pattern rank 528 as one of other suitable examples of grip patterns 330 that can grip the target object 112. In one embodiment, the evaluation module 512 can continue to determine grippable grip points 518 in a manner similar to that described above. For example, the evaluation module 512 can determine an offset 604 to move the grip pattern location 530 to another location of the grip point 518, such as position "C". If this grip point 518 is grippable, the evaluation module 512 can assign position “C” to a grip pattern rank 528 as one of other suitable examples of grip patterns 330 capable of gripping the target object 112.

[0147]

[0157] In one embodiment, if the evaluation module 512 determines that any grip points 518 are not capable of being gripped, e.g., within a prohibited area 602, the evaluation module 512 may ignore those grip points 518 and not include those grip points 518 in the grip pattern rank 528.

[0148]

[0158] 7, an exemplary control flow for the online state 406 of the robotic system 100 of FIG 1 is shown in one embodiment of the present invention. In the embodiment shown in FIG 7, the suction control pattern generator 404 of FIG 4 and 5 may be executed such that the grip patterns 330 of FIG 3 and 5, the grip points 518 of FIG 5, the grip pattern ranks 528 of FIG 5, or a combination thereof, for various target objects 112 of FIG 1 are known to the robotic system 100 and stored in the storage unit 206 of FIG 2 as part of the master data 226 of FIG 2.

[0149]

[0159] The robotic system 100 can implement the online state 406 using various functional units of the robotic system 100 of Figures 2, 3 and 4, one or more components external to the robotic system 100, or a combination thereof. In one embodiment, the online state 406 can be implemented using the scan module 502, the object identification module 704, the base plan generation module 706, the base plan execution module 708, the established grip measurement module 716, the base plan execution continue module 720, the iteration count module 722, the object regrip module 724, the base plan execution stop module 726, the storage unit 206, the error generation module 728, or a combination thereof.

[0150]

[0160] In one embodiment, the scan module 502 may be connected to an object identification module 704. The object identification module 704 may be connected to a base plan generation module 706. The base plan generation module 706 may be connected to a base plan execution module 708 and the storage unit 206. The base plan execution module 708 may be connected to an established grip measurement module 716. The established grip measurement module 716 may be connected to an object regrip module 724, a base plan execution and continuation module 720, and an iteration count module 722. The base plan execution and continuation module 720 may be connected to the scan module 502. The iteration count module 722 may be connected to the object regrip module 724 and the base plan execution stop module 726. The base plan execution stop module 726 may be connected to an error generation module 728.

[0151]

[0161] A scan module 502 similar to that described with reference to FIG. 5 may enable the sensor unit 230 of FIG. 2, including one or more imaging devices, such as a two-dimensional (2D) camera, a three-dimensional (3D) camera, an infrared camera, a lidar, a radar, other distance measuring or imaging devices, or a combination thereof, to perform a scanning or imaging function. The scan module 502 may enable scanning of the designated area 114 of FIG. 1 or the work place 116 of FIG. 1 for the target object 112. The scanning may be performed using techniques similar to those described with reference to FIG. 5. Execution of the scan module 502 may then generate similar outputs, including generating the sensor readings 246 of FIG. 2 (e.g., the scan image 504). The scan image 504 may be used to identify or recognize the target object 112 in the online state 406. For example, the scan module 502 may use the sensor unit 230 to generate a scan image 504, such as a digital image, point cloud / depth map, or combination thereof, of the target object 112 within a bin, pallet, box, conveyor belt, truck, or combination thereof. In one embodiment, the scan image 504 may be used to generate a digital model of the target object 112, FIG. 5, that may be used to identify the target object 112 by further components of the robotic system 100. In one embodiment, once the scan is performed, the sensor readings 246, such as the scan image 504, the digital model 540, or combination thereof, as well as control, may be handed over to an object identification module 704.

[0152]

[0162] The object identification module 704 may enable identification of the target object 112 based on the received sensor readings 246 (e.g., the scan image 504). The object identification module 704 may recognize the target object 112 in a process similar to that described with reference to FIG. 5 and the grip point mapping module 508, such as determining geometric shapes in the scan image 504 and comparing the geometric shapes to a set of known shapes or objects. Based on this comparison, the target object 112 may be mapped to the geometric shapes or known objects. If a match is found, the object identification module 704 may generate object parameters 702 that indicate for which known object or geometric shape the target object 112 has been recognized. The object parameters 702 may be variables, flags, or a combination thereof that may be assigned by the object identification module 704 to map the target object 112 to the known object or geometric shape. For example, if the object identification module 704 recognizes the target object 112 as a "box," then object parameters 702 can be set or assigned to the "box" so that further components of the robotic system 100 can recognize that the target object 112 is a "box."

[0153]

[0163] In another embodiment, the object identification module 704 can identify the target object 112 by searching for a tag or label known to the robotic system 100 that can identify the target object 112. For example, the tag or label can include a barcode, a Quick Response (QR) code, a logo, or a combination thereof, that can identify the target object 112. The tag or label can be mapped or associated with the object and recognized by the robotic system 100. The tag or label can be scanned as part of the scanning operation of the scan module 502 and can be included as part of the scanned image 504, the digital model 540, or a combination thereof. Similar to that described with reference to FIG. 5, if a match is found, the object identification module 704 can generate object parameters 702 that indicate what object or geometric shape the target object 112 was recognized as, and can set or assign the object parameters 702 in a manner similar to that described above.

[0154]

[0164] In one embodiment, once the object identification module 704 identifies the target object 112, the object identification module 704 can pass the object parameters 702, the scanned image 704, or a combination thereof, to a base plan generation module 706 to generate a base plan 730 for performing the operation 118 of FIG. 1 on the target object 112. The base plan generation module 706 can generate the base plan 730 for performing the operation 118 on the target object 112 based on the object parameters 702, the scanned image 504, or a combination thereof.

[0155]

[0165] The base plan 730 refers to a sequence of steps required to perform an operation 118, such as gripping, maneuvering, transporting, or a combination thereof, of a target object 112. The base plan 730 may include steps required to grip a target object 112 and transport the target object 112 from one location to another. For example, in one embodiment, the base plan 730 may include the robotic system 100 selecting the target object 112 from a current location to another location according to the scanned image 504.

[0156]

[0166] As part of generating the base plan 730, the robotic system 100 can calculate a set of commands, settings, or combinations thereof for the actuation unit 220 of FIG. 2 that will operate the structural joint 312 of FIG. 3, the arm portion 314 of FIG. 3, the gripping unit 306 of FIG. 3, the suction gripper 308 of FIG. 3, or a combination thereof, to implement the base plan 730. The set of commands, settings, or combinations thereof may be based on one or more constraints, goals, rules, or combinations thereof. For example, the base plan generation module 706 may calculate a set of commands, settings, or combinations thereof for the actuation unit 220 of FIG. 2 that will operate the structural joint 312 of FIG. 3, the arm portion 314 of FIG. 3, the gripping unit 306 of FIG. 3, the suction gripper 308 of FIG. 3, or a combination thereof, to * Algorithm, D * One or more processes, including an algorithm or other grid-based search, may be used to calculate a path through space to move the target object 112 from a pick-up location to a drop location. A set of commands, settings, or combinations thereof may be used to convert this path into a set of commands or settings for the actuation unit 220 using further processes, functions, formulas, conversion tables, or combinations thereof.

[0157]

[0167] Generating the base plan 730 may also include determining which surfaces or locations of the target object 112 can be gripped by the suction gripper 308, the suction cup 324, or a combination thereof. For example, in one embodiment, the base plan 730 may include determining which grip points 518 of the target object 112 are available for gripping, the location and orientation of the grip points 518 relative to the work location 116 from the designated area 114, other objects surrounding the target object 112, or a combination thereof. This determination may be made, for example, by determining whether there are any adjacent objects or surfaces near the grip point 518 that would cause a particular location of the grip point 518 to be unable to be gripped.

[0158]

[0168] In another embodiment, the determination may be made by determining whether there is another object or surface blocking the grip point 518 of the target object 112 such that the grip point 518 cannot be gripped by, for example, the suction gripper 308, the suction cup 324, or a combination thereof. In another embodiment, the determination may be made by determining, for example, that the target object 112 is within a container, box, or pallet and a location of the grip point 518 of the target object 112 can be gripped from a side of the container, box, or pallet, that the grip point 518 on that side of the container, box, or pallet can be used to grip the target object 112, that the target object 112 can be gripped at an angle 338 relative to the container, box, or pallet using the location of the grip point 518, or a combination thereof.

[0159]

[0169] In another embodiment, the determination may be made based on, for example, the orientation of the target object 112. For example, if the target object 112 shifts within a container, box, or pallet such that one or more of its graspable surfaces are at an angle 338 relative to a plane, the base plan generation module 706 may determine which grip points 518 should be gripped and at what instance of the angle 338. As a result, a base plan 730 may be generated to determine which areas or surfaces of the target object 112 may be gripped to properly implement the base plan 730.

[0160]

[0170] In one embodiment, the base plan generation module 706 may further determine not to use one or more instances of the grip pattern 330 based on a determination of which grip points 518 of the target object 112 are blocked by other objects. For example, in one embodiment, if there is an adjacent object or surface that blocks one or more of the grip points 518 of the target object 112, the base plan generation module 706 may determine that a grip pattern 330 associated with a particular location of the grip point 518 cannot be used to grip the target object 112 because the grip pattern 330 cannot be used to grip the target object 112. In one embodiment, for example, if the base plan generation module 706 determines that a surface is blocked but can still be gripped by a certain instance of the grip pattern 330, the base plan generation module 706 may generate a base plan 730 using that instance of the grip pattern 330.

[0161]

[0171] Similarly, if the target object 112 has one or more grip points 518 at an angle 338 relative to another object or plane, the base plan generation module 706 may determine whether an instance of a grip pattern 330 can be used to grip the target object 112 at a particular instance of the angle 338, for example, because the grip pattern 330, suction gripper 308, suction cup 324, or a combination thereof cannot operate or be used at the particular instance of the angle 338. If the base plan generation module 706 determines that one or more grip patterns 330 cannot be used, the base plan generation module 706 may ignore those grip patterns 330 when generating the base plan 730.

[0162]

[0172] In one embodiment, the base plan generation module 706 can further use the grip pattern rank 528 in generating the base plan 730. For example, the base plan generation module 706 can use the grip pattern rank 528 to generate the base plan 730 by assigning the highest ranked configuration of grip patterns 330 for a particular instance of the target object 112 as the grip pattern 330 used in the base plan 730. If the base plan generation module 706 determines that one or more grip patterns 330 cannot be used, the base plan generation module 706 can exclude those grip pattern 330 instances from the base plan 730, but can still maintain the grip pattern rank 528 by assigning the next highest ranked configuration and grip point 518 of an available grip pattern 330 in the grip pattern rank 528 used in the base plan 730 for gripping the target object 112.

[0163]

[0173] Once the base plan generation module 706 generates the base plan 730, control and the base plan 730 may be handed over to the base plan execution module 708. The base plan execution module 708 may enable the execution of the base plan 730 based on enabling the operation of the actuation unit 220 and other functional units of the robotic system 100 according to a set of commands, settings, or combinations thereof. For example, the base plan execution module 708 may activate a first set of actions, commands, instructions, or combinations thereof in the base plan 730. As a specific example, the base plan execution module 708 may enable the operation of the actuation unit 220 to position the grip unit 306, the suction gripper 308, or a combination thereof, in a location or orientation at a starting location for gripping the target object 112. In one embodiment, the starting location may default to the grip point 518 at the center of mass 236 of the target object 112, for example. If the target object 112 cannot be gripped according to the base plan 730, the base plan execution module 708 may enable the actuation unit 220 to move to the next available surface that can grip the target object 112 according to the base plan 730.

[0164]

[0174] The base plan execution module 708 can enable operation of the actuation unit 220 to cause the gripping unit 306, the suction gripper 308, the suction cup 324, or a combination thereof to engage or grip the target object 112. The target object 112 can be engaged or gripped according to the grip pattern rank 528 and the grip pattern 330, as predetermined by the evaluation module 512.

[0165]

[0175] In one embodiment, the base plan execution module 708 can monitor the execution of the base plan 730. For example, if while executing the base plan 730, the base plan execution module 708 determines that it is unable to grip or engage one of the surfaces of the target object 112 or use one of the grip patterns 330 assigned in the base plan 730 to grip the target object 112, the base plan execution module 708 can cycle through the grip patterns 330 and grip pattern ranks 528 to find other examples of grip patterns 330 that can be used to engage or grip the target object 112.

[0166]

[0176] In one embodiment, the base plan execution module 708 may further set, reset, or initialize an iteration counter 732 used to track a certain number of gripping actions. The iteration counter 732 may be a parameter or variable used to keep track of the number of times the base plan execution module 708 enables the operation of the actuation unit 220 to attempt to grip the target object 112. For example, when performing a first grip of the target object 112, the base plan execution module 708 may set the iteration counter 732 to a value of "1". A value of "1" refers to the first attempt to grip the target object 112. The iteration counter 732 may be used to determine whether to continue executing the base plan 730 or to stop executing the base plan 730. The iteration counter 732 is described in more detail below.

[0167]

[0177] In one embodiment, once the base plan execution module 708 enables the actuation unit 220 to engage and grip the target object 112 according to the base plan 730, the base plan execution module 708 further enables the actuation unit 220 to perform an initial lift by enabling movement of the gripping unit 306, the suction gripper 308, or a combination thereof. The engagement and gripping of the target object 112 may be performed according to the principles described with reference to FIG. 3 and in particular with respect to the operation of the gripping unit 306, the suction gripper 308, the suction cup 324, and the sensor unit 230 engaging a surface of the target object 112, establishing a grip on the target object 112, and measuring whether the grip is sufficient to grip the target object 112.

[0168]

[0178] In one embodiment, once the base plan execution module 708 enables the first lift, the iteration counter 732 and control are handed over to the established grip measurement module 716 to enable the robotic system 100 to determine whether the grip 710 established on the target object 112 is sufficient to continue executing the base plan 730.

[0169]

[0179] The established grip measurement module 716 enables measurement of the established grip 710 using various methods illustrated in FIG. 3. The established grip 710 refers to a quantity, variable, measurement, or combination thereof related to the force and torque applied to the target object 112 by the suction gripper 308, the suction cup 324, or a combination thereof. The established grip measurement module 716 may enable measurement of the established grip 710 by obtaining sensor readings 246 from the sensor unit 230 via the communication unit 212 to determine the force or torque being applied to the target object 112 by the suction gripper 308, the suction cup 324, or a combination thereof as a result of the initial lift. The established grip 710 may be an example of a contact measurement 320.

[0170]

[0180] Once the established grip measurement module 716 has obtained the forces and torques being applied to the target object 112, the established grip measurement module 716 can determine whether the forces and torques being applied to the target object 112 by the suction gripper 308 and suction cup 324 at least meet the force threshold 322 of Figure 3 such that the target object 112 is adequately gripped and the base plan 730 can continue to be executed. To make this determination, the established grip measurement module 716 can compare the forces and torques to the force threshold 322 using the same principles as described with reference to Figure 3 to determine whether the target object 112 is sufficiently gripped to perform the operation 118 on the target object 112 in accordance with the base plan 730.

[0171]

[0181] In one embodiment, for example, if the established grip measurement module 716, after comparing the forces and torques, determines that the force and torque applied to the target object 112 is lower than the force threshold 322 and does not meet the force threshold 322 for maintaining a sufficient grip to perform the task 118, control may be passed to the iteration count module 722 to increment the iteration counter 732 and initiate an attempt to re-grip the target object 112.

[0172]

[0182] The iteration count module 722 may increment a value of an iteration counter 732 when an attempt by the robotic system 100 to grip the target object 112 fails. In one embodiment, once the iteration counter 732 is incremented, the iteration count module 722 may further evaluate whether the iteration counter 732 exceeds an iteration threshold 736. The iteration threshold 736 may be a number, value, or parameter that may represent the number of grip patterns 330 in the base plan 730 that are available to grip the target object 112. In one embodiment, while the iteration counter 732 is equal to or less than the iteration threshold 736, there are still grip patterns 330 available to grip the target object 112, and the robotic system 100 may attempt to regrip the target object 112 based on the grip patterns 330 remaining to grip the target object 112. Once the iteration counter 732 is incremented, control may be handed over to the object regrip module 724.

[0173]

[0183] The object regrip module 724 may enable regrip of the target object 112. The object regrip module 724 enables regrip of the target object 112 by enabling the base plan 730 to continue to be implemented based on operating the actuation unit 220 according to a set of commands, settings, or combinations thereof to regrip the target object 112. In one embodiment, if a particular configuration of grip patterns 330 fails to grip the target object 112, the object regrip module 724 may attempt to regrip the target object 112 by searching for viable grip patterns 330 to regrip the target object 112 according to the base plan 730.

[0174]

[0184] The object re-grip module 724 can cycle through the grip patterns 330 in a variety of ways. For example, in one embodiment, after a grip attempt fails, the object re-grip module 724 can attempt to re-grip the target object 112 at the same location or locations of the grip point 518, if possible, using a different configuration of the grip pattern 330. In another embodiment, the object re-grip module 724 can select another location of another grip point 518 of the target object 112 that has been determined to be grippable according to the base plan 730, and attempt to re-grip the target object 112 at the different location of the grip point 518, using the one or more grip patterns 330 associated with that grip point 518.

[0175]

[0185] In one embodiment, the object re-grip module 724 may further attempt to regrip the target object 112 according to the grip pattern rank 528. For example, the object re-grip module 724 may attempt to regrip the target object 112 by cycling through the grip pattern rank 528 from the configuration of the highest ranked grip pattern 330 to the lowest ranked grip pattern 330 and attempting to grip the target object 112 based on the grip pattern 330 along the relevant location of the surface of the target object 112. As a result, the object re-grip module 724 may attempt to regrip the target object 112 based on the grip pattern rank 528 using the grip points 518 and grip patterns 330 that will provide a uniform distribution of force across the target object 112 based on the grip pattern rank 528.

[0176]

[0186] In one embodiment, after attempting to regrip the target object 112, control may again be passed to the established grip measurement module 716 to determine the forces and torques being applied to the target object 112 in a manner similar to that described above. If the attempted regrip still produces forces and torques that do not meet the force threshold 322, the robotic system 100 may attempt to perform a regrip of the target object 112 as described above until the iteration counter 732 exceeds the iteration threshold 736. In one embodiment, once the iteration counter 732 exceeds the iteration threshold 736, control may be passed to the base plan execution stop module 726.

[0177]

[0187] The base plan execution stop module 726 can stop the execution of the base plan 730 based on the iteration counter 732 exceeding the iteration threshold 736. The base plan execution stop module 726 can allow the base plan 730 to terminate by turning off the vacuum hose, the vacuum generator, and the actuation unit 220 of the robotic system 100. In one embodiment, once the base plan execution stop module 726 allows the base plan 730 to terminate, control can be passed to an error generation module 728.

[0178]

[0188] The error generation module 728 may enable transmission of an error 738 based on a failure to grip the target object 112. The error 738 may be a visual or audio message, a signal, a numeric code, or a combination thereof. In one embodiment, the error generation module 728 may transmit the error 738 indicating the grip failure via the communication unit 212 to other functional units of the robotic system 100, a user of the robotic system 100, an external system, or a combination thereof.

[0179]

[0189] To further illustrate, for example, if the grip or re-grip of the target object 112 generates forces and torques equal to or greater than the force threshold 322, the established grip measurement module 716 determines that the forces and torques applied to the target object 112 are strong enough to continue executing the base plan 730, and control is handed over to the base plan execution continuation module 720 to continue executing the base plan 730. Thus, the robotic system 100 can continue executing the base plan 730 according to the remainder of the sequence of commands or settings. For example, the robotic system 100 can transport the target object 112, for example, vertically, horizontally, or a combination thereof, or reorient the target object 112 according to the base plan 730.

[0180]

[0190] The base plan execution and continuation module 720 further enables monitoring of the base plan 730 by continuing to monitor the forces and torques on the target object 112 during the continued execution of the base plan 730. The base plan execution and continuation module 720 may do so during the continued execution of the base plan 730 by connecting to the established grip measurement module 716 and having the established grip measurement module 716 determine whether the established grip 710 on the target object 112 is sufficient to continue execution of the base plan 730. In one embodiment, if the established grip measurement module 716 determines that the forces and torques on the target object 112 fall below the force threshold 322 at any time during the continued execution of the base plan 730, the base plan execution and continuation module 720 may adjust the velocity 340, FIG. 3 and acceleration 342, FIG. 3 of the arm unit 102 such that the forces and torques on the target object 112 are sufficient to grip the target object 112 without dropping it, and re-establish a grip on the target object 112 as described above with reference to FIG. 3.

[0181]

[0191] As an example, if it is determined that the torque on the target object 112 is greater than the force generated by the suction gripper 308 or suction cup 324 to maintain a grip on the target object 112, the base plan execution continuation module 720 can adjust the velocity of the arm unit 102 to compensate for the torque on the target object 112, for example, by reducing the velocity 340 and acceleration 342 of the arm unit 102.

[0182]

[0192] In another embodiment, if one or more suction grippers 308 lose their grip on the target object 112 during execution of the base plan 730, the base plan execution and continuation module 720 can return the target object 112 to the box, bin, or pallet, or to the location from which the target object 112 was picked up, and have the object regrip module 724 attempt to regrip the target object 112. In another embodiment, the base plan execution and continuation module 720 can also adjust the position of the arm unit 102, such as by rotating the arm unit 102, to maintain a grip on the target object 112.

[0183]

[0193] In one embodiment, if the base plan execution continuation module 720 determines that the target object 112 has been safely transported to the destination based on the base plan 730, the base plan execution continuation module 720 may hand over control to the scan module 502 to begin the process again for the next example of the target object 112. In one embodiment, if there are no more examples of the target object 112 for which to generate a base plan 730, the robotic system 100 may stop operating.

[0184]

[0194] The robotic system 100 has been described with an exemplary module function order. The robotic system 100 may separate or order the modules differently than described above. For example, the software 210 of FIG. 2 may include the modules of the robotic system 100. As a specific example, the software 210 may include a scan module 502, an object identification module 704, a base plan generation module 706, a base plan execution module 708, an established grip measurement module 716, a base plan execution continue module 720, an iteration count module 722, an object regrip module 724, a base plan execution stop module 726, an error generation module 728, and associated submodules therein.

[0185]

[0195] 2 may execute the software 210 to operate the modules. For example, the control unit 202 may execute the software to implement the scan module 502, the object identification module 704, the base plan generation module 706, the base plan execution module 708, the established grip measurement module 716, the base plan execution continue module 720, the iteration count module 722, the object regrip module 724, the base plan execution stop module 726, the error generation module 728, and associated sub-modules contained therein.

[0186]

[0196] The modules described in this application may be implemented as instructions stored on a non-transitory computer-readable medium that are executed by the control unit 202. The non-transitory computer-readable medium may include the storage unit 206. The non-transitory computer-readable medium may include non-volatile memory, such as a hard disk drive, a non-volatile random access memory (NVRAM), a solid-state storage device (SSD), a compact disk (CD), a digital video disk (DVD), or a universal serial bus (USB) flash memory device. The non-transitory computer-readable medium may be integrated as part of the robotic system 100 or may be installed as a removable part from the robotic system 100.

[0187]

[0197] 8, a flow chart of a method 800 of operation of a robotic system in one embodiment of the present invention is shown. The method 800 includes receiving sensor readings associated with a target object in box 802, generating a base plan for performing a task on the target object in box 804, where generating the base plan includes determining a grip point and a grip pattern associated with the grip point for gripping the target object based on a designated area, a work location, and a position of the grip point relative to another target object, executing the base plan to perform the task by operating an actuation unit and one or more suction grippers according to a grip pattern rank in box 806 to generate an established grip on the target object, where the established grip is at a grip pattern location associated with the grip pattern used to grip the target object, measuring the established grip in box 808, comparing the established grip to a force threshold in box 810, and re-gripping the target object based on the established grip falling below the force threshold in box 812.

[0188]

[0198] The above detailed description and embodiments of the robotic system 100 are not intended to be exhaustive or to limit the disclosed robotic system 100 to the precise form described above. Although specific examples of the robotic system 100 have been described for illustrative purposes, those skilled in the art will recognize that various equivalent modifications are possible within the scope of the disclosed robotic system 100. For example, while processes or methods are provided in a given order, alternative implementations may perform routines having steps or employ systems having blocks in a different order, and some processes or methods may be deleted, moved, added, subdivided, combined, and / or modified to provide alternative or subcombinations. Each of these processes or methods may be performed in a variety of different ways. Also, while processes or methods are illustrated as being performed sequentially, these processes or blocks may instead be performed or executed in parallel or may be performed at different times.

[0189]

[0199] The resulting methods, processes, apparatus, devices, products and systems are cost-effective, versatile, and precise, and can be implemented by adapting components for ease, efficiency, and economical manufacture, use, and utilization. Another important aspect of embodiments of the present invention is that they beneficially support and serve historical trends of reduced cost, simplified systems, and improved performance.

[0190]

[0200] The above and other aspects of the embodiments herein result in elevating the state of the art to at least the following level. Although the present invention has been described in connection with specific best modes, many alternatives, modifications, and variations will become apparent to those skilled in the art in light of the description herein. Accordingly, all such alternatives, modifications, and variations are intended to be encompassed within the scope of the claims contained herein. All matters described herein or shown in the accompanying drawings are to be interpreted as illustrative and not limiting in any sense.

Claims

1. 1. A method of operating a robotic system, comprising: Identifying an object of interest from the sensor readings; generating a base plan for performing a task of gripping and transporting the target object based on one or more grip patterns for gripping the target object; Implementing the base plan for performing the work; and and adjusting a velocity of the base plan based on the established grip dropping below a force threshold.

2. The method of claim 1 , further comprising releasing the target object and re-gripping the target object based on the established grip falling below the force threshold.

3. The method of claim 1 , further comprising generating an error based on the iteration counter exceeding an iteration threshold.

4. The method of claim 1 , further comprising identifying a grip point for gripping the target object based on a center of mass of the target object.

5. The method of claim 1 , further comprising identifying a grip point for the target object based on a surface contour of the target object.

6. The method of claim 1 , further comprising determining the grip pattern location based on a geometric center of the suction cup and a weighting factor.

7. 10. The method of claim 1, further comprising generating a grip pattern rank for the one or more grip patterns based on a grip that provides stability to the target object for the one or more grip patterns.

8. monitoring the established grip while implementing the base plan; The method of claim 1 , further comprising: generating instructions for adjusting a velocity and acceleration of an arm unit based on the established grip falling below the force threshold.

9. 1. A robot system comprising: Identifying an object of interest from the sensor readings; generating a base plan for performing a task of gripping and transporting the target object based on one or more grip patterns for gripping the target object; Implementing the base plan for performing the work; and adjusting a velocity of the base plan based on the established grip dropping below a force threshold; A robotic system comprising a control unit configured to:

10. The system of claim 9 , wherein the control unit is further configured to release the target object and re-grip the target object based on the established grip falling below the force threshold.

11. The system of claim 9 , wherein the control unit is further configured to generate an error based on an iteration counter exceeding an iteration threshold.

12. The system of claim 9 , wherein the control unit is further configured to identify a grip point for gripping the target object based on a center of mass of the target object.

13. The system of claim 9 , wherein the control unit is further configured to identify a grip point for the target object based on a surface contour of the target object.

14. The system of claim 9 , wherein the control unit is further configured to determine the grip pattern location based on a geometric center of the suction cup and a weighting factor.

15. 10. The system of claim 9, wherein the control unit is further configured to generate a grip pattern rank for the one or more grip patterns based on a grip that provides stability to the target object for the one or more grip patterns.

16. 1. A non-transitory computer readable medium containing instructions for a robotic system, comprising: The instruction: Identifying an object of interest from the sensor readings; generating a base plan for performing a task of gripping and transporting the target object based on one or more grip patterns for gripping the target object; Implementing the base plan for performing the work; and and adjusting a velocity of the base plan based on the established grip dropping below a force threshold.

17. 20. The non-transitory computer-readable medium of claim 16, further comprising: releasing the target object and re-gripping the target object based on the established grip falling below the force threshold.

18. 20. The non-transitory computer readable medium of claim 16 having instructions thereon, further comprising identifying a grip point for gripping the target object based on a center of mass of the target object.

19. 20. The non-transitory computer readable medium of claim 16 having instructions thereon, further comprising determining a grip pattern location based on a geometric center of the suction cup and a weighting factor.

20. 20. The non-transitory computer readable medium of claim 16, having instructions thereon, further comprising generating a grip pattern rank for the one or more grip patterns based on a grip that provides the one or more grip patterns with stability for the target object.

Images