System and method for controlling a robot using parallelizable evaluation of grasp configurations
The method and system optimize robot control on production lines by evaluating part and equipment states to select optimal grasp configurations, addressing real-time decision-making challenges and ensuring smooth operation.
Patent Information
- Application Number
- PCT/CA2025/051002
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-07-23
- Filing Date
- 2025-07-23
- Publication Date
- 2026-01-29
AI Technical Summary
The challenge in optimizing production lines lies in the efficient prioritization and movement of robots that handle diverse parts with varying geometries and material properties, requiring real-time decision-making to prevent stalls and ensure smooth operation.
A method and system for controlling robots on production lines that involve collecting and evaluating part and equipment states to select optimal grasp configurations, considering safety factors, collision avoidance, and predefined rules, allowing continuous and uninterrupted operation.
Enables efficient and uninterrupted robot operation by reducing processing delays and optimizing grasp configurations, ensuring stable and safe handling of diverse parts across complex environments.
Smart Images

Figure CA2025051002_29012026_PF_FP_ABST
Abstract
Description
TITLE OF THE INVENTIONSYSTEM AND METHOD FOR CONTROLLING A ROBOT USING PARALLELIZABLE EVALUATION OF GRASP CONFIGURATIONSPRIORITY STATEMENT
[0001] This patent application claims priority on US patent application No. 63 / 674,406 filed on July 23, 2024.TECHNICAL FIELD
[0002] The present invention relates to production lines, more particularly, to controlling a robot on a production line.BACKGROUND
[0003] In manufacturing environments, the efficiency of a production line is of importance. Production lines are often designed to move, process or assemble parts in a streamlined and timely manner. Efficiency in this context translates into reduced production costs, increased throughput, and improved overall productivity.
[0004] One of the challenges in optimizing a production line lies in the task of prioritizing and optimizing the movements of robots responsible for picking and moving parts. Robots may have to navigate complex environments, handle a variety of parts with differing geometries and material properties, and interact with multiple pieces of equipment. The need for real-time decision puts upward pressure on processing power requirements and downward pressure on the number of options that can be evaluated. When the robot is unable to quickly determine the best course of action, the production line may stall momentarily. Ensuring that robots can efficiently prioritize tasks and optimize their movements is necessary to maintaining the smooth operation of the production line. The system and methods described herein may address some of these challenges.SUMMARY
[0005] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
[0006] In a first aspect, the technique described herein relates to a method for controlling a robot on a production line. A state of the production line may be collected, including one or more part states, one or more equipment states, and a robot state. An eligible grasp configuration subset may be assembled from a plurality of grasp configurations and at least one part state of the one or more part states. A compatible grasp configuration subset may be assembled from the eligible grasp configuration subsets and at least one equipment state from the one or more equipment states. An optimal grasp configuration may be selected from the compatible grasp configuration subset and the robot state. The optimal grasp configuration may be executed with the robot to perform at least one of picking action and / or placing action of a part.
[0007] In embodiments, at least one part state from the one or more part states may include a geometry of the part, a material of the part, and / or a location of the part. The geometry, the material and / or the location of the part may be evaluating when assembling the eligible grasp configuration subsets.
[0008] In embodiments, a safety factor indicative of a likelihood to accidentally drop the part may be computed and at least one grasp configuration from the plurality of grasp configurations may be selected when the safety factor is above a minimum safety threshold.
[0009] In embodiments, the one or more part states may include a temperature measurement, and / or a relative humidity measurement. The safety factor may be adjusted according to the temperature measurement and / or the relative humidity measurement.
[0010] In embodiments, the grasp configurations may include a relative positioning coordinate of a grasp point on the part and a grasp orientation, and the safety factor may be evaluated when the part is held by the robot from the relative positioning coordinate and the grasp orientation. Optionally, the relative positioning coordinate of the grasp point may include a selected anchor point within a plurality of anchor points on the part, and a distance relative to the selected anchor point. The distance relative to the selected anchor point may be proportional to a dimension of the part, thereby allowing the relative positioning coordinate to adapt to a variety of part geometries.
[0011] In embodiments, the compatible grasp configuration subset may be represented as a Boolean sequence. The compatible grasp configuration subset may be assembled across two or more equipment states by performing an intersection operation over the Boolean sequence of the two or more equipment states.
[0012] In embodiments, an acceleration of the robot may be adjusted based on the safety factor.
[0013] In embodiments, the state of the production line may include a collision avoidance parameter describing potential collisions within a proximity of the robot. At least one grasp configuration from the plurality of grasp configurations may be selected when the collision avoidance parameter suggests avoidance of a collision.
[0014] In embodiments, the state of the production line may include a predefined rule for handling at least one of a specific part with a specific feature, and a specific process comprising a specific assembly step. At least one grasp configuration may be selected from the plurality of grasp configurations when the predefined rule is compatible with the grasp configuration. Optionally, the specific feature may be a hole, a groove, and / or an irregular shape.
[0015] In embodiments, the method may be performed repeatably in real time and the robot may operates continuously and uninterruptedly.
[0016] In embodiments, an equipment state of the one or more equipment states may include a geometrical configuration of an equipment with a workspace surface and a reachability of the equipment from the robot. At least one eligible grasp configuration may be selected from the eligible grasp configuration subsets resulting in the part being stable and operational therewith when placed onto the workspace surface across the reachability of the equipment. Optionally, the geometrical configuration may include one or more vertical obstacles and at least one eligible grasp configuration may be selected from the eligible grasp configuration subsets when the one or morevertical obstacles suggest avoidance of a collision therewith when transporting the part onto the workspace surface across the reachability of the equipment.
[0017] In embodiments, the equipment state may include a current part occupancy of the workspace surface. At least one grasp configuration from the eligible grasp configuration subsets may be discarded when resulting in the part being unsafe when placed therewith onto the current part occupancy of the workspace surface.
[0018] In embodiments, the state of the production line may include a priority list and an optimization strategy. For a compatible grasp configuration in the compatible grasp configuration subset, a compatible grasp configuration score may be computed according to the optimization strategy, the priority list and the robot state, thereby optimizing the optimal grasp configuration according to the optimization strategy. Optionally, the compatible grasp configuration score may be compared against a second score of a second grasp configuration. The compatible grasp configuration may be selected from the compatible grasp configuration subset when the compatible grasp configuration score is higher than the second score of the second grasp configuration. Optionally, the optimization strategy may optimize an equipment usage score, a throughput score, a safety score, a minimized travel time score, and / or a clearance score. Optionally, the equipment usage score, the throughput score, the safety score and / or the clearance score may be adjusted according to a diminishing marginal returns function. Optionally, the priority list may include transferring the part to a processing equipment, transferring the part to an accumulation equipment and / or transferring the part to a conveying equipment.
[0019] In embodiments, the grasp configurations may include a plurality of engagement points and a plurality of engagement point activation parameters. The plurality of engagement points may identify one or more locations of engagement between the part and one or more gripping tools. The plurality of engagement point activation parameters may activate the one or more gripping tools. At least one engagement point activation parameter from the plurality of engagement point activation parameters may be disabled, preventing a second part proximate to the part from being picked. Optionally, the state of the production line may include a predefined rule for handling a specific part with a specific feature consisting of a weak engagement point. At least one engagement point activation parameters from the plurality of engagement point activation parameters may be disabled to avoiding engaging the weak engagement point with one or more gripping tools. Optionally, the plurality of engagement point activation parameters may be represented as a Boolean sequence and the part and the second part may be held by performing a union operation over the Boolean sequence. Optionally, the plurality of engagement points may include at least 8 engagement points.
[0020] In embodiments, at least one gripping tool from the one or more gripping tools uses a pressure differential to engage the part and wherein the pressure differential is generated according to the plurality of engagement point activation parameters.
[0021] In embodiments, the robot state may include one or more previous grasp configurations of one or more other currently held parts. One or more compatible grasp configurations may be discarded when posing a risk of dropping at least one of the one or more other currently held parts when picking the part.
[0022] In embodiments, at least one gripping tool within the one or more gripping tools may be configured to retractably extend and the grasp configuration may include one or more extension parameters. The at least one gripping tool may be positioned according to the one or more extension parameters, thereby allowing the part and the one or more other currently held parts to overlap.
[0023] In embodiments, a risk of a collision of the part may be computed when transporting the one or more other currently held parts. The optimal grasp configuration may be rejected when the risk of collision above a risk of collision threshold.
[0024] In embodiments, the one or more equipment states may include an equipment state for any one of a doweling jig station, a welding station, a painting and coating station, an inspection station, a deburring station, a gluing and sealing station, an accumulation table, a storage station, a palletizing station, a conveyor, a transfer station, a turntable, a nesting router, a pod and rail router, a vertical CNC, a CNC-Drilling and dowel insertion machine, a horizontal boring machine, a panel dividing saw, a hardware insertion, a waterjet cutting center, a disk cutting center and other CNC machining centers, an edgebanding machine, a sanding machine, a polishing machine, an assembly system and / or a storage and retrieval system.
[0025] In a second aspect, the technique described herein relates to a system for controlling a robot on a production line. The system may include one or more parts with one or more part states, one or more processing equipment with one or more equipment states, a robot with a robot state, and one or more processors. The one or more processors may collect a state of the production line comprising the one or more part states, the one or more equipment states and the robot state. An eligible grasp configuration subsets may be assembled from a plurality of grasp configurations and at least one part state of the one or more part states. A compatible grasp configuration subset may be assembled from the eligible grasp configuration subsets and at least one equipment state from the one or more equipment states. An optimal grasp configuration may be selected from the compatible grasp configuration subset and the robot state. The robot may execute at least one of a picking action and a placing action of one or more parts using the optimal grasp configuration.
[0026] In embodiments, at least one part state from the one or more part states may include a geometry of the part, a material of the part and / or a location of the part. The geometry, the material and / or the location of the part may be evaluated when assembling the eligible grasp configuration subsets.
[0027] In embodiments, the one or more processors may compute a safety factor indicative of a likelihood to accidentally drop the part and select at least one grasp configuration from the plurality of grasp configurations when the safety factor is above a minimum safety threshold.
[0028] In embodiments, the system may include a temperature sensor and / or a humidity sensor for collecting a temperature measurement and / or a relative humidity measurement. The safety factor may be adjusted according to the temperature measurement and / or the relative humidity measurement.
[0029] In embodiments, the system may include a positioning module configured to capture a relative positioning coordinate of a grasp point and a grasp orientation. The safety factor may be evaluated when the part is held by therobotfrom the relative positioning coordinate and the grasp orientation. Optionally, the relative positioning coordinate of the grasp point may include a selected anchor point within a plurality of anchor points on the part, and a distance relative to the selected anchor point. The distance relative to the selected anchor point may be proportional to a dimension of the part, thereby allowing the relative positioning coordinate to adapt to a variety of part geometries.
[0030] In embodiments, the compatible grasp configuration subset may be represented as a Boolean sequence. The compatible grasp configuration subset may be assembled across two or more equipment states by performing an intersection operation over the Boolean sequence of the two or more equipment states.
[0031] In embodiments, an acceleration of the robot may be adjusted based on the safety factor.
[0032] In embodiments, the state of the production line may include a collision avoidance parameter describing potential collisions within a proximity of the robot. At least one grasp configuration from the plurality of grasp configurations may be selected when the collision avoidance parameter suggests avoidance of a collision.
[0033] In embodiments, the state of the production line may include a predefined rule for handling at least one of a specific part with a specific feature, and a specific process comprising a specific assembly step. At least one grasp configuration may be selected from the plurality of grasp configurations when the predefined rule is compatible with the grasp configuration. Optionally, the specific feature may be a hole, a groove, and / or an irregular shape.
[0034] In embodiments, the one or more processors may collect the state of the production line, assemble the eligible grasp configuration subsets, assemble the compatible grasp configuration subset and select the optimal grasp configuration repeatably in real time. The robot may operate continuously and uninterruptedly.
[0035] In embodiments, an equipment state of the one or more equipment states may include a geometrical configuration of an equipment with a workspace surface and a reachability of the equipment from the robot. At least one eligible grasp configuration may be selected from the eligible grasp configuration subsets resulting in the part being stable and operational therewith when placed onto the workspace surface across the reachability of the equipment. Optionally, the geometrical configuration may include one or more vertical obstacles and at least one eligible grasp configuration may be selected from the eligible grasp configuration subsets when the one or more vertical obstacles suggest avoidance of a collision therewith when transporting the part onto the workspace surface across the reachability of the equipment.
[0036] In embodiments, the equipment state may include a current part occupancy of the workspace surface. At least one grasp configuration from the eligible grasp configuration subsets may be discarded when resulting in the part being unsafe when placed therewith onto the current part occupancy of the workspace surface.
[0037] In embodiments, the state of the production line may include a priority list and an optimization strategy. For a compatible grasp configuration in the compatible grasp configuration subset, a compatible grasp configuration score may be computed according to the optimization strategy, the priority list and the robot state, thereby optimizing the optimal grasp configuration according to the optimization strategy. Optionally, the one or more processors, when selecting the optimal grasp configuration, may compare the compatible grasp configuration score against a second score of a second grasp configuration and select the compatible grasp configuration from the compatible graspconfiguration subset when the compatible grasp configuration score is higher than the second score of the second grasp configuration. Optionally, the optimization strategy may optimize an equipment usage score, a throughput score, a safety score, a minimized travel time score, and / or a clearance score. Optionally, the equipment usage score, the throughput score, the safety score and / or the clearance score may be adjusted according to a diminishing marginal returns function. Optionally, the processing equipment may include at least one of a processing equipment, an accumulation equipment, and a conveying equipment. The priority list may include transferring the part to the processing equipment, transferring the part to the accumulation equipment and / or transferring the part to the conveying equipment.
[0038] In embodiments, the robot may include one or more gripping tools configured to engage the one or more parts. The grasp configurations may include a plurality of engagement points and a plurality of engagement point activation parameters. The plurality of engagement points may identify one or more locations of engagement between the part and the one or more gripping tools. The plurality of engagement point activation parameters may activate the one or more gripping tools. At least one engagement point activation parameter from the plurality of engagement point activation parameters may be disabled, preventing a second part proximate to the part from being picked. Optionally, the state of the production line may include a predefined rule for handling a specific part with a specific feature consisting of a weak engagement point. At least one engagement point activation parameters from the plurality of engagement point activation parameters may be disabled to avoiding engaging the weak engagement point with one or more gripping tools. Optionally, the plurality of engagement point activation parameters may be represented as a Boolean sequence and the part and the second part may be held by performing a union operation over the Boolean sequence. Optionally, the plurality of engagement points may include at least 8 engagement points.
[0039] The system of claim wherein at least one gripping tool from the one or more gripping tools comprises a vacuum system, uses a pressure differential to engage the part and wherein the pressure differential is generated according to the plurality of engagement point activation parameters.
[0040] In embodiments, the robot state may include one or more previous grasp configurations of one or more other currently held parts. One or more compatible grasp configurations may be discarded when posing a risk of dropping at least one of the one or more other currently held parts when picking the part.
[0041] In embodiments, at least one gripping tool within the one or more gripping tools may comprise an actuator configured to retractably extend the gripping tools and the grasp configuration may include one or more extension parameters. The one or more processors, when executing the optimal grasp configuration, may position the at least one gripping tool according to the one or more extension parameters, thereby allowing the part and the one or more other currently held parts to overlap.
[0042] In embodiments, the one or more processors may compute a risk of a collision of the part when transporting the one or more other currently held parts and reject the optimal grasp configuration when the risk of collision above a risk of collision threshold.
[0043] In embodiments, the one or more equipment states may include an equipment state for any one of a doweling jig station, a welding station, a painting and coating station, an inspection station, a deburring station, a gluing and sealing station, an accumulation table, a storage station, a palletizing station, a conveyor, a transfer station, a turntable, a nesting router, a pod and rail router, a vertical CNC, a CNC-Drilling and dowel insertion machine, a horizontal boring machine, a panel dividing saw, a hardware insertion, a waterjet cutting center, a disk cutting center and other CNC machining centers, an edgebanding machine, a sanding machine, a polishing machine, an assembly system and / or a storage and retrieval system.
[0044] In embodiments, the system may comprise a general-purpose processing unit with at least one of the one or more processors, a programmable logic controller unit with a least one of the one or more processors and a robot controller unit with at least one of the at least one of the one or more processors.
[0045] In a third aspect, the technique described herein relates to a non-transitory computer-readable medium storing a set of instructions for controlling a robot on a production line, the set of instructions comprising one or more instructions. When executed by one or more processors, a state of the production line may be collected, including one or more part states, one or more equipment states, and a robot state. An eligible grasp configuration subset may be assembled from a plurality of grasp configurations and at least one part state of the one or more part states. A compatible grasp configuration subset may be assembled from the eligible grasp configuration subsets and at least one equipment state from the one or more equipment states. An optimal grasp configuration may be selected from the compatible grasp configuration subset and the robot state. The optimal grasp configuration may be executed with the robot to perform at least one of picking action and / or placing action of a part.
[0046] In embodiments, at least one part state from the one or more part states may include a geometry of the part, a material of the part, and / or a location of the part. The geometry, the material and / or the location of the part may be evaluating when assembling the eligible grasp configuration subsets.
[0047] In embodiments, a safety factor indicative of a likelihood to accidentally drop the part may be computed and at least one grasp configuration from the plurality of grasp configurations may be selected when the safety factor is above a minimum safety threshold.
[0048] In embodiments, the one or more part states may include a temperature measurement, and / or a relative humidity measurement. The safety factor may be adjusted according to the temperature measurement and / or the relative humidity measurement.
[0049] In embodiments, the grasp configurations may include a relative positioning coordinate of a grasp point on the part and a grasp orientation, and the safety factor may be evaluated when the part is held by the robot from the relative positioning coordinate and the grasp orientation. Optionally, the relative positioning coordinate of the grasp point may include a selected anchor point within a plurality of anchor points on the part, and a distance relative to the selected anchor point. The distance relative to the selected anchor point may be proportional to a dimension of the part, thereby allowing the relative positioning coordinate to adapt to a variety of part geometries.
[0050] In embodiments, the compatible grasp configuration subset may be represented as a Boolean sequence. The compatible grasp configuration subset may be assembled across two or more equipment states by performing an intersection operation over the Boolean sequence of the two or more equipment states.
[0051] In embodiments, an acceleration of the robot may be adjusted based on the safety factor.
[0052] In embodiments, the state of the production line may include a collision avoidance parameter describing potential collisions within a proximity of the robot. At least one grasp configuration from the plurality of grasp configurations may be selected when the collision avoidance parameter suggests avoidance of a collision.
[0053] In embodiments, the state of the production line may include a predefined rule for handling at least one of a specific part with a specific feature, and a specific process comprising a specific assembly step. At least one grasp configuration may be selected from the plurality of grasp configurations when the predefined rule is compatible with the grasp configuration. Optionally, the specific feature may be a hole, a groove, and / or an irregular shape.
[0054] In embodiments, collecting the state of the production line, assembling the eligible grasp configuration subsets, assembling the compatible grasp configuration subset and selecting the optimal grasp configuration may be performed repeatably in real time and the robot may operate continuously and uninterruptedly.
[0055] In embodiments, an equipment state of the one or more equipment states may include a geometrical configuration of an equipment with a workspace surface and a reachability of the equipment from the robot. At least one eligible grasp configuration may be selected from the eligible grasp configuration subsets resulting in the part being stable and operational therewith when placed onto the workspace surface across the reachability of the equipment. Optionally, the geometrical configuration may include one or more vertical obstacles and at least one eligible grasp configuration may be selected from the eligible grasp configuration subsets when the one or more vertical obstacles suggest avoidance of a collision therewith when transporting the part onto the workspace surface across the reachability of the equipment.
[0056] In embodiments, the equipment state may include a current part occupancy of the workspace surface. At least one grasp configuration from the eligible grasp configuration subsets may be discarded when resulting in the part being unsafe when placed therewith onto the current part occupancy of the workspace surface.
[0057] In embodiments, the state of the production line may include a priority list and an optimization strategy. For a compatible grasp configuration in the compatible grasp configuration subset, a compatible grasp configuration score may be computed according to the optimization strategy, the priority list and the robot state, thereby optimizing the optimal grasp configuration according to the optimization strategy. Optionally, the compatible grasp configuration score may be compared against a second score of a second grasp configuration. The compatible grasp configuration may be selected from the compatible grasp configuration subset when the compatible grasp configuration score is higher than the second score of the second grasp configuration. Optionally, the optimization strategy may optimize an equipment usage score, a throughput score, a safety score, a minimized travel time score, and / or a clearance score. Optionally, the equipment usage score, the throughput score, the safety score and / or the clearance score may be adjusted according to a diminishing marginal returns function. Optionally, the priority list may includetransferring the part to a processing equipment, transferring the part to an accumulation equipment and / or transferring the part to a conveying equipment.
[0058] In embodiments, the grasp configurations may include a plurality of engagement points and a plurality of engagement point activation parameters. The plurality of engagement points may identify one or more locations of engagement between the part and one or more gripping tools. The plurality of engagement point activation parameters may activate the one or more gripping tools. At least one engagement point activation parameter from the plurality of engagement point activation parameters may be disabled, preventing a second part proximate to the part from being picked. Optionally, the state of the production line may include a predefined rule for handling a specific part with a specific feature consisting of a weak engagement point. At least one engagement point activation parameters from the plurality of engagement point activation parameters may be disabled to avoiding engaging the weak engagement point with one or more gripping tools. Optionally, the plurality of engagement point activation parameters may be represented as a Boolean sequence and the part and the second part may be held by performing a union operation over the Boolean sequence. Optionally, the plurality of engagement points may include at least 8 engagement points.
[0059] In embodiments, at least one gripping tool from the one or more gripping tools uses a pressure differential to engage the part and wherein the pressure differential is generated according to the plurality of engagement point activation parameters.
[0060] In embodiments, the robot state may include one or more previous grasp configurations of one or more other currently held parts. One or more compatible grasp configurations may be discarded when posing a risk of dropping at least one of the one or more other currently held parts when picking the part.
[0061] In embodiments, at least one gripping tool within the one or more gripping tools may be configured to retractably extend and the grasp configuration may include one or more extension parameters. The at least one gripping tool may be positioned according to the one or more extension parameters, thereby allowing the part and the one or more other currently held parts to overlap.
[0062] In embodiments, a risk of a collision of the part may be computed when transporting the one or more other currently held parts. The optimal grasp configuration may be rejected when the risk of collision above a risk of collision threshold.
[0063] In embodiments, the one or more equipment states may include an equipment state for any one of a doweling jig station, a welding station, a painting and coating station, an inspection station, a deburring station, a gluing and sealing station, an accumulation table, a storage station, a palletizing station, a conveyor, a transfer station, a turntable, a nesting router, a pod and rail router, a vertical CNC, a CNC-Drilling and dowel insertion machine, a horizontal boring machine, a panel dividing saw, a hardware insertion, a waterjet cutting center, a disk cutting center and other CNC machining centers, an edgebanding machine, a sanding machine, a polishing machine, an assembly system and / or a storage and retrieval system.BRIEF DESCRIPTION OF THE DRAWINGS
[0064] Further features and exemplary advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the appended drawings, in which:
[0065] Figure 1A is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line in accordance with the teachings of the present invention;
[0066] Figure 1 B is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line executed by a general-purpose processing unit, a programmable logic controller, and a robot controller, in accordance with the teachings of the present invention;
[0067] Figure 1C is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line executed by a general-purpose processing unit and a robot controller, in accordance with the teachings of the present invention;
[0068] Figure 1 D is a flow diagram depicting another exemplary embodiment of a method for controlling a robot on a production line executed by a general-purpose processing unit and a robot controller, in accordance with the teachings of the present invention;
[0069] Figure 2 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving evaluation of geometry, material and location of parts, in accordance with the teachings of the present invention;
[0070] Figure 3 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor, in accordance with the teachings of the present invention;
[0071] Figure 4 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor adjusted according to temperature and / or humidity, in accordance with the teachings of the present invention;
[0072] Figure 5 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor evaluated at a relative location and orientation, in accordance with the teachings of the present invention;
[0073] Figure 6 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor used to adjust an acceleration of the robot, in accordance with the teachings of the present invention;
[0074] Figure 7 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving avoiding collisions, in accordance with the teachings of the present invention;
[0075] Figure 8 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving predefined rules, in accordance with the teachings of the present invention;
[0076] Figure 9 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving stable placement of parts, in accordance with the teachings of the present invention;
[0077] Figure 10 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving avoidance of vertical obstacles, in accordance with the teachings of the present invention;
[0078] Figure 11 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving safe placement of parts, in accordance with the teachings of the present invention;
[0079] Figure 12 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving optimizations, in accordance with the teachings of the present invention;
[0080] Figure 13 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving comparing the compatible grasp configurations, in accordance with the teachings of the present invention;
[0081] Figure 14 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving discriminating the picked parts, in accordance with the teachings of the present invention;
[0082] Figure 15 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving avoidance of weak engagement points, in accordance with the teachings of the present invention;
[0083] Figure 16 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving reducing risk of dropping the part, in accordance with the teachings of the present invention;
[0084] Figure 17 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving overlapping picked parts, in accordance with the teachings of the present invention;
[0085] Figure 18 is a flow diagram depicting an exemplary embodiment of a method for controlling a robot on a production line involving avoidance of collision when transporting a part, in accordance with the teachings of the present invention; and
[0086] Figure 19 is a block diagram depicting an exemplary system for controlling a robot on a production line, in accordance with the teachings of the present invention.DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0087] Efficient control of robots in a production line is important for efficiency purposes. When the robot is monitored and requires real time adjustment, operations may be slowed or stalled. For example, when a gripper is designed to pick differently shaped objects, the gripper may need to adjust the grip after lowering the arm onto the objects. When using vacuum to pick up objects, it may be necessary to detect which zones of the gripper are engaging with the object, such that the other zones are disabled, either to avoid accidentally picking another object, or to avoid the vacuum from escaping. Although these kinds of adjustments may be performed by monitoring andperforming real-time adjustments, delays may be necessary to ensure that the right decisions are made with reasonable certainty.
[0088] A first aspect of the techniques described herein relates to a method, a system and a non-transitory computer-readable medium for controlling a robot 40 on a production line. Reference is now made to the drawings in which the a flow diagram of Figure 1 A depicts an exemplary embodiment of a method for controlling a robot on a production line in accordance with the teachings of the present invention, Figure 1 B depicts an exemplary embodiment of a method for controlling a robot on a production line executed by a general-purpose processing unit, a programmable logic controller, and a robot controller, Figure 1C depicts an exemplary embodiment of a method for controlling a robot on a production line executed by a general-purpose processing unit and a robot controller, and Figure 1 D depicts another exemplary embodiment of a method for controlling a robot on a production line executed by a general-purpose processing unit and a robot controller. The flow diagram of Figure 2 depicts an exemplary embodiment of a method for controlling a robot on a production line involving evaluation of geometry, material and location of parts. The flow diagram of Figure 3 depicts an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor. The flow diagram of Figure 4 depicts an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor adjusted according to temperature and / or humidity. The flow diagram of Figure 5 depicts an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor evaluated at a relative location and orientation. The flow diagram of Figure 6 depicts an exemplary embodiment of a method for controlling a robot on a production line involving a safety factor used to adjust an acceleration of the robot. The flow diagram of Figure 7 depicts an exemplary embodiment of a method for controlling a robot on a production line involving avoiding collisions. The flow diagram of Figure 8 depicts an exemplary embodiment of a method for controlling a robot on a production line involving predefined rules. The flow diagram of Figure 9 depicts an exemplary embodiment of a method for controlling a robot on a production line involving stable placement of parts. The flow diagram of Figure 10 depicts an exemplary embodiment of a method for controlling a robot on a production line involving avoidance of vertical obstacles. The flow diagram of Figure 11 depicts an exemplary embodiment of a method for controlling a robot on a production line involving safe placement of parts. The flow diagram of Figure 12 depicts an exemplary embodiment of a method for controlling a robot on a production line involving optimizations. The flow diagram of Figure 13 depicts an exemplary embodiment of a method for controlling a robot on a production line involving comparing the compatible grasp configurations. The flow diagram of Figure 14 depicts an exemplary embodiment of a method for controlling a robot on a production line involving discriminating the picked parts. The flow diagram of Figure 15 depicts an exemplary embodiment of a method for controlling a robot on a production line involving avoidance of weak engagement points. The flow diagram of Figure 16 depicts an exemplary embodiment of a method for controlling a robot on a production line involving reducing risk of dropping the part. The flow diagram of Figure 17 depicts an exemplary embodiment of a method for controlling a robot on a production line involving overlapping picked parts. The flow diagram of Figure 18 depicts an exemplary embodiment of a method for controlling a robot on a production line involving avoidance of collision when transporting a part. The block diagram of Figure 19 depicts an exemplary system for controlling a robot on a production line.
[0089] In one embodiment, the parts may comprise wood panels used in the assembly of furniture. The parts may include arbitrary shapes, such as concave, convex, curved, of various numbers of sides and shapes, and either hollowed or solid. In certain instances, the parts may be elongated. In the production line, the parts may be delivered to the robot 40 in a mixture of shapes, both large and small, in a nesting configuration or positioned proximate to one another. In one embodiment, the gripping tools 100 may be used to pick, hold, and place the parts using a pressure differential generated according to the plurality of engagement point activation parameters 415.
[0090] In other embodiments, the method 1000 may be applied to other industrial sectors involving different types of parts and materials. For instance, the method 1000 may be applied in the automotive industry, where parts may include metal sheets used for car body assembly with complex geometries and varying thicknesses. Handling of these parts may be optimized using robotic arms equipped with magnetic grippers or pneumatic clamps. In the electronics manufacturing sector, parts may consist of delicate circuit boards and components made from materials such as silicon, plastic, and composites. Robots 40 may utilize vacuum suction cups to handle parts without causing damage. In the pharmaceutical industry, parts may include glass vials, plastic syringes, and blister packs. In the food processing industry, parts may include varied food items such as fruits, vegetables, and packaged goods. The method 1000 may be employed with various types of robots 40 including articulated robots, SCARA robots, delta robots, and gantry, each equipped with different types of robotic arms tailored to the specific requirements of their respective applications. Techniques described may be adapted to a wide range of parts, industries, equipment, and gripping tools 100.
[0091] Broadly, a plurality of grasp configurations 400 may be predefined, describing a quantity of interesting grasping scenarios. In embodiments, the plurality of grasp configurations 400 may be represented as a table wherewith the rows may represent a grasp configuration, and the columns may provide the attributes of the grasp configuration. The table of grasp configurations 400 may be shared with multiple processing equipment 60, allowing the configuration to refer to a grasp configuration by index. As will be further explained hereinbelow, the selection may be represented as a sequence of bits, where each bit may specify whether a configuration is eligible. For example, a sequence of 16 bytes may be used to communicate a selection within up to 128 grasp configurations 400.
[0092] In embodiments, using various approaches that will be further presented hereinbelow, it was found that interesting grasping scenarios may be possible with as few as 30 to 40 grasp configurations 400, and complex embodiments may be possible with 100 to 150 grasp configurations 400. These estimates may serve as illustrative examples, and more sophisticated embodiments may implement a larger number of grasp configurations 400 with limited impact on the complexity of the method 1000 given the efficient representation and handling thereof. Additionally, several aspects of the method 1000 may be executed in parallel, including the evaluation 1252 of each grasp configuration.
[0093] Broadly, a state of the production line 500 may be collected 1100, including grasp configurations 400. From the plurality of grasp configurations 400 and other constraints of the state of the production line 500, an eligible grasp configuration subset 402 may be assembled 1200. From the eligible grasp configuration subset 402 and other constraints of the state of the production line 500, a compatible grasp configuration subset 404 may be assembled1300. From the compatible grasp configuration subset 404, an optimal grasp configuration 406 may be selected 1400 and executed 1500 with a robot 40, thereby causing the robot 40 to perform a picking, holding, or placing action. Under most circumstances, the technique reduces the search domain and converges with each step towards the optimal grasp configuration 406 using separation of concerns. While the technique may allow for reversal of progress, such as when the robot 40 refuses an optimal grasp configuration 406 and a new one set must be selected 1400 from the compatible grasp configuration subset 404, in a typical embodiment, the reversal of operations may not be part of the typical execution and would happen exceptionally. Typically, the technique may reach the optimal grasp configuration 406 in predictable time, allowing for the technique to be performed repeatably in real time and such that the robot 40 may operate continuously and uninterruptedly.
[0094] The techniques described herein may take the form of a method 1000, a system 2000, or a non-transitory computer-readable medium, such as software. When implemented as a system 2000, the method 1000 may be executed by one or more processors 210. In one embodiment, the processors 210 may be distributed across different devices for practical purposes, as depicted in Figure 1 B, Figure 1C and Figure 1 D. For example, a general- purpose processing unit 250, such as a computer, workstation, server, or similar computing devices, may be used to collect 1100 the state of the production line 500 and assemble 1200 the eligible grasp configuration subset 402. A general-purpose processing unit 250 may be well-suited for scenarios that involve connecting to various monitoring devices, including cameras, sensors, and networked devices, such as configuration servers. A general- purpose processing unit 250 may also be more cost-effective for tasks requiring a large amount of memory or accelerated devices, such as GPUs, for processing complex geometries.
[0095] In one embodiment, the entire method may be executed on a general-purpose processing unit 250. In other embodiments, assembling 1300 the compatible grasp configuration subset 404 may be executed on one or multiple specialized processing units such as a programmable logic controller (PLC) unit 260. In some embodiments, it may be advantageous to execute assembling 1300 the compatible grasp configuration subset 404 and / or selecting 1400 the optimal grasp configuration 406 on the programmable logic controller unit 260.
[0096] A programmable logic controller 260 may be hardware-based, comprising a physical device with dedicated circuits and components designed for industrial control applications. Hardware-based programmable logic controllers 260 may include an enclosure housing processors 210, memory, input / output modules, and power supply units, tailored for robustness and reliability in industrial environments. Real-time control over machinery and automated processes may be provided through direct interfacing with sensors and actuators. Alternatively, a programmable logic controller 260 may be software-based, referring to applications or programs executing logic control tasks on a general-purpose computing device 250. Software-based programmable logic controllers 260 may simulate the functionality of traditional hardware programmable logic controllers 260 by processing inputs and managing outputs through virtual interfaces. These applications may be deployed on computers, embedded systems, or virtual machines connecting to remote input and output systems over network protocols. A wide range of implementations between a purely hardware-based and purely software-based programmable logic controller 260 may exist without impacting the teaching of the present disclosure.
[0097] A programmable logic controller may offer better guarantees with regards to real-time processing than a general-purpose processing unit 250 and thus may be more adequate for interacting with production line equipment. Optionally, assembling 1300 the compatible grasp configuration subset 404 may include collecting a state of various production line equipment. In one embodiment, selecting 1400 the optimal grasp configuration 406 and executing 1500 the optimal grasp configuration 406 may be performed by a robot controller unit 270. Other embodiments may prefer to distribute the workload differently depending on circumstances and other factors such as available resources on each unit, and in particular, on the robot controller unit 270, or cost-effectiveness, for example.
[0098] In one embodiment, depicted in Figure 1 B, the method 1000 may be distributed across one or several general- purpose processing units 250, responsible for collecting 1100 the state of the production line and assembling 1200 a subset of eligible grasp configurations. One or several programmable logic controllers 260 may be responsible for assembling 1300 the subset of compatible grasp configurations. Selecting 1400 the optimal grasp configuration and executing 1500 the optimal grasp configuration may be performed by a robot controller. In another embodiment, depicted in Figure 1C, the responsibilities of the role of the programmable logic controllers 260 may be performed by general-purpose processing units 250. In yet another embodiment, depicted in Figure 1D, all tasks, other than executing the optimal grasp configuration, may be performed by the general-purpose processing units 250. These exemplary embodiments are not exhaustive, and other configurations will be evident to the person skilled in the art.
[0099] A part state 510 may include a geometry 511 of the part, a material 512 of the part, and / or a location 513 of the part. As depicted in Figure 2, the geometry 511 , the material 512 and / or the location 513 of the part may be evaluated 1210 when assembling 1200 the eligible grasp configuration subsets 402.
[0100] The state of the production line 500 being collected 1100 may include static and dynamic states of the elements being part of the production line. These include, for example, part states 510, equipment states 520, and a robot state 530. The part states 510 may be used to assemble 1200 the eligible grasp configuration subset 402, the equipment states 520 may be used when assembling 1300 the compatible grasp configuration subset 404, and the robot state 530 may be used when selecting 1400 the optimal grasp configuration 406. The state may also include a catalogue of parts with definitions, characteristics, and predefined rules 550, the features of parts, such as geometry 511 or material 512, and the characteristics identifying one or more properties. The rules 550 may specify permissible handling conditions or other usage guidelines for specific parts 551 or processes 555. The state may further include dynamic positions of the parts within the production line, such as current part occupancy 528 or reachability 526, the presence of obstacles 523, and the state of workspace surfaces 522.
[0101] The part states 510 may include the geometry 511 of the part, a material 512 of the part, and a location 513 of the part.
[0102] The geometry 511 of the part may refer to the physical shape and dimensions of the part. The geometry 511 may include various attributes such as the length, width, height, curvature, or any unique features like holes, grooves, or indentations. The geometry 511 may also include the orientation of the part. The geometry 511 may be evaluated 1210 when assembling 1200 the eligible grasp configuration subsets 402. For example, a part with a cylindrical shape may require a different type of grip compared to a flat or irregularly shaped part. In embodimentsusing multi-grippers, geometry 511 may be used to determine which gripper zone should be used to engage with the part. The geometry 511 may also influence the expected stability of the grasp and be used when selecting 1400 the optimal grasp configuration 406 and the positioning of the one or more gripping tools 100.
[0103] The geometry may also be used when evaluating 1252 the distances described by the grasp configurations 400. A grasp configuration 400 may include a relative positioning coordinate 410 of a grasp point on the part. The relative positioning coordinate 410 may refer to the specific location on the part where the gripper of the robot will engage. This positioning may be defined in relation to a reference point on the part, such as an edge, a corner, or a central point. For example, the relative positioning coordinate 410 might specify a point that is a certain distance from the part’s edge or center. This relative placement allows the grasp configuration 400 to adapt to various shapes and sizes of parts by providing a flexible and precise location for engagement. A grasp configuration 400 may also include a grasp orientation 420. The grasp orientation 420 may refer to the angle or direction at which the gripper of the robot will approach and engage the part. This orientation may be defined in terms of rotational and angular parameters, ensuring that the gripper aligns correctly with the part’s surface and geometry. Proper grasp orientation 420 may provide a stable and secure grasp, especially for parts with complex shapes or specific handling requirements. For example, the grasp configuration 400 may include a relative positioning coordinate 410 of a grasp point on the part, and a grasp orientation 420. The grasp configuration 400 may provide a position of the gripper arm relative to a feature such as an edge, a center or a bounding box of the geometry. In embodiments, the grasp configurations may provide a position relative to a corner of a bounding box, in proportion of an edge’s length. For example, a position might target the bottom left corner of the part’s bounding box and position the gripper arm at 25% of the distance towards the center of the bounding box. This approach may enable the set of grasp configurations 400 to support a wide range of geometries.
[0104] The material 512 of the part may refer to the substance or substances from which the part is made. The material 512 may include metals, plastics, composites, wood, glass, or other materials. The material properties, such as hardness, elasticity, surface texture, porosity, air permeability, and weight may affect the choice of grasp configuration 400 and the safety factor computation 1250. For instance, a part made of a brittle material like glass may necessitate a more delicate handling approach compared to a part made of a durable metal. Additionally, different materials may require different types of gripping tools 100, such as vacuum grippers for smooth surfaces or magnetic grippers for ferrous metals.
[0105] The location 513 of the part may refer to the position within the production line 500 or workspace. The location 513 may include both the absolute position (specific coordinates in a predefined space) and relative position (in relation to other objects or parts in the vicinity). The location 513 may be used when selecting 1430 a grasp configuration 406 with the appropriate positioning or may be used as a reference point when the grasp configurations 400 provide positioning that is relative to features of the part. For example (and as will be further discussed hereinbelow), the positioning provided by the grasp configuration 400 may be defined relative to the location 513 of the part.
[0106] The relative positioning coordinate 410 of the grasp point may include a selected anchor point 411 within a plurality of anchor points on the part, and a distance relative to the selected anchor point 412. A selected anchor point 411 may refer to a specific reference location on the part that is used as a baseline for determining the grasp point. The part may have multiple anchor points, such as edges, corners, or marked features, or a grid distributed over and in proportion to the bounding box of the geometry 511 or other geometrical aspects of the geometry 511, from which one may be chosen as the selected anchor point 411 for the grasping operation.
[0107] The relative positioning coordinate 410 of the grasp point may also include a distance relative to the selected anchor point 412. The distance relative to the selected anchor point 412 may define how far the grasp point is from the selected anchor point 411 . This distance can be measured in terms of linear units, such as millimeters or inches, and may be specified along one or more axes (e.g., x, y, z Cartesian coordinates, or polar coordinates).
[0108] Alternatively, the distance relative to the selected anchor point 412 may be proportional to a dimension of the part, thereby allowing the relative positioning coordinate 410 to adapt to a variety of part geometries. The geometrical aspects used to evaluate 1210 the reference proportions may vary according to specific embodiments, including the width, length, height, or a diagonal of the part. Other references may be considered, such as an edge length or an angle. By defining the grasp point relative to the selected anchor point 411, the grasp configuration 400 may be adapted to various dimensions and features of the part.
[0109] An eligible grasp configuration subset 402 may be assembled 1200 from grasp configurations 400 and part states 510. When assembling the eligible grasp configuration subset 402, the geometry 511 , the material 512, and the location 513 of the part may be evaluated 1210. When evaluating 1210 the geometry 511 of the part, specific attributes such as shape, dimensions, and unique features may be considered. Grasp configurations that are compatible with the evaluated geometry 511 may be selected. For example, if the part has a cylindrical shape, grasp configurations designed for cylindrical objects may be selected, while those designed for flat or irregular shapes may be discarded. The evaluation 1210 of the geometry 511 may ensure that only those grasp configurations which can securely and effectively engage the specific shape and features of the part are retained.
[0110] The evaluation of the material 512 of the part may involve assessing properties such as hardness, elasticity, surface texture, porosity, air permeability, and weight. Grasp configurations that are suitable for the evaluated material 512 may be selected. For instance, grasp configurations using vacuum grippers may be selected for parts with smooth surfaces, while those using magnetic grippers may be selected for ferrous metals. Conversely, grasp configurations incompatible with the evaluated material 512 may be discarded. Evaluation of the material 512 may ensure that the selected grasp configurations can handle the part without causing damage or compromising the grip.
[0111] The evaluation of the location 513 of the part may involve determining its absolute position within the production line 500 or workspace, as well as its relative position to other objects or parts. Only grasp configurations that can be executed from the evaluated location 513 may be selected 1260.
[0112] The evaluation of geometry 511 , material 512, and location 513 of the part may also be performed collectively when determining the eligible grasp configurations 402. For example, a part that is located at a far reachof the robot 40 may still be picked when the geometry 511 and material 512 permit an off-centered grasp. If the part is made of a hard material 512 and has a balanced weight distribution, the stability of the grasp may be maintained even when the grasp point is not centrally located. Consequently, grasp configurations that accommodate an off- centered pick may be selected 1260, provided the geometry 511 allows a secure grip and the material 512 supports the weight without deformation.
[0113] By evaluating the geometry 511, material 512, and location 513 of the part, the eligible grasp configuration subsets 402 may be refined. Grasp configurations that meet the criteria based on these evaluations may be selected 1260, while those that do not meet the criteria may be discarded 1440. After filtering out ineligible grasp configurations based on the geometry 511, material 512, and location 513, the eligible grasp configuration subsets 402 may contain only those configurations that are compatible with the specific characteristics of the part.
[0114] As depicted in Figure 3, in one embodiment, when assembling the eligible grasp configuration subset 402, a safety factor indicative of a likelihood to accidentally drop the part may be computed 1250 and at least one grasp configuration from the plurality of grasp configurations 400 may be selected 1260 when the safety factor is above a minimum safety threshold. The safety factor may represent a quantitative measure of how securely the part may be held by the gripping tool 100. The safety factor may take into account various parameters such as the part’s geometry 511 , material 512, weight, and the grasp configuration 400 itself. The safety factor may help in assessing the risk of the part slipping or being dropped during handling.
[0115] The grasp configuration 400 may be selected 1260 when the safety factor is above a minimum safety threshold. In other embodiments, a filtering out approach may be used whereby a grasp configuration 400 is being discarded 1440 from the initial plurality of grasp configurations 400. In discarding 1440 embodiments, the grasp configuration 400 may be discarded 1440 when the safety factor is below a minimum safety threshold. Persons skilled in the art will readily recognize that both selecting and discarding 1440 approaches may be considered under different circumstances, and one approach may be adapted from the other as needed.
[0116] The minimum safety threshold may be a predefined value that represents the acceptable level of risk for securely grasping and moving the part. Grasp configurations that result in a safety factor below this threshold may be discarded 1440, as they indicate a higher risk of dropping the part. Conversely, only those grasp configurations that meet or exceed the minimum safety threshold may be included in the eligible grasp configuration subsets 402.
[0117] In one embodiment, the safety factor is scaled such that a factor of 0.0 may be equivalent to a null engaging force on the part, a 1 .0 factor may indicate that the expected engaging force is just enough to engage the part, and a factor of 2.0 may indicate that the engaging force is considered enough to pose very little risk of dropping the part. When scaling the safety factor in the range 0.0, 1.0 and 2.0 as described, the minimum safety threshold may be established somewhere between 1 .0 and 2.0 according to empirical tests and tolerance to risk. For example, fragile and expensive parts may rely on a threshold of 2.0, while durable and inexpensive parts may rely on a threshold of 1.2. As will be further discussed hereinbelow, embodiments may execute 1500 an optimal grasp configuration 406 with a lower safety factor differently, such as by reducing the acceleration of the robot 40. When adapting the robot 40 to lower safety factors, the minimum safety threshold of such embodiments may be lowered accordingly.
[0118] By computing 1250 the safety factor and selecting 1260 grasp configurations based on this factor, the selection may ensure that only those configurations that provide a secure and reliable grip are used, thereby potentially reducing the risk of accidents, part losses and enhancing the overall safety and efficiency of the production line.
[0119] The one or more part states 510 may further include a temperature measurement 515, a relative humidity measurement 516. As depicted in Figure 4, when computing 1250 the safety factor, the safety factor may be adjusted 1254 according to the temperature measurement 515 and the relative humidity measurement 516. The temperature measurement 515 may refer to the current temperature of the part. Temperature measurement 515 may be taken using sensors that are in contact with the part or in proximity to the part. A relative humidity measurement 516 may refer to the amount of moisture in the air surrounding the part. Relative humidity measurement 516 may be taken using hygrometers or other humidity-sensing devices located near the part. When computing 1250 the safety factor, the temperature measurement 515 and the relative humidity measurement 516 may be used to adjust 1254 the safety factor. The temperature measurement 515 may affect the material properties of the part, such as its elasticity or brittleness, thereby influencing the grip strength required. The relative humidity measurement 516 may affect the surface properties of the part, such as slipperiness or adhesion, thereby influencing the likelihood of the part slipping from the grip.
[0120] Adjusting 1254 the safety factor according to the temperature measurement 515 and the relative humidity measurement 516 may involve modifying the safety factor to account for changes in the part's material and surface properties due to temperature and humidity. For example, a higher temperature measurement 515 may reduce the material strength, thereby lowering the safety factor, whereas a higher relative humidity measurement 516 may increase the slipperiness of the part, thereby also lowering the safety factor.
[0121] As depicted in Figure 5, when assembling the eligible grasp configuration subset 402, the safety factor when the part is held by the robot 40 may be evaluated 1252 from the relative positioning coordinate 410 and the grasp orientation 420. The relative positioning coordinate 410 may refer to the specific location on the part where the gripper of the robot 40 engages. The relative positioning coordinate 410 may be defined in relation to a reference point on the part, such as an edge, a corner, or a central point. The relative positioning coordinate 410 may influence the stability and security of the grasp when the part is held by the robot 40. The grasp orientation 420 may refer to the angle or direction at which the gripper of the robot 40 approaches and engages the part. The grasp orientation 420 may be defined in terms of rotational and angular parameters. The grasp orientation 420 may ensure that the gripper aligns correctly with the part’s surface and geometry. Evaluating 1252 the safety factor from the relative positioning coordinate 410 and the grasp orientation 420 may involve assessing how these parameters affect the likelihood of securely holding the part. The evaluation 1252 may consider factors such as the distribution of forces on the part, the potential for slippage, and the overall balance of the part when held by the robot 40. The safety factor may be adjusted 1254 based on the relative positioning coordinate 410 and the grasp orientation 420 to ensure that only the grasp configurations that provide a stable and secure hold are included in the eligible grasp configuration subset 402.
[0122] The state of the production line 500 may include a collision avoidance parameter 540 describing potential collisions within a proximity of the robot 40. As depicted in Figure 7, when assembling the eligible grasp configuration subset 402, at least one grasp configuration may be selected 1225 from the plurality of grasp configurations 400 when the collision avoidance parameter 540 suggests avoidance of a collision. The collision avoidance parameter 540 may provide information about obstacles or other parts that are near the robot 40 and may interfere with its movements. When assembling the eligible grasp configuration subsets 402, the collision avoidance parameter 540 may be considered. The grasp configurations may be selected 1225 based on the collision avoidance parameter 540. The selection may involve evaluating whether a grasp configuration avoids potential collisions as indicated by the collision avoidance parameter 540.
[0123] For example, if the collision avoidance parameter 540 indicates the presence of nearby obstacles, only grasp configurations that would avoid a collision with these obstacles may be selected 1225. Only those grasp configurations that do not pose a risk of collision may be included in the eligible grasp configuration subsets 402.
[0124] For example, consider a part that is an elongated wood panel. The wood panel may have a grasp point that is decentered, meaning the robot 40 grips the panel closer to one end rather than in the center. The decentered grasp point may cause the longer, extruding section of the panel to extend outward significantly when it is moved by the robot 40. If the collision avoidance parameter 540 indicates the presence of nearby obstacles, such as other machinery, parts, or structural elements, the extruding section of the wood panel may potentially collide with these obstacles when the panel is moved. The collision avoidance parameter 540 may provide data on the location and dimensions of these obstacles. In this scenario, only grasp configurations that result in the extruding section of the panel avoiding a collision with obstacles may be selected 1225. Only those grasp configurations that avoid collisions, based on the collision avoidance parameter 540, may be included in the eligible grasp configuration subsets 402.
[0125] The collision avoidance parameter 540 may be determined using sensors, cameras, or other monitoring devices that detect the presence and position of obstacles relative to the robot 40. In embodiments, the collision avoidance parameter 540 may be continuously updated to reflect changes in the production line environment, ensuring that the robot 40 operates safely. In other embodiments, the collision avoidance parameter 540 may be considered static and be calibrated ahead of the operations.
[0126] The state of the production line 500 may include a predefined rule 550. The predefined rule 550 may be established for handling a specific part 551 with a specific feature 552 or a specific process 555 with a specific assembly step 556. The predefined rule 550 may provide guidelines or constraints on how the specific part 551 or specific process 555 should be managed.
[0127] When assembling the eligible grasp configuration subsets 402, the predefined rule 550 may be considered. As depicted in Figure 8, the grasp configuration 400 may be selected 1227 from the plurality of grasp configurations 400 when the predefined rule 550 is compatible with the grasp configuration 400. Conversely, the grasp configuration 400 may be discarded if it is not compatible with the predefined rule 550.
[0128] The specific feature 552 of the specific part 551 may include attributes such as a hole, a groove, or an irregular shape. The attributes may also include weak engagement points, such as locations of stickers, mobile, fragile areas, or areas that may damage the gripping tool 100, for example. These features may require particular handling considerations to ensure that the specific part 551 is correctly picked, held, or placed by the robot 40. For example, a specific part 551 with a hole (specific feature 552) may necessitate a grasp configuration 400 that avoids engaging the hole to prevent instability or damage.
[0129] To illustrate, consider a specific part 551 that is a metal sheet with a large central hole (specific feature 552). The predefined rule 550 may state that the metal sheet must be grasped at points away from the hole to ensure stability and prevent deformation. In this illustrative example, only grasp configurations 400 that position the gripper of the robot 40 at locations around the perimeter of the metal sheet, avoiding the central hole, may be selected 1220 as part of the eligible grasp configuration subsets 402.
[0130] Similarly, the specific process 555 may involve specific assembly steps 556 that dictate how parts should be handled during the production process. The predefined rule 550 may specify the orientations, positions, or sequences required for these specific assembly steps 556.
[0131] To illustrate, consider a specific process 555 that involves placing a delicate glass vial into a cushioning station specific assembly step 556 for packaging. The cushioning station may be compatible only with a subset of grasp configurations 400 that gently place the vial without causing damage. The predefined rule 550 may state that the glass vial must be grasped using configurations that minimize pressure and ensure precise placement. As such, only grasp configurations 400 that apply minimal force and align the vial correctly with the cushioning station may be selected 1220 as part of the eligible grasp configuration subsets 402.
[0132] In embodiments, a grasp configuration 400 may include a plurality of engagement points 414. The plurality of engagement points 414 may identify one or more locations of engagement between the part and one or more gripping tools 100. Each engagement point 414 may correspond to a specific spot on the part where a gripping tool 100 may engage.
[0133] The grasp configuration 400 may also include a plurality of engagement point activation parameters 415. The engagement point activation parameters 415 may be used for activating the one or more gripping tools 100. The engagement point activation parameters 415 may determine which of the plurality of engagement points 414 are enabled or disabled during the grasping operation.
[0134] In one embodiment, the original grasp configurations 400 may have all the engagement points 414 enabled. The initial configuration may be designed to engage the part securely by activating all relevant engagement points 414. As the evaluation steps proceed, certain engagement points 414 may be discarded based on specific criteria, reducing their number.
[0135] For example, during the evaluation 1210 of the geometry 511 , some engagement points 414 may be disabled when they are considered outside of the geometry 511 . Similarly, during the evaluation of the predefinedrule 550, engagement points 414 that do not comply with handling guidelines may be disabled, for example, to prevent damage to the part.
[0136] When the number of engagement points 414 reaches zero, the grasp configuration 400 may be considered ineligible. Additionally, if disabling certain engagement points 414 would result in the safety factor falling below the minimum safety threshold, the grasp configuration 400 may also be discarded. For instance, if the original grasp configuration 400 includes ten engagement points 414, and during the evaluations, seven of these points are disabled due to various criteria, the remaining three points 414 must still provide a secure grip. If the safety factor with these three points falls below the acceptable level, the grasp configuration 400 may be discarded.
[0137] As depicted in Figure 14, when assembling the eligible grasp configuration subset 402, engagement point activation parameters 415 may be disabled 1230 to prevent a second part proximate to the part from being picked. Disabling specific engagement point activation parameters 415 may ensure that only the intended part is engaged by the gripper, thereby avoiding accidental engagement with nearby parts.
[0138] For example, consider a scenario where multiple small parts are placed close to each other on a production line. The gripper may have several engagement points 414 that can be activated to pick up parts. If the gripper were to engage all its engagement points 414, it may inadvertently pick up not only the intended part but also a second part in close proximity. To prevent inadvertently picking unintended parts, certain engagement point activation parameters 415 may be disabled 1230.
[0139] For instance, if the gripper has ten engagement points 414 and the intended part is located at the center, the four engagement point activation parameters at the corners may be disabled 1230 to avoid picking up adjacent parts. By disabling these engagement point activation parameters 415, the gripper may precisely engage only the central part without disturbing the surrounding parts.
[0140] When the state of the production line 500 includes a predefined rule 550 for handling a specific part 551 , the predefined rule 550 may include guidelines for handling a specific feature 552 consisting of a weak engagement point. The weak engagement point may refer to a location on the specific part 551 that is structurally less robust and may be prone to damage or deformation when engaged by one or more gripping tools 100. As depicted in Figure 15, when assembling the eligible grasp configuration subsets 402, an engagement point activation parameter 415 may be disabled 1240 to avoid engaging the weak engagement point with the gripping tools 100. Disabling certain engagement point activation parameters 415 may ensure that the gripping tools 100 do not apply force to the weak engagement point, thereby preventing potential damage or instability.
[0141] For example, consider a specific part 551 that is a fragile glass panel with an area that has a known weak engagement point specific feature 552. The predefined rule 550 may state that the glass panel should not be grasped at this weak engagement point to avoid breakage. In this situation, the engagement point activation parameters 415 corresponding to the weak engagement point may be disabled 1240.
[0142] If the gripper has multiple suction cups as gripping tools 100, and one of these suction cups aligns with the weak engagement point on the glass panel, the corresponding engagement point activation parameter 415 may bedisabled 1240. This action may deactivate the suction cup, preventing it from engaging the weak engagement point specific feature 552.
[0143] The nature of the gripping tools 100 may depend on the type of production line. The gripping tools 100 may, for example, use a pressure differential to engage the part and the pressure differential may be generated according to the engagement point activation parameters 415. Examples of gripping tools 100 may include vacuum cups, magnetic grippers, mechanical grippers, pneumatic grippers, hydraulic grippers, electrostatic grippers, adhesive grippers, needle grippers, soft grippers, and vacuum conveyor belts.
[0144] In some embodiments, the eligible grasp configuration subsets 402 may be represented as a Boolean sequence. The eligible grasp configuration subsets 402 may be assembled 1200 by evaluating different aspects of the part and production line. Each aspect may be processed separately, possibly, but not necessarily in parallel, and each process may eliminate certain configurations that do not meet specific criteria.
[0145] For example, different processes may evaluate the geometry 511 of the part, the material 512 of the part, the location 513 of the part, the collision avoidance parameter 540, and predefined rules 550 for handling specific features 552. Each process may generate a Boolean sequence representing the grasp configurations 400 that are eligible based on the evaluated aspect. A '1 ' in the sequence may indicate that the grasp configuration 400 is eligible, while a 'O' may indicate that it is not eligible.
[0146] The final eligible grasp configuration subsets 402 may be assembled 1200 by performing a Boolean operation, such as an intersection operation, over the Boolean sequences generated by each process. The intersection operation may involve performing a logical AND operation between the Boolean sequences. This operation may compute a new Boolean sequence where only the grasp configurations marked as 'T in all original sequences are retained as eligible.
[0147] For example, consider the following Boolean sequences generated by evaluating different aspects:Geometry evaluation: 101110Material evaluation: 111011Location evaluation : 111111Collision avoidance evaluation: 011111Predefined rule evaluation: 111110
[0148] Performing an intersection operation over these sequences may result in the following Boolean sequence for the eligible grasp configuration subsets 402:Intersection result: 001010
[0149] The result '001010' may indicate that the third and fifth grasp configurations are eligible across all six evaluated grasp configurations.
[0150] Similarly, the plurality of engagement point activation parameters 415 may be represented as a Boolean sequence and the part and the second part may be held by performing a union operation over the Boolean sequence. A Boolean sequence may consist of binary values, where a ’ 1 ' may indicate an active engagement point and a ’O' may indicate an inactive engagement point.
[0151] Constraints on geometry 511 and weak engagement points, may result in certain engagement points being disabled. Initially, all bits in the Boolean sequence may be start with '1', indicating that all engagement points are active. Consider a part with an initial Boolean sequence representing the engagement point activation parameters:Initial: 111111
[0152] If, for instance, the part has a specific geometry 511 that makes certain engagement points unsuitable, and there is a predefined rule 550 identifying a weak engagement point, the Boolean sequence may be adjusted to disable those engagement points. Assume the following constraints:
[0153] The geometry 511 evaluation disables the second and fourth engagement points, coded as:Geometry evaluation: 101011
[0154] The predefined rule 550 for the weak engagement point disables the fifth engagement point, coded as:Predefined rule evaluation: 111101
[0155] The adjusted Boolean sequence may be computed by finding the intersection of the Boolean sequences, resulting in:Adjusted: 101001
[0156] In this example, the adjusted Boolean sequence “101001” may indicate that the first, third, and sixth engagement points are active, while the second, fourth, and fifth engagement points are disabled due to the constraints from geometry 511 and weak engagement point.
[0157] The gripping tools 100 may be configured to retractably extend and the grasp configuration 400 may further include one or more extension parameters 430. The extension parameters 430 may enable the gripping tool to adjust its length or reach, allowing for better positioning and engagement with parts of varying sizes and shapes. The extension parameters 430 may specify the extent to which the gripping tool should extend or retract during the grasping operation, or describe the extension as a binary state, indicating that the gripping tool is either extended or not.
[0158] As depicted in Figure 17, when executing 1500 the optimal grasp configuration 406, the adjustable engagement point may be positioned 1520 according to the extension parameters 430. The positioning 1520 based on the extension parameters 430 may allow the gripping tool to adjust its reach dynamically, thereby enabling the simultaneous handling of multiple parts. The ability to retractably extend may facilitate the overlap of the part and one or more other currently held parts, optimizing the use of space and improving the efficiency of the grasping operation.
[0159] For example, consider a scenario where a robot 40 first picks a small part and then a large part. The small part may be picked by a single retracted gripping tool 100, while the large part may be subsequently picked using multiple extended zones. Initially, the small part may be picked with the following grasp configuration 400:Initial grasp configuration:Activate: 100000Extend: 000000
[0160] The small part may be securely held by the first engagement point in its retracted position. After the small part has been picked, the robot may then proceed to pick a large part with the gripper zones 3, 4, and 5. The large part may require multiple engagement points to extend further. The extension parameters 430 for the large part may specify the required extensions:Second grasp configuration:Activate: 101110Extend: 001110
[0161] The adjusted grasp configuration 400 for picking the large part may now reflect the new positions:
[0162] Adjusted grasp configuration Activate 101110 may correspond to the union of the first and second part activation parameters 415, ensuring that the first part is not dropped while picking the second. The extension parameters 430, with configuration 001110, may allow the second part to be picked even when there is an overlap between the first and the second part.
[0163] When executing 1500 the optimal grasp configuration 406, the engagement points 2, 3, and 4 may, for example, be extended by 20 mm to securely hold the large part. In one embodiment, this extension may be 100 mm, allowing the engagement points to reach parts that may be otherwise difficult to reach because of nearby stacks. The small part, initially picked by the retracted first engagement point, may remain securely held while the large part is picked using the extended engagement points.
[0164] In embodiments, the plurality of engagement points 414 may be used. Having the plurality of engagement points 414 may provide a balance between flexibility and control when engaging various parts. Eight to twelve engagement points 414 may, for instance, enable the gripper to pick two or three parts concurrently in common scenarios. The plurality of engagement points 414 may allow for multiple points of contact, ensuring stability and reducing the risk of slippage or dropping when handling multiple parts.
[0165] For example, consider a scenario where the gripper needs to pick two parts simultaneously. With 8 engagement points 414, the gripper may allocate 4 points to each part, ensuring a secure grip on both parts. With 12 engagement points 414, the gripper may allocate 4 points to the first part, 4 points to the second part, and the remaining 4 points may be used as needed to enhance grip stability or to pick a third part. In some scenarios, fewerengagement points 414 may be used for each part, including a single one, but a number of engagement points 414 may provide flexibility in providing enough spacing between two or more parts to prevent them from overlapping.
[0166] In more complex configurations, having 18 to 24 engagement points 414 may enable the gripper to pick 4 or 5 parts concurrently or pick larger parts with more complex shapes. For example, in a scenario where the gripper may need to pick four parts simultaneously, with 18 engagement points 414, the gripper may allocate 4 to 5 points per part, ensuring a secure grip on each of the four parts. With 24 engagement points 414, the gripper may allocate 4 to 5 points per part for up to five parts, or leave some engagement points unallocated to provide proper spacing between the parts, ensuring a balanced and stable grip on all parts.
[0167] In embodiment, the equipment state 520 may include a geometrical configuration 521 of an equipment and a reachability 526 of the equipment from the robot 40. The geometrical configuration 521 may describe the spatial arrangement and dimensions of the equipment within the production line. The geometrical configuration 521 may include attributes such as the height, width, depth, and positional coordinates of the equipment. The geometrical configuration 521 may also encompass the shape and orientation of the equipment, as well as any relevant features such as protrusions, indentations, or mounting points. The reachability 526 may refer to the ability of the robot 40 to access and interact with the equipment based on the range of motion of the robot 40 and positioning capabilities. The reachability 526 may take into account factors such as the length of the arm of the robot, articulation points, and movement constraints. The reachability 526 may determine which parts of the equipment can be reached by the robot 40 for performing actions such as picking, placing, or manipulating parts.
[0168] For example, consider a scenario where the equipment is a conveyor belt. The geometrical configuration 521 of the conveyor belt may include its length, width, height, and the coordinates of its starting and ending points. The reachability 526 of the conveyor belt from the robot 40 may describe the zones of the conveyor belt that the robot 40 can access based on its range of motion. If the robot 40 has a limited reach, certain sections of the conveyor belt may be outside the reachability 526 of the robot, thereby influencing the ability of the robot to interact with those sections.
[0169] In another example, consider a scenario where the equipment is a storage rack. The geometrical configuration 521 of the storage rack may include its overall dimensions, the spacing between shelves, and the positional coordinates of each shelf. The reachability 526 of the storage rack from the robot 40 may describe which shelves can be accessed by the robot 40 for retrieving or placing parts. If the robot 40 has a limited vertical reach, the upper shelves of the storage rack may be outside the reachability 526 of the robot, thereby affecting the interaction of the robot with those shelves.
[0170] The geometrical configuration 521 of the equipment may include a workspace surface 522. The workspace surface 522 may refer to the flat or contoured area of the equipment where parts are placed, processed, or manipulated. The workspace surface 522 may have specific dimensions, material properties, and positional attributes that define the area available for operations.
[0171] A compatible grasp configuration subset 404 may be assembled 1300 from the eligible grasp configuration subsets 402 and equipment states 520. As depicted in Figure 9, when assembling the compatible grasp configuration subset 404 from the eligible grasp configuration subsets 402, the grasp configuration may be selected 1310 such that the part is stable and operational when placed onto the workspace surface 522. Stability may refer to the part remaining securely in place without tipping, sliding, or causing interference with other parts or equipment. Operationality may refer to the part being correctly positioned and oriented for subsequent processes or actions.
[0172] The reachability 526 of the equipment may further influence selecting 1310 the grasp configuration. The reachability 526 may describe the zones of the workspace surface 522 that the robot 40 can access based on its range of motion. The selected grasp configuration may ensure that the part is placed within the accessible zones of the workspace surface 522, thereby allowing the robot 40 to perform necessary operations on the part.
[0173] For example, consider a scenario where the equipment is a machining table with a workspace surface 522. The geometrical configuration 521 of the machining table may include the dimensions and material of the workspace surface 522. The reachability 526 may describe the sections of the workspace surface 522 that the robot 40 can reach. When assembling the compatible grasp configuration subset 404, an eligible grasp configuration may be selected 1310 that places the part stably on the workspace surface 522 within the reachable zones. The part may be positioned securely and ready for machining operations.
[0174] In another example, consider a scenario where the equipment is a conveyor belt with a workspace surface 522 for placing parts. The geometrical configuration 521 of the conveyor belt may include the width, length, and material of the workspace surface 522. The reachability 526 may describe the sections of the conveyor belt that the robot 40 can access. When assembling the compatible grasp configuration subset 404, an eligible grasp configuration may be selected 1310 that places the part stably on the conveyor belt within the accessible zones. The part may remain stable during transportation and be operationally positioned for subsequent processes.
[0175] Optionally, the geometrical configuration 521 may include one or more vertical obstacles 523. The vertical obstacles 523 may refer to any upright structures or features that extend above the workspace surface 522 and may interfere with the movement or placement of parts. Vertical obstacles 523 may include items such as support pillars, posts, machinery parts, or other protrusions.
[0176] As depicted in Figure 10, when assembling the compatible grasp configuration subset 404, the selection 1320 of grasp configuration may consider the presence of the one or more vertical obstacles 523, ensuring that the chosen grasp configuration avoids collisions with these obstacles. The selected grasp configuration may guide the robot 40 to transport the part onto the workspace surface 522 in a manner that circumvents the vertical obstacles 523.
[0177] The reachability 526 of the equipment may describe the zones of the workspace surface 522 that the robot 40 can access. The selected grasp configuration may ensure that the part is transported within the accessible zones while avoiding vertical obstacles 523. This may prevent potential damage to both the part and the equipment.
[0178] For example, consider a scenario where the equipment is a storage rack with multiple shelves as workspace surfaces 522. The geometrical configuration 521 of the storage rack may include vertical obstacles 523 such as support beams or structural posts. When assembling the compatible grasp configuration subset 404, an eligible grasp configuration may be selected 1320 that positions the part on the desired shelf while avoiding collisions with the support beams. The robot 40 may transport the part through a path that navigates around the vertical obstacles 523 and safely places the part on the shelf.
[0179] In another example, consider a scenario where the equipment is a workbench with various tools and fixtures as vertical obstacles 523 on the workspace surface 522. The geometrical configuration 521 of the workbench may include the positions and heights of these vertical obstacles 523. When assembling the compatible grasp configuration subset 404, an eligible grasp configuration may be selected 1320 that ensures the part is placed on the workbench without interfering with the tools and fixtures.
[0180] To prevent or enable stacking of parts, the equipment state 520 may include a current part occupancy 528 of the workspace surface 522. The current part occupancy 528 may refer to the existing parts or objects already placed on the workspace surface 522. The current part occupancy 528 may describe the positions, orientations, and dimensions of these parts, indicating how much space is available and where potential collisions or interferences may occur.
[0181] When assembling the compatible grasp configuration subset 404, a grasp configuration may be discarded when the grasp configuration results in the part being unsafe when placed onto the workspace surface 522 considering the current part occupancy 528. An unsafe placement may refer to scenarios where the new part may collide with existing parts, cause instability, or interfere with ongoing processes.
[0182] For example, consider a scenario where the workspace surface 522 is a section of a conveyor belt. The current part occupancy 528 may indicate that several parts are already positioned on the conveyor belt, occupying specific areas. As depicted in Figure 11 , when assembling the compatible grasp configuration subset 404, a grasp configuration may be discarded 1330 if it positions the new part in a way that overlaps or collides with the existing parts on the conveyor belt. Unless discarded 1330, the grasp configuration may not ensure a safe and stable placement of the new part.
[0183] In another example, consider a scenario where the workspace surface 522 is a storage shelf. The current part occupancy 528 may describe the arrangement of parts already stored on the shelf, including their dimensions and positions. When assembling the compatible grasp configuration subset 404, a grasp configuration may be discarded if it places the new part in a manner that causes it to be unstable or interfere with the parts already on the shelf. The grasp configuration may be discarded to prevent potential tipping, falling, or damage to the parts.
[0184] Optionally, the compatible grasp configuration subset 404 may be represented as a Boolean sequence. A Boolean sequence may consist of binary values, where each bit represents whether a particular grasp configuration is compatible (1 for compatible, 0 for not compatible). The Boolean sequence may provide a compact and efficient way to encode and process the information about the compatible grasp configuration subset 404.
[0185] The compatible grasp configuration subset 404 may be represented as a Boolean sequence. A Boolean sequence may consist of binary values, where each bit represents whether a particular grasp configuration is compatible (1 for compatible, 0 for not compatible). The Boolean sequence may provide a compact and efficient way to encode and process information about the compatible grasp configuration subset 404.
[0186] In addition to using the equipment state 520 when assembling 1300 the compatible grasp configuration subset 404, other information may also be considered, including the part state 510. In some embodiments, for example, the geometry 511 of the parts may be considered while assembling 1300 the compatible grasp configuration subset 404 instead, or in addition to being considered during the assembling 1200 of the eligible grasp configuration subsets 402. Persons skilled in the art will readily recognize that the steps that have been described hereinabove to assemble the eligible grasp configuration subsets 402 may be, in some embodiments, deferred to the assembling 1300 the compatible grasp configuration subset 404. In some embodiments, assembling 1200 of the eligible grasp configuration subsets 402 and assembling 1300 the compatible grasp configuration subset 404 may be performed as a single step, for example.
[0187] The compatible grasp configuration subset 404 may be assembled 1300 across two or more equipment states 520 by performing an intersection operation over the Boolean sequence of the compatible grasp configurations 404 evaluated according to the two or more equipment states 520. The two primary equipment states 520 to consider may typically be the equipment where the part is being picked and the equipment where the part is being placed. Additional equipment states 520 may also be considered, such as those involving potential collisions during the part’s transport.
[0188] For example, consider a scenario where grasp configurations are evaluated for compatibility based on two primary equipment states 520:Equipment State 1 : The equipment where the part is being picked.Equipment State 2: The equipment where the part is being placed.
[0189] Each evaluation may yield a Boolean sequence representing the compatibility of the grasp configurations:Compatibility with Equipment State 1 : 101110Compatibility with Equipment State 2: 111011
[0190] Performing an intersection operation over these Boolean sequences may result in the following combined Boolean sequence:Intersection result: 101010
[0191] The intersection result “101010” may indicate that only the first, third, and fifth grasp configurations are compatible across both primary equipment states 520. This may ensure that the selected grasp configurations are suitable for picking the part from Equipment State 1 and placing it onto Equipment State 2.
[0192] Consider another scenario involving additional equipment states 520 for collision avoidance:Equipment State 1 : 110110 (Picking equipment)Equipment State 2: 101111 (Placing equipment)Equipment State 3: 111001 (Collision avoidance during transport)
[0193] Performing an intersection operation over these Boolean sequences may result in the following combined Boolean sequence:Intersection result: 100000
[0194] The intersection result “100000” may indicate that only the first grasp configuration is compatible across all three equipment states 520.
[0195] The state of the production line 500 may include a priority list 560 and an optimization strategy 570. The priority list 560 may define the relative importance of various factors or objectives within the production process. The optimization strategy 570 may outline the criteria and methods used to evaluate and select the optimal grasp configuration 406.
[0196] An optimal grasp configuration 406 may be selected 1400 from the compatible grasp configuration subset 404 and the robot state 530. As depicted in Figure 12, when selecting 1400 the optimal grasp configuration 406 for a compatible grasp configuration in the compatible grasp configuration subset 404, a compatible grasp configuration score may be computed 1410. The score may be computed according to the optimization strategy 570, the priority list 560, and the robot state 530. The robot state 530 may include various parameters such as the current position, speed, and load capacity of the robot 40.
[0197] The computed score may represent how well a compatible grasp configuration aligns with the defined optimization strategy 570 and priority list 560. The score may take into account factors such as efficiency, safety, stability, and throughput. The optimization strategy 570 may guide the computation of the score by providing weighted values or specific algorithms to evaluate each factor.
[0198] The score of a given compatible grasp configuration may be compared against a second score of a second grasp configuration. The comparison may involve evaluating the numerical values of the scores to determine which grasp configuration better satisfies the optimization criteria. The compatible grasp configuration may be selected from the compatible grasp configuration subset 404 when the compatible grasp configuration score is higher than the second score of the second grasp configuration.
[0199] An equipment usage score, a throughput score, a safety score, a minimized travel time score, or a clearance score may be optimized 1470 by the optimization strategy 570. The equipment usage score may measure how effectively the equipment is utilized. For example, a grasp configuration that allows the robot 40 to maximize the usage of a machine’s capacity may receive a higher equipment usage score. The throughput score may evaluate the efficiency of the production line in terms of the number of parts processed within a given time period. For example, a grasp configuration that enables the robot 40 to handle more parts per hour may receive a higher throughput score. The safety score may assess the risk associated with the grasp configuration. For example, agrasp configuration that minimizes the likelihood of dropping a part or causing a collision may receive a higher safety score. The minimized travel time score may measure the efficiency of the movements of the robot. For example, a grasp configuration that reduces the distance the robot 40 needs to travel to pick and place a part may receive a higher minimized travel time score. The clearance score may evaluate the availability of space on an equipment to accept new parts. For example, a grasp configuration that ensures the robot 40 places parts in such a way that the equipment is not overfilled and can continue to accept new parts may receive a higher clearance score. This may prevent bottlenecks and ensure continuous operation of the production line. In each case, the optimization strategy 570 may use these scores to determine the most effective grasp configuration for the robot 40 to use.
[0200] As depicted in Figure 13, in one embodiment, the process of selecting (1400) the optimal grasp configuration (406) from multiple candidates may include evaluating each candidate within the compatible grasp configuration subset (404) and computing a score for each candidate according to the optimization strategy (570) and the priority list (560). The comparison (1420) of the computed score of a candidate to a second score of a second grasp configuration may involve examining metrics such as a safety score, a minimized travel time score, and / or a throughput score. Selecting (1430) the candidate with the higher score ensures that the chosen grasp configuration (406) addresses the criteria set forth by the optimization strategy (570). For example, consider two candidate grasp configurations for picking a large and oddly shaped part. The first candidate may hold the part near the geometric center to increase balance, resulting in a higher safety score, whereas the second candidate may hold the part from one edge, leading to a shorter travel path but a slightly lower safety margin. By comparing (1420) both scores, the approach with a higher overall result— accounting for safety, efficiency, and other selected priorities— may be chosen (1430) for final execution. Alternative embodiments may provide comparisons of two or more candidates that integrate tie-breaker rules, scaling factors, and / or additional numerical criteria.
[0201] The equipment usage score, the throughput score, the safety score, and the clearance score may be adjusted according to a diminishing marginal returns function. A diminishing marginal returns function may describe a principle where the incremental benefit gained from an increase in a particular factor decreases as the level of that factor increases.
[0202] The equipment usage score may be adjusted using a diminishing marginal returns function. For example, as the utilization of equipment increases, the additional benefit of further increasing utilization may decrease. When the equipment is already near full capacity, the incremental improvement in production efficiency from further utilization may be minimal. The throughput score may also be adjusted using a diminishing marginal returns function. For instance, as the number of parts processed per hour increases, the additional benefit of processing even more parts may decrease. When the production line is already operating at high efficiency, the incremental gain in throughput may be less significant. The safety score may be adjusted using a diminishing marginal returns function. For example, as safety measures are enhanced, the additional benefit of further safety improvements may decrease. When the risk of accidents is already very low, the incremental improvement in safety from additional measures may be minimal. The clearance score may be adjusted using a diminishing marginal returns function. For instance, as the space available on equipment for placing new parts increases, the additional benefit of further increasing thisspace may decrease. When there is already ample space for parts, the incremental gain in clearance from making more space available may be less impactful.
[0203] By adjusting these scores according to a diminishing marginal returns function, multiple optimizations may be combined effectively. This approach may prevent the optimization strategy 570 from disproportionately weighting one factor when there is only marginal improvement to be gained. As a result, the optimization may ensure a balanced and efficient allocation of resources, enhancing overall production performance by considering multiple factors simultaneously.
[0204] The priority list 560 may also include a preference in transferring the part to a processing equipment, transferring the part to an accumulation equipment, or transferring the part to a conveying equipment. Transferring the part to a processing equipment may involve moving the part to a machine or station where it undergoes further operations such as cutting, welding, painting, or assembling. The priority list 560 may prioritize this transfer if immediate processing is required to maintain production flow or meet quality standards. Transferring the part to an accumulation equipment may involve moving the part to a location where it can be temporarily stored or queued. Accumulation equipment may include bins, tables, or racks that hold parts until they are needed for the next stage of the production process. The priority list 560 may prioritize this transfer to manage workflow and ensure that parts are readily available for subsequent operations. Transferring the part to a conveying equipment may involve moving the part to a conveyor system that transports it to different areas within the production line or facility. Conveying equipment may include belts, rollers, or automated guided vehicles (AGVs). The priority list 560 may prioritize this transfer to facilitate efficient movement of parts across the production floor, reducing manual handling and speeding up the overall process.
[0205] The optimal grasp configuration 406 may then be executed 1500 with the robot 40 to perform at least one of picking action and / or placing action of a part. A picking action may refer to the step of physically engaging and lifting a part from a surface or storage location using one or more gripping tools 100. The picking action may occur when the part is on a conveyor, inside a container, or placed on a table. The robot 40 may employ a mechanical clamp, a multi-finger gripper, or a system that relies on a pressure differential to pull the part against one or more engagement points 414. A placing action may refer to depositing the part onto a target location such as a workspace surface 522, another conveyor, or an equipment state 520 for further processing. For instance, the robot 40 may orient the part, navigate around obstacles, and then place the part into a fixture or storage rack.
[0206] When multiple gripping tools 100 are available, the robot 40 may decide which subset of these tools 100 to employ for the picking action or the placing action. The robot 40 may also consider whether to hold multiple parts concurrently or to place a currently picked part before retrieving another. The robot 40 may further evaluate the sequence of picking and placing steps when exchanging one part for another to ensure uninterrupted operation. Without an optimal grasp configuration 406, the robot 40 may face many choices regarding tool selection, orientation, collision avoidance, and other adjustments. By using the optimal grasp configuration 406, the robot 40 may proceed swiftly with reduced uncertainty and fewer decisions during the picking action and the placing action. Some checks are still performed by the robot 40 to monitor changing conditions, and the optimal grasp configuration406 may be rejected under exceptional circumstances, such as when the updated state of the production line 500 reveals safety concerns that were not previously detected.
[0207] As depicted in Figure 6, when executing 1500 the optimal grasp configuration 406, an acceleration of the robot 40 may be adjusted 1510 based on the safety factor. The safety factor may represent a measure of how securely the part is being held by the robot 40. For example, consider a scenario where the safety factor is high. A high safety factor may indicate that the part is being held very securely, with minimal risk of dropping or mishandling. In this case, the acceleration of the robot 40 may be increased. By increasing the acceleration, the robot 40 may move more quickly, thereby improving the efficiency of the production process. This may be particularly useful in high-throughput environments where speed is a priority. In another scenario, the safety factor may be lower. A lower safety factor may indicate that the part is being held with some risk of instability or potential dropping. In this case, the acceleration of the robot 40 may be decreased. By reducing the acceleration, the robot 40 may move more slowly and carefully, ensuring that the part remains secure during transport. This adjustment may help to prevent accidents, damage to the part, or interruptions in the production process. For instance, if the part being handled is a delicate glass vial with a low safety factor due to its fragile nature, the robot 40 may reduce its acceleration to avoid sudden movements that could cause the vial to slip or break. On the other hand, if the part is a robust metal component with a high safety factor, the robot 40 may increase its acceleration, allowing for faster and more efficient handling.
[0208] The degree to which the acceleration of the robot 40 is increased or decreased may be directly related to the value of the safety factor. For example, if the safety factor is low, indicating a less secure grip, the acceleration of the robot 40 may be decreased proportionally. This proportional decrease means that the lower the safety factor, the more the acceleration may be reduced. This proportional relationship ensures that the movement of the robot 40 is dynamically adjusted based on the security of the grip on the part. If the safety factor is at its maximum value, the robot 40 may operate at its highest acceleration. If the safety factor is at its minimum value, the robot 40 may operate at its lowest acceleration. By maintaining this proportional relationship, the robot 40 can optimize its speed and efficiency while ensuring the safety and stability of the part being handled.
[0209] When multi-picking, the robot state 530 may include one or more previous grasp configurations of one or more other currently held parts 532. The one or more previous grasp configurations of the one or more other currently held parts 532 may provide information about how the robot 40 is currently holding other parts, including the positions, orientations, and stability of those parts.
[0210] As depicted in Figure 16, when selecting 1400 the optimal grasp configuration 406, one or more compatible grasp configurations posing a risk of dropping at least one of the one or more other currently held parts may be discarded 1440 when picking the part. In other words, during the selection process, the robot 40 may evaluate the potential impact of adding a new part to its current load. If a new grasp configuration is found to be likely to compromise the stability or security of the previously held parts, that grasp configuration may be discarded. For example, if the robot 40 is already holding a part using a specific grasp configuration and finds another grasp configuration that allows it to pick an additional part, the robot 40 may assess whether this new grasp configurationposes a risk. If the new grasp configuration may cause the robot 40 to move in a way that could destabilize the previously held part, or if it may interfere with the grip on the existing part, then the robot 40 may discard that new grasp configuration. By including the previous grasp configurations of the currently held parts 532 in the robot state 530 and considering them during the selection of the optimal grasp configuration 406, the robot 40 may ensure that the integrity and security of all currently held parts are maintained.
[0211] Similarly, a risk of a collision of the part when transporting the one or more other currently held parts may be computed. The risk of collision may refer to the likelihood that the part being transported by the robot may come into contact with other parts, equipment, or obstacles within the production environment. This computation may take into account the trajectories, dimensions, and positions of the parts and equipment involved. The optimal grasp configuration may be rejected when the risk of collision is above a risk of collision threshold. The risk of collision threshold may represent a predefined level of acceptable risk. If the computed risk of collision exceeds this threshold, the grasp configuration may be considered too risky to execute safely.
[0212] For example, as depicted in Figure 18, if the robot 40 is transporting multiple parts and needs to select an optimal grasp configuration 406 for an additional part, the robot 40 may first compute 1450 the risk of a collision for this new configuration. If the new grasp configuration may cause the part to collide with other currently held parts or nearby equipment during transport, and the risk of such a collision is above the acceptable threshold, then the new grasp configuration may be rejected 1460.
[0213] Under some circumstances, such as the late detection of a collision when picking multiple parts sequentially or concurrently described hereinabove, the optimal grasp configuration 406 may be discarded and it may be advantageous to consider another of the compatible grasp configurations 404 as a new optimal grasp configuration 406 candidate. When the system 2000 is embodied such that the conflict is detected on the robot 40 hardware and the optimal grasp configuration 406 may have been computed on different hardware, such as a programmable logic controller unit 260 for example, it may be inconvenient to elect a new optimal grasp configuration 406. In some embodiments, it may be advantageous to select an optimal grasp configuration subset comprising more than one optimal grasp configuration 406. The optimal grasp configuration subset may be assembled by selecting 1400, from the compatible grasp configuration 404, the grasp configuration evaluated by the optimization strategy 570 with a score above an acceptable threshold. By assembling an optimal grasp configuration subset, it may be possible to efficiently select one or several alternative optimal grasp configurations 406 when a late constraint, such as a collision between two picked parts, is detected. In other embodiments, the optimal grasp configuration subset may be a sorted set of optimal grasp configurations, sorted according to the optimization strategy 570 scores. When a sorted set of optimal grasp configurations is provided, the optimal grasp configuration 406 may then be considered sequentially until one that can be safely executed 1500 is found.
[0214] Other late constraints may exist that justify assembling a compatible grasp configuration subset 404. For example, when picking multiple one or more parts 50 sequentially or concurrently, it may be advantageous to prioritize picking a large one or more parts 50 such that it may overlap several smaller one or more parts 50 and contribute to securing them against the one or more gripping tools 100 during transport. In this example, if the robot40 is provided a compatible grasp configuration subset 404, the robot 40 may be capable of evaluating the compatible grasp configuration subset 404 against the robot state 530 and elect the final optimal grasp configuration 406.
[0215] The execution may be performed repeatably in real time, ensuring the robot does not need to pause for a period longer than a few milliseconds while processing the actions to undertake. In embodiments, the maximum observed delays were below 250 ms, observed in scenarios of multi-picking involving detected collisions. In most iterations, the observed delays may be unnoticeable. The robot 40 may operate continuously and uninterruptedly even as the states change.
[0216] For example, as parts move along the production line, the robot 40 may constantly receive updates about the state of the production line 500, including part states 510, equipment states 520, and robot state 530. The method for controlling the robot 40 may process this information in real time, allowing the robot 40 to make immediate adjustments to its grasp configurations and movements. By executing the method repeatably in real time, the robot 40 may continuously adapt to new parts, changing positions, and varying conditions.
[0217] In embodiments, the one or more equipment states 520 may include an equipment state for a doweling jig station, a welding station, a painting and coating station, an inspection station, a deburring station, a gluing and sealing station, an accumulation table, a storage station, a palletizing station, a conveyor, a transfer station, a turntable, a nesting router, a pod and rail router, a vertical CNC, a CNC-Drilling and dowel insertion machine, a horizontal boring machine, a panel dividing saw, a hardware insertion station, a waterjet cutting center, a disk cutting center, and other CNC machining centers, an edgebanding machine, a sanding machine, a polishing machine, an assembly system, or a storage and retrieval system. These are non-exhaustive examples of equipment where the method, system, and non-transitory computer-readable medium described herein may be particularly useful. For instance, doweling jig stations and welding stations may require precise alignment and insertion, ensuring strong joints and reducing manual errors. Painting and coating stations may demand uniform application. Inspection stations may benefit from accurate positioning for defect detection, while deburring stations may need careful handling to remove sharp edges without damaging parts. In gluing and sealing stations, precise adhesive application may ensure robust and consistent bonds. Accumulation tables, storage stations, and palletizing stations may require optimized part placement to prevent collisions and maximize space utilization. Conveyors, transfer stations, and turntables may necessitate precise timing and positioning to ensure smooth part transitions. Routers and CNC machines may demand exact part placement for accurate cutting and drilling, while hardware insertion stations may require precise alignment for reliable installation. Cutting centers, such as waterjet and disk cutting centers, may benefit from consistent part handling to optimize cutting paths and reduce material waste. Sanding machines and polishing machines may rely on consistent handling for high-quality finishes. Assembly systems and storage and retrieval systems may depend on dynamic adjustments to optimize the organization and accessibility of parts, thereby improving workflow continuity and production efficiency.
[0218] When embodied as a system 2000, multiple parts may collaborate to implement the technique described herein. The system 2000 may include one or more parts 50 with one or more part states 510, one or more processingequipment 60 with one or more equipment states 520, a robot 70 with a robot state 530, and one or more processors 210.
[0219] The parts 50 may refer to objects or articles intended to be manipulated, transferred, processed, or assembled on the production line. The parts 50 may include smaller items or larger components in either singular or multiple forms. The parts 50 may be metallic sheets, wooden boards, assembly-ready subcomponents, molded pieces of plastic, or any other type of industrially relevant item. In another example, the parts 50 may be automotive panels, electronics housings, circuit boards, or other structures requiring robotic handling. The role of the parts 50 in this context may be to undergo one or more processes such as picking, placing, orienting, inspecting, or transporting, where the method 1000, system 2000, or non-transitory computer-readable medium interacts with the parts 50 based on their geometry, material properties, and location within the production line. Each part 50 may be processed in accordance with states and rules, such as safety criteria, collision avoidance parameters, or predefined instructions, to ensure a consistent and uninterrupted operation.
[0220] Processing equipment 60 may refer to machines or stations configured to modify, reshape, assemble, or finish the part 50. The role of the processing equipment 60 may be to provide an environment where the part 50 can be acted upon according to the required manufacturing or assembly steps, such as changing the geometry 511 of the part or carrying out a specific process 555. Examples of processing equipment 60 may include a doweling jig station, a welding station, a painting and coating station, a deburring station, or a gluing and sealing station. Each of these processing equipment 60 may define one or more equipment states 520, including a workspace surface 522 or a set of operational constraints that the robot 40 may adhere to when executing 1500 a picking or placing action for the part 50.
[0221] A robot 40 may be an automated machine with actuators, sensors, or both, configured to handle one or more parts according to instructions from one or more processors 210. For example, the robot 40 may be an articulated arm capable of rotating across multiple axes to pick a part from a storage location and place the part onto a workspace surface 522. In another example, the robot 40 may be a cartesian manipulator that travels linearly along x, y, and z axes to transfer the part between stations. The role of the robot 40 in the context provided may include moving parts in a coordinated manner, aligning them for subsequent steps, and adapting grasp configurations based on the state of the production line 500.
[0222] In one embodiment, the robot 40 may have gripping tools 100 for physically engaging a part such that the robot 40 can pick, hold or place the part. The gripping tools 100 may operate with various mechanisms, for example, a mechanical clamp may rely on hinged jaws, or a vacuum-based tool may create a pressure differential. The actuators may be modules configured to control the movement or orientation of the gripping tools 100. A linear actuator may extend or retract the tool along a specified axis, and a rotary actuator may pivot the tool around an axis. Additional examples may include telescopic actuators, stepping actuators or servo-driven actuators, thereby allowing diverse arrangements of gripping tools 100 to handle a range of part sizes and shapes.
[0223] The one or more processors 210 may be specialized or general-purpose processing devices configured to collect states of the production line 500, assemble grasp configuration subsets 402, 404, and select an optimalgrasp configuration 406. A general-purpose processing unit 250 may be a server, workstation, or embedded computer capable of running complex algorithms or interfacing with sensors and monitoring devices. A programmable logic controller unit 260 may be a hardware- or software-based controller providing real-time control and direct interaction with actuators and machine inputs. A robot controller unit 270 may be a dedicated controller coordinating the movements of the robot 40. Such processors 210 may, for instance, orchestrate part picking and placing tasks, adjust safety factors, or evaluate collisions. For example, a processor 210 hosted on a workstation may handle computationally intensive geometry calculations, whereas a programmable logic controller unit 260 may execute time-sensitive operations involving collision avoidance or gripping tool 100 activation.
[0224] A temperature sensor may be used to collect a temperature measurement 515 reflecting a thermal property of the part. The temperature sensor may be in contact with or in proximity to the part and may influence a safety factor by adjusting 1254 the likelihood to accidentally drop the part when temperatures vary. A humidity sensor may be used to collect a relative humidity measurement 516 capturing the moisture level around the part and may also adjust 1254 the safety factor when higher humidity suggests slipperier surfaces or different material behaviors. A set of sensors providing data for the collision avoidance parameter 540 may detect proximity of obstacles within the workspace to suggest avoidance of collisions and may guide assembly 1200 of an eligible grasp configuration subset 402.
[0225] A positioning module may be used to capture the relative positioning coordinate 410 of the grasp point on a part from the one or more parts 50, and to capture a grasp orientation 420. The positioning module may use sensing or measurement devices to determine how the part is oriented or located before a grasp configuration 400 is assembled 1200. In one embodiment, the positioning module may rely on a 2D or 3D vision system combined with pattern recognition to locate the part in space, where a camera array may provide image data that is processed to identify edges, corners, or fiducial markers, thereby computing the relative positioning coordinate 410 of the grasp point and the grasp orientation 420. In another embodiment, the positioning module may employ structured light or laser scanning, projecting beams across the part and interpreting reflections to generate a depth map. The depth map may serve to find positions on the part surface for the grasp point and supply the correct angular approach for the grasp orientation 420. In one embodiment, the positioning module may rely on embedded tactile or proximity sensors, where the part is probed or scanned via contact or near-contact sensing to measure local features and align a reference anchor point 411 , enabling the determination of both the location and orientation required for the grasp.
[0226] As used in this specification and claim(s), the expression “at least one of” followed by a set of elements suggests that any combination of the elements from the set is being considered, including a single element from the set, and all elements from the set. For clarity, “at least one of” followed by a set does not strictly refer to having at least the whole set once, and possibly multiple times.
[0227] As used in this specification and claim(s), the words “comprising” (and any form of comprising, such as “comprise” and “comprises”), “having” (and any form of having, such as “have” and “has”), “including” (and any formof including, such as “include” and “includes”) or “containing” (and any form of containing, such as “contain” and “contains”), are inclusive or open-ended and do not exclude additional, unrecited elements or process steps.
[0228] While illustrative and presently preferred embodiments of the invention have been described in detail hereinabove, it is to be understood that the inventive concepts may be otherwise variously embodied and employed and that the appended claims are intended to be construed to include such variations except insofar as limited by the prior art.
[0229] A method is generally conceived to be a self-consistent sequence of steps leading to a desired result. These steps require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic / electromagnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It is convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, parameters, items, elements, objects, symbols, characters, terms, numbers, or the like. It should be noted, however, that all of these terms and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The description of the present invention has been presented for purposes of illustration but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments were chosen to explain the principles of the invention and its practical applications and to enable others of ordinary skill in the art to understand the invention in order to implement various embodiments with various modifications as might be suited to other contemplated uses.
[0230] A system may generally be conceived as an arrangement of multiple components that work together to achieve a particular function or result. These components each may have distinct roles and contribute to the overall operation of the system. It is convenient to describe these components as units, modules, parts, or elements. Components may be physical or logical in nature. In practice, systems may be implemented in various forms. While systems are typically comprised of multiple distinct components, in some embodiments, all or some components may coexist within a single device. This integration does not alter the fundamental understanding of the operation but rather represents an embodiment where functionality is consolidated. Such a configuration may be advantageous for specific applications where space, efficiency, or other considerations are paramount. Regardless of configuration, systems are understood to operate through physical interactions, which may, in some embodiments, be achieved through electronic components such as RAM, buses and processors.
Claims
CLAIMS1. A method (1000) for controlling a robot (40) on a production line, the method comprising: collecting (1100) a state of the production line (500) comprising- one or more part states (510);- one or more equipment states (520); and- a robot state (530); assembling (1200) an eligible grasp configuration subset (402) from a plurality of grasp configurations (400) and at least one part state of the one or more part states (510); assembling (1300) a compatible grasp configuration subset (404) from the eligible grasp configuration subsets (402) and at least one equipment state from the one or more equipment states (520); selecting (1400) an optimal grasp configuration (406) from the compatible grasp configuration subset (404) and the robot state (530); and executing (1500) the optimal grasp configuration (406) with the robot (40) to perform at least one of picking action and placing action of a part.
2. The method (1000) of claim 1, wherein at least one part state from the one or more part states (510) comprises at least one of:- a geometry (511) of the part;- a material (512) of the part;- a location (513) of the part; and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: evaluating (1210) at least one of the geometry (511), the material (512) and the location (513) of the part.
3. The method (1000) of claim 1 or claim 2, wherein the compatible grasp configuration subset (404) is represented as a Boolean sequence and wherein assembling (1300) the compatible grasp configuration subset (404) across two or more equipment states (520) is achieved by performing an intersection operation over the Boolean sequence of the two or more equipment states (520).
4. The method (1000) of any one of claims 1 to 3, wherein assembling (1200) the eligible grasp configuration subsets (402) comprises: computing (1250) a safety factor indicative of a likelihood to accidentally drop the part; and selecting (1260) at least one grasp configuration from the plurality of grasp configurations (400) when the safety factor is above a minimum safety threshold.
5. The method (1000) of claim 4, wherein the one or more part states (510) further comprises at least one of: a temperature measurement (515); a relative humidity measurement (516); and wherein computing (1250) the safety factor comprises:adjusting (1254) the safety factor according to at least one of the temperature measurement (515) and the relative humidity measurement (516).
6. The method (1000) of claim 4 or claim 5, wherein the plurality of grasp configurations (400) comprises: a relative positioning coordinate (410) of a grasp point on the part; a grasp orientation (420); and wherein computing (1250) the safety factor further comprises: evaluating (1252) the safety factor when the part is held by the robot (40) from the relative positioning coordinate (410) and the grasp orientation (420).
7. The method (1000) of claim 6, wherein the relative positioning coordinate (410) of the grasp point comprises: a selected anchor point (411) within a plurality of anchor points on the part; a distance relative to the selected anchor point (412); and wherein the distance relative to the selected anchor point (412) is proportional to a dimension of the part, thereby allowing the relative positioning coordinate (410) to adapt to a variety of part geometries.
8. The method (1000) of any one of claims 4 to 7, wherein executing (1500) the optimal grasp configuration (406) comprises: adjusting (1510) an acceleration of the robot (40) based on the safety factor.
9. The method (1000) of any one of claims 1 to 8, wherein the state of the production line (500) further comprises: a collision avoidance parameter (540) describing potential collisions within a proximity of the robot (40); and wherein assembling (1200) the eligible grasp configuration subsets (402) comprises: selecting (1225) at least one grasp configuration from the plurality of grasp configurations (400) when the collision avoidance parameter (540) suggests avoidance of a collision.
10. The method (1000) of any one of claims 1 to 9, wherein the state of the production line (500) further comprises: a predefined rule (550) for handling at least one of a specific part (551) comprising a specific feature (552), and a specific process (555) comprising a specific assembly step (556); and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: selecting (1227) at least one grasp configuration (400) from the plurality of grasp configurations (400) when the predefined rule (550) is compatible therewith.
11. The method (1000) of claim 10, wherein the specific feature (552) is any one of a hole, a groove, and an irregular shape.
12. The method (1000) of any one of claims 1 to 11, performed repeatably in real time and wherein the robot (40) operates continuously and uninterruptedly.
13. The method (1000) of any one of claims 1 to 12, wherein an equipment state of the one or more equipment states (520) comprises: a geometrical configuration (521) of an equipment comprising:- a workspace surface (522); and a reachability (526) of the equipment from the robot (40); and wherein assembling (1300) the compatible grasp configuration subset (404) comprises: selecting (1310) at least one eligible grasp configuration from the eligible grasp configuration subsets (402) resulting in the part being stable and operational therewith when placed onto the workspace surface (522) across the reachability (526) of the equipment.
14. The method (1000) of claim 13, wherein the geometrical configuration (521) further comprises: one or more vertical obstacles (523); and wherein assembling (1300) the compatible grasp configuration subset (404) further comprises: selecting (1320) at least one eligible grasp configuration from the eligible grasp configuration subsets (402) when the one or more vertical obstacles (523) suggest avoidance of a collision therewith when transporting the part onto the workspace surface (522) across the reachability (526) of the equipment.
15. The method (1000) of claim 13 or claim 14, wherein the equipment state further comprises: a current part occupancy (528) of the workspace surface (522); and wherein assembling (1300) the compatible grasp configuration subset (404) comprises: discarding (1330) at least one grasp configuration from the eligible grasp configuration subsets (402) resulting in the part being unsafe when placed therewith onto the current part occupancy (528) of the workspace surface (522).
16. The method (1000) of any one of claims 1 to 15, wherein the state of the production line (500) further comprises: a priority list (560); an optimization strategy (570); and wherein selecting (1400) the optimal grasp configuration (406) comprises: for a compatible grasp configuration in the compatible grasp configuration subset (404):- computing (1410) a compatible grasp configuration score according to the optimization strategy (570), the priority list (560) and the robot state (530); and thereby optimizing (1470) the optimal grasp configuration according to the optimization strategy (570).
17. The method (1000) of claim 16, wherein selecting (1400) the optimal grasp configuration (406) further comprises: comparing (1420) the compatible grasp configuration score against a second score of a second grasp configuration; andselecting (1430) the compatible grasp configuration from the compatible grasp configuration subset (404) when the compatible grasp configuration score is higher than the second score of the second grasp configuration.
18. The method (1000) of claim 16 or claim 17, wherein the optimization strategy (570) optimizes (1470) at least one of an equipment usage score, a throughput score, a safety score, a minimized travel time score, and a clearance score.
19. The method (1000) of claim 18, wherein the equipment usage score, the throughput score, the safety score and the clearance score are adjusted according to a diminishing marginal returns function.
20. The method (1000) of any one of claims 16 to 19, wherein the priority list (560) comprises at least one of transferring the part to a processing equipment, transferring the part to an accumulation equipment and transferring the part to a conveying equipment.
21. The method (1000) of any one of claims 1 to 20, wherein the plurality of grasp configurations (400) comprises: a plurality of engagement points (414) for identifying one or more locations of engagement between the part and one or more gripping tools (100); and a plurality of engagement point activation parameters (415) for activating the one or more gripping tools (100); and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: disabling (1230) at least one engagement point activation parameter from the plurality of engagement point activation parameters (415), thereby preventing a second part proximate to the part from being picked.
22. The method (1000) of claim 21, wherein the state of the production line (500) further comprises: a predefined rule (550) for handling a specific part (551) comprising a specific feature (552) consisting of a weak engagement point; and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: disabling (1240) at least one engagement point activation parameters from the plurality of engagement point activation parameters (415), thereby avoiding engaging the weak engagement point with one or more gripping tools (100).
23. The method (1000) of claim 21 or claim 22, wherein the plurality of engagement point activation parameters (415) is represented as a Boolean sequence, and the part and the second part may be held by performing a union operation over the Boolean sequence.
24. The method (1000) of any one of claims 21 to 23, wherein the plurality of engagement points (414) comprises at least 8 engagement points (414).
25. The method (1000) of any one of claims 21 to 24, wherein at least one gripping tool from the one or more gripping tools (100) uses a pressure differential to engage the part and wherein the pressure differential is generated according to the plurality of engagement point activation parameters (415).
26. The method (1000) of any one of claims 21 to 25, wherein the robot state (530) comprises:one or more previous grasp configurations of one or more other currently held parts (532); and wherein selecting (1400) the optimal grasp configuration (406) comprises: discarding (1440) one or more compatible grasp configurations posing a risk of dropping at least one of the one or more other currently held parts when picking the part.
27. The method (1000) of claim 26, wherein at least one gripping tool within the one or more gripping tools (100) is configured to retractably extend and wherein the plurality of grasp configurations (400) further comprises: one or more extension parameters (430); and wherein executing (1500) the optimal grasp configuration (406) comprise: positioning (1520) the at least one gripping tool according to the one or more extension parameters (430), thereby allowing the part and the one or more other currently held parts to overlap.
28. The method (1000) of claim 1 or claim 27, further comprising: computing (1450) a risk of a collision of the part when transporting the one or more other currently held parts; and rejecting (1460) the optimal grasp configuration (406) when the risk of collision is above a risk of collision threshold.
29. The method (1000) of any one of claims 1 to 28, wherein the one or more equipment states (520) comprise an equipment state for any one of a doweling jig station, a welding station, a painting and coating station, an inspection station, a deburring station, a gluing and sealing station, an accumulation table, a storage station, a palletizing station, a conveyor, a transfer station, a turntable, a nesting router, a pod and rail router, a vertical CNC, a CNC-Drilling and dowel insertion machine, a horizontal boring machine, a panel dividing saw, a hardware insertion, a waterjet cutting center, a disk cutting center and other CNC machining centers, an edgebanding machine, a sanding machine, a polishing machine, an assembly system and a storage and retrieval system.
30. A system (2000) for controlling a robot (40) on a production line comprising: one or more parts (50) comprising one or more part states (510); one or more processing equipment (60) comprising one or more equipment states (520); a robot (70) comprising a robot state (530); and one or more processors (210) configured to:- collect (1100) a state of the production line (500) comprising:- the one or more part states (510)- the one or more equipment states (520); and- the robot state (530);- assemble (1200) an eligible grasp configuration subsets (402) from a plurality of grasp configurations (400) and at least one part state of the one or more part states (510);- assemble (1300) a compatible grasp configuration subset (404) from the eligible grasp configuration subsets (402) and at least one equipment state from the one or more equipment states (520); and- select (1400) an optimal grasp configuration (406) from the compatible grasp configuration subset (404) and the robot state (530); and wherein the robot (40) is configured to execute (1500) at least one of a picking action and a placing action of one or more parts (50) using the optimal grasp configuration (406).31 . The system (2000) of claim 30, wherein a part state of the one or more part states (510) comprise at least one of: a geometry (511 ) of the part; a material (512) of the part; a location (513) of the part; and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: evaluating (1210) at least one of the geometry (511), the material (512) and the location (513) of the part.
32. The system (2000) of claim 30 or claim 31 , wherein the compatible grasp configuration subset (404) is represented as a Boolean sequence and assembling (1300) the compatible grasp configuration subset (404) across two or more equipment states (520) is achieved by performing an intersection operation over the Boolean sequence of the two or more equipment states (520).
33. The system (2000) of any one of claims 30 to 32, wherein the one or more processors (210), when assembling (1200) the eligible grasp configuration subsets (402), are configured to: compute (1250) a safety factor indicative of a likelihood to accidentally drop the part; and select (1260) at least one grasp configuration from the plurality of grasp configurations (400) when the safety factor is above a minimum safety threshold.
34. The system (2000) of claim 33, further comprising at least one of: a temperature sensor for collecting a temperature measurement (515); and a humidity sensor for collecting a temperature measurement (515); wherein computing (1250) the safety factor comprises: adjusting (1254) the safety factor according to at least one of the temperature measurement (515) and the relative humidity measurement (516).
35. The system (2000) of claim 33 or claim 34, further comprising: a positioning module configured to:- capture a relative positioning coordinate (410) of a grasp point on a part from the one or more parts (50); and- capture a grasp orientation (420);wherein computing (1250) the safety factor comprises: evaluating (1252) the safety factor when the part is held by the robot (40) from the relative positioning coordinate (410) and the grasp orientation (420).
36. The system (2000) of claim 35, wherein the relative positioning coordinate (410) of the grasp point comprises: a selected anchor point (411) within a plurality of anchor points on the part; a distance relative to the selected anchor point (412); and wherein the distance relative to the selected anchor point (412) is proportional to a dimension of the part, thereby allowing the relative positioning coordinate (410) to adapt to a variety of part geometries.
37. The system (2000) of any one of claims 33 to 36, wherein the one or more processors (210), when executing (1500) the optimal grasp configuration (406), are configured to: adjust (1510) an acceleration of the robot (40) based on the safety factor.
38. The system (2000) of any one of claims 30 to 37, wherein the state of the production line (500) further comprises: a collision avoidance parameter (540) describing potential collisions within a proximity of the robot (40); and wherein assembling (1200) the eligible grasp configuration subsets (402) comprises: selecting (1225) at least one grasp configuration from the plurality of grasp configurations (400) when the collision avoidance parameter (540) suggests avoidance of a collision.
39. The system (2000) of any one of claims 30 to 38, wherein the state of the production line (500) further comprises: a predefined rule (550) for handling at least one of a specific part (551) comprising a specific feature (552), and a specific process (555) comprising a specific assembly step (556); and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: selecting (1227) at least one grasp configuration (400) from the plurality of grasp configurations (400) when the predefined rule (550) is compatible with the grasp configuration (400).
40. The system (2000) of claim 39, wherein the specific feature (552) is any one of a hole, a groove, and an irregular shape.
41. The system (2000) of any one of claims 30 to 40, wherein the one or more processors (210) are further configured to collect (1100) the state of the production line (500), assemble (1200) the eligible grasp configuration subsets (402), assemble (1300) the compatible grasp configuration subset (404) and select (1400) the optimal grasp configuration (406) repeatably in real time and wherein the robot (40) operates continuously and uninterruptedly.
42. The system (2000) of any one of claims 30 to 41 , wherein an equipment state of the one or more equipment states (520) comprises:a geometrical configuration (521) of an equipment comprising:- a workspace surface (522); and- a reachability (526) of the equipment from the robot (40); and wherein assembling (1300) the compatible grasp configuration subset (404) comprises: selecting (1310) at least one eligible grasp configuration from the eligible grasp configuration subsets (402) resulting in the part being stable and operational therewith when placed onto the workspace surface (522) across the reachability (526) of the equipment.
43. The system (2000) of claim 42, wherein the geometrical configuration (521) further comprises: one or more vertical obstacles (523); and wherein assembling (1300) the compatible grasp configuration subset (404) further comprises: selecting (1320) at least one eligible grasp configuration from the eligible grasp configuration subsets (402) when the one or more vertical obstacles (523) suggest avoidance of a collision therewith when transporting the part onto the workspace surface (522) across the reachability (526) of the equipment.
44. The system (2000) of claim 42 or claim 43, wherein the equipment state further comprises: a current part occupancy (528) of the workspace surface (522); and wherein assembling (1300) the compatible grasp configuration subset (404) comprises: discarding (1330) at least one grasp configuration from the eligible grasp configuration subsets (402) resulting in the part being unsafe when placed therewith onto the current part occupancy (528) of the workspace surface (522).
45. The system (2000) of any one of claims 30 to 44, wherein the state of the production line (500) further comprises: a priority list (560); an optimization strategy (570); and wherein selecting (1400) the optimal grasp configuration (406) comprises: for a compatible grasp configuration in the compatible grasp configuration subset (404):- computing (1410) a compatible grasp configuration score according to the optimization strategy (570), the priority list (560) and the robot state (530); and thereby optimizing (1470) the optimal grasp configuration according to the optimization strategy (570).
46. The system (2000) of claim 45, wherein the one or more processors (210), when selecting (1400) the optimal grasp configuration (406), are configured to: compare (1420) the compatible grasp configuration score against a second score of a second grasp configuration; and select (1430) the compatible grasp configuration from the compatible grasp configuration subset (404) when the compatible grasp configuration score is higher than the second score of the second grasp configuration.
47. The system (2000) of claim 45 or claim 46, wherein the optimization strategy (570) optimizes (1470) at least one of an equipment usage score, a throughput score, a safety score, a minimized travel time score, and a clearance score.
48. The system (2000) of claim 47, wherein the equipment usage score, the throughput score, the safety score and the clearance score are adjusted according to a diminishing marginal returns function.
49. The system (2000) of any one of claims 45 to 48, wherein the processing equipment (60) comprises at least one of: a processing equipment; an accumulation equipment; and a conveying equipment; and wherein the priority list (560) comprises at least one of transferring the part to the processing equipment, transferring the part to the accumulation equipment and transferring the part to the conveying equipment.
50. The system (2000) of any one of claims 30 to 49, wherein the robot (70) comprises: one or more gripping tools (100) configured to engage the one or more parts (50); wherein the plurality of grasp configurations (400) comprises: a plurality of engagement points (414) for identifying one or more locations of engagement between the part and the one or more gripping tools (100); and a plurality of engagement point activation parameters (415) for activating the one or more gripping tools (100); and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: disabling (1230) at least one engagement point activation parameter from the plurality of engagement point activation parameters (415), thereby preventing a second part proximate to the part from being picked.51 . The system (2000) of claim 50, wherein the state of the production line (500) further comprises: a predefined rule (550) for handling a specific part (551) comprising a specific feature (552) consisting of a weak engagement point; and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: disabling (1240) at least one engagement point activation parameters (415) from the plurality of engagement point activation parameters (415), thereby avoiding engaging the weak engagement point with one or more gripping tools (100).
52. The system (2000) of claim 50 or claim 51 , wherein the plurality of engagement point activation parameters (415) is represented as a Boolean sequence and the part and the second part are held by performing a union operation over the Boolean sequence.
53. The system (2000) of any one of claims claim 50 to 52, wherein the plurality of engagement points (414) comprises at least 8 engagement points (414).
54. The system (2000) of any one of claims 50 to 53, wherein at least one gripping tool from the one or more gripping tools (100) comprises a vacuum system, uses a pressure differential to engage the part and wherein the pressure differential is generated according to the plurality of engagement point activation parameters (415).
55. The system (2000) of any one of claims 50 to 54, wherein the robot state (530) comprises: one or more previous grasp configurations of one or more other currently held parts (532); and wherein selecting (1400) the optimal grasp configuration (406) comprises: discarding (1440) one or more compatible grasp configurations posing a risk of dropping at least one of the one or more other currently held parts when picking the part.
56. The system (2000) of claim 55, further wherein at least one gripping tool within the one or more gripping tools (100) comprises: an actuator configured to retractably extend the gripping tools (100); wherein the grasp configuration (400) further comprises: one or more extension parameters (430); and wherein the one or more processors (210) are configured to when executing (1500) the optimal grasp configuration (406):- position (1520) the at least one gripping tool according to the one or more extension parameters (430), thereby allowing the part and the one or more other currently held parts to overlap.
57. The system (2000) of claim 55 or claim 56, wherein the one or more processors (210) are further configured to: compute (1450) a risk of a collision of the part when transporting the one or more other currently held parts; and reject (1460) the optimal grasp configuration (406) when the risk of collision is above a risk of collision threshold.
58. The system (2000) of any one of claims 30 to 57, wherein the one or more equipment states (520) comprise an equipment state for any one of a doweling jig station, a welding station, a painting and coating station, an inspection station, a deburring station, a gluing and sealing station, an accumulation table, a storage station, a palletizing station, a conveyor, a transfer station, a turntable, a nesting router, a pod and rail router, a vertical CNC, a CNC-Drilling and dowel insertion machine, a horizontal boring machine, a panel dividing saw, a hardware insertion, a waterjet cutting center, a disk cutting center and other CNC machining centers, an edgebanding machine, a sanding machine, a polishing machine, an assembly system and a storage and retrieval system.
59. The system (2000) of any one of claims 30 to 58, further comprising: a general-purpose processing unit (250) comprising at least one of the one or more processors (210);a programmable logic controller unit (260) comprising at least one of the one or more processors (210); and a robot controller unit (270) comprising at least one of the at least one of the one or more processors (210).
60. A non-transitory computer-readable medium storing a set of instructions for controlling a robot (40) on a production line, the set of instructions comprising: one or more instructions that, when executed by one or more processors (210) of a device, cause the device to:- collect (1100) a state of the production line (500) comprising- one or more part states (510)- one or more equipment states (520); and- a robot state (530);- assemble (1200) an eligible grasp configuration subsets (402) from a plurality of grasp configurations (400) and at least one part state of the one or more part states (510);- assemble (1300) a compatible grasp configuration subset (404) from the eligible grasp configuration subsets (402) and at least one equipment state from the one or more equipment states (520);- select (1400) an optimal grasp configuration (406) from the compatible grasp configuration subset (404) and the robot state (530); and- execute (1500) the optimal grasp configuration (406) with the robot (40) to perform at least one of picking action and placing action of a part.61 . The non-transitory computer-readable medium of claim 60, wherein a part state of the one or more part states (510) comprise at least one of: a geometry (511 ) of the part; a material (512) of the part; a location (513) of the part; and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: evaluating (1210) at least one of the geometry (511), the material (512) and the location (513) of the part.
62. The non-transitory computer-readable medium of claim 60 or claim 61 , wherein the compatible grasp configuration subset (404) is represented as a Boolean sequence and assembling (1300) the compatible grasp configuration subset (404) across two or more equipment states (520) is achieved by performing an intersection operation over the Boolean sequence of the two or more equipment states (520).
63. The non-transitory computer-readable medium of any one of claims 60 to 62, wherein the one or more instructions, when assembling (1200) the eligible grasp configuration subsets (402), cause the device to:compute (1250) a safety factor indicative of a likelihood to accidentally drop the part; and select (1260) at least one grasp configuration from the plurality of grasp configurations (400) when the safety factor is above a minimum safety threshold.
64. The non-transitory computer-readable medium of claim 63, wherein the one or more part states (510) further comprises at least one of: a temperature measurement (515); a relative humidity measurement (516); and wherein computing (1250) the safety factor comprises: adjusting (1254) the safety factor according to at least one of the temperature measurement (515) and the relative humidity measurement (516).
65. The non-transitory computer-readable medium of claim 63 or claim 64, wherein the plurality of grasp configurations (400) comprises: a relative positioning coordinate (410) of a grasp point on the part; a grasp orientation (420); and wherein computing (1250) the safety factor comprises: evaluating (1252) the safety factor when the part is held by the robot (40) from the relative positioning coordinate (410) and the grasp orientation (420).
66. The non-transitory computer-readable medium of claim 65, wherein the one or more instructions, when the relative positioning coordinate (410) of the grasp point, cause the device to: a selected anchor point (411) within a plurality of anchor points on the part; a distance relative to the selected anchor point (412); and wherein the distance relative to the selected anchor point (412) is proportional to a dimension of the part, thereby allow the relative positioning coordinate (410) to adapt to a variety of part geometries.
67. The non-transitory computer-readable medium of any one of claims 63 to 66, wherein the one or more instructions, when executing (1500) the optimal grasp configuration (406), cause the device to: adjust (1510) an acceleration of the robot (40) based on the safety factor.
68. The non-transitory computer-readable medium of any one of claims 60 to 67, wherein the state of the production line (500) further comprises: a collision avoidance parameter (540) describing potential collisions within a proximity of the robot (40); and wherein assembling (1200) the eligible grasp configuration subsets (402) comprises: selecting (1225) at least one grasp configuration from the plurality of grasp configurations (400) when the collision avoidance parameter (540) suggests avoidance of a collision.
69. The non-transitory computer-readable medium of any one of claims 60 to 68, wherein the state of the production line (500) further comprises:a predefined rule (550) for handling at least one of a specific part (551) comprising a specific feature (552), and a specific process (555) comprising a specific assembly step (556); and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: selecting (1227) at least one grasp configuration (400) from the plurality of grasp configurations (400) when the predefined rule (550) is compatible with the grasp configuration (400).
70. The non-transitory computer-readable medium of claim 69, wherein the specific feature (552) is any one of a hole, a groove, and an irregular shape.71 . The non-transitory computer-readable medium of any one of claims 60 to 70, wherein collecting (1100) the state of the production line (500), assembling (1200) the eligible grasp configuration subsets (402), assembling (1300) the compatible grasp configuration subset (404) and selecting (1400) the optimal grasp configuration (406) is performed repeatably in real time and wherein the robot (40) operates continuously and uninterruptedly.
72. The non-transitory computer-readable medium of any one of claims 60 to 71, wherein the one or more instructions further cause the device to: performed repeatably in real time and wherein the robot (40) operates continuously and uninterruptedly.
73. The non-transitory computer-readable medium of any one of claims 60 to 72, wherein an equipment state of the one or more equipment states (520) comprises: a geometrical configuration (521) of an equipment comprising:- a workspace surface (522); and- a reachability (526) of the equipment from the robot (40); and wherein assembling (1300) the compatible grasp configuration subset (404) comprises: selecting (1310) at least one eligible grasp configuration from the eligible grasp configuration subsets (402) resulting in the part being stable and operational therewith when placed onto the workspace surface (522) across the reachability (526) of the equipment.
74. The non-transitory computer-readable medium of claim 73, wherein the geometrical configuration (521) further comprises: one or more vertical obstacles (523); and wherein assembling (1300) the compatible grasp configuration subset (404) further comprises: selecting (1320) at least one eligible grasp configuration from the eligible grasp configuration subsets (402) when the one or more vertical obstacles (523) suggest avoidance of a collision therewith when transporting the part onto the workspace surface (522) across the reachability (526) of the equipment.
75. The non-transitory computer-readable medium of claim 73 or claim 74, wherein the equipment state further comprises: a current part occupancy (528) of the workspace surface (522); andwherein assembling (1300) the compatible grasp configuration subset (404) comprises: discarding (1330) at least one grasp configuration from the eligible grasp configuration subsets (402) resulting in the part being unsafe when placed therewith onto the current part occupancy (528) of the workspace surface (522).
76. The non-transitory computer-readable medium of any one of claims 60 to 75, wherein the state of the production line (500) further comprises: a priority list (560); an optimization strategy (570); and wherein selecting (1400) the optimal grasp configuration (406) comprises: for a compatible grasp configuration in the compatible grasp configuration subset (404):- computing (1410) a compatible grasp configuration score according to the optimization strategy (570), the priority list (560) and the robot state (530); and thereby optimizing (1470) the optimal grasp configuration according to the optimization strategy (570).
77. The non-transitory computer-readable medium of claim 75 or claim 76, wherein the one or more instructions, when selecting (1400) the optimal grasp configuration (406), cause the device to: compare (1420) the compatible grasp configuration score against a second score of a second grasp configuration; and select (1430) the compatible grasp configuration from the compatible grasp configuration subset (404) when the compatible grasp configuration score is higher than the second score of the second grasp configuration.
78. The non-transitory computer-readable medium of claim 75, wherein the optimization strategy (570) optimizes (1470) at least one of an equipment usage score, a throughput score, a safety score, a minimized travel time score, and a clearance score.
79. The non-transitory computer-readable medium of claim 77 or claim 78, wherein the equipment usage score, the throughput score, the safety score and the clearance score are adjusted according to a diminishing marginal returns function.
80. The non-transitory computer-readable medium of any one of claims 75 to 79, wherein the priority list (560) comprises at least one of transferring the part to a processing equipment, transferring the part to an accumulation equipment and transferring the part to a conveying equipment.81 . The non-transitory computer-readable medium of any one of claims 60 to 80, wherein the plurality of grasp configurations (400) comprises: a plurality of engagement points (414) for identifying one or more locations of engagement between the part and one or more gripping tools (100); and a plurality of engagement point activation parameters (415) for activating the one or more gripping tools (100); and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises:disabling (1230) at least one engagement point activation parameter from the plurality of engagement point activation parameters (415), thereby preventing a second part proximate to the part from being picked.
82. The non-transitory computer-readable medium of claim 80 or claim 81 , wherein the state of the production line (500) further comprises: a predefined rule (550) for handling a specific part (551) comprising a specific feature (552) consisting of a weak engagement point; and wherein assembling (1200) the eligible grasp configuration subsets (402) further comprises: disabling (1240) at least one engagement point activation parameters from the plurality of engagement point activation parameters (415), thereby avoiding engaging the weak engagement point with one or more gripping tools (100).
83. The non-transitory computer-readable medium of any one of claims 80 to 82, wherein the plurality of engagement point activation parameters (415) is represented as a Boolean sequence and the part and the second part are held by performing a union operation over the Boolean sequence.
84. The non-transitory computer-readable medium of any one of claims 80 to 83 wherein the plurality of engagement points (414) comprises at least 8 engagement points (414)85. The non-transitory computer-readable medium of any one of claims 80 to 84, wherein at least one gripping tool from the one or more gripping tools (100) uses a pressure differential to engage the part and the pressure differential is generated according to the plurality of engagement point activation parameters (415).
86. The non-transitory computer-readable medium of any one of claims 80 to 85, wherein the robot state (530) comprises: one or more previous grasp configurations of one or more other currently held parts (532); and wherein selecting (1400) the optimal grasp configuration (406) comprises: discarding (1440) one or more compatible grasp configurations posing a risk of dropping at least one of the one or more other currently held parts when picking the part.
87. The non-transitory computer-readable medium of claim 85 or claim 86, wherein at least one gripping tool within the one or more gripping tools (100) is configured to retractably extend and wherein the grasp configuration (400) further comprises: one or more extension parameters (430); and wherein executing (1500) the optimal grasp configuration (406) comprise: positioning (1520) the at least one gripping tool according to the one or more extension parameters (430), thereby allowing the part and the one or more other currently held parts to overlap.
88. The non-transitory computer-readable medium of any one of claims 85 to 87, wherein the one or more instructions further cause the device to:compute (1450) a risk of a collision of the part when transporting the one or more other currently held parts; and reject (1460) the optimal grasp configuration (406) when the risk of collision is above a risk of collision threshold.
89. The non-transitory computer-readable medium of any one of claims 60 to 88, wherein the one or more equipment states (520) comprise an equipment state for any one of a doweling jig station, a welding station, a painting and coating station, an inspection station, a deburring station, a gluing and sealing station, an accumulation table, a storage station, a palletizing station, a conveyor, a transfer station, a turntable, a nesting router, a pod and rail router, a vertical CNC, a CNC-Drilling and dowel insertion machine, a horizontal boring machine, a panel dividing saw, a hardware insertion, a waterjet cutting center, a disk cutting center and other CNC machining centers, an edgebanding machine, a sanding machine, a polishing machine, an assembly system and a storage and retrieval system.
Citation Information
Patent Citations
Vacuum gripping apparatus
US20100135760A1
Optimization-based spring lattice deformation model for soft materials
US20190291282A1
Task-specific robot grasping system and method
US20200189105A1
Systems and methods for determining pose of objects held by flexible end effectors
US20210101292A1
Handling device and control device
US20210291366A1