System, method, and computer program product for interacting with an object with a robotic device

Abstract

Provided are systems, methods, and devices for identifying contact regions on an object for interaction by a robotic device. A system includes at least one computing device configured to determine a configuration space within the boundary of an object, determine a plurality of segments from partitioning the configuration space, determine a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space, and control a robotic device to interact with the object based on the at least one contact region.

Description

Attorney Docket No. 08993-2600638SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR INTERACTING WITH AN OBJECT WITH A ROBOTIC DEVICE CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Patent Application No.63 / 762,228, filed February 24, 2025, the disclosure of which is hereby incorporated by reference in its entirety.GOVERNMENT LICENSE RIGHTS

[0002] This invention was made with Government support under 80NSSC17K0140 awarded by the National Aeronautics and Space Administration (NASA). The Government has certain rights in the invention.BACKGROUND1. Field

[0003] This disclosure relates generally to robotic devices and, in non-limiting embodiments, systems, methods, and devices for interacting with an object with a robotic device and identifying contact regions on the object.2. Technical Considerations

[0004] Independent contact regions (ICRs) are recognized as a foundational tool for planning an interaction between a robotic device and an object (e.g., such as grasping the object). Large ICRs imply that the grasp can accommodate uncertainties and have geometries conducive to contact. However, ICR computation has been plagued by computational complexity. The search space for ICRs is exponential in the number of contacts, and existing algorithms and approaches can only be used in special instances or for locally optimal solutions.

[0005] Algorithms for grasping and performing other manipulations of objects rely on measures of optimality or quality metrics. These methods have produced grasps as sets of precise target contact points. This point-based approach leads to grasps that are impractical for a robot, such as grasping a small surface element or location that experiences instability due to position errors. Quality metrics have evolved to include the consideration of dynamics, uncertainty, sensing, and policy performance evaluation based on real-world data and simulation. However, determining good policies remains challenging, and many manipulation demonstrations benefit from the physical understanding and control of contacts and forces.6B54195.DOCX Page 1 of 22Attorney Docket No. 08993-2600638SUMMARY

[0006] According to non-limiting embodiments or aspects, provided is a method for identifying contact regions on an object for interaction by a robotic device, comprising: determining, with at least one computing device, a configuration space within the boundary of an object; determining, with at least one computing device, a plurality of segments from partitioning the configuration space; determining, with at least one computing device, a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and controlling a robotic device to interact with the object based on the set of contact regions.

[0007] In non-limiting embodiments or aspects, the boundary of the object is mapped to a mathematical representation according to the dimensionality of the object. In non-limiting embodiments or aspects, wherein a n-dimensional object is mapped to a (n-l)-dimensional interval. In non-limiting embodiments or aspects, the method includes determining the configuration space, the configuration space is represented by a d-fold cartesian product of the boundary of the object. In non-limiting embodiments or aspects, the method includes creating an order simplex by applying an ordering constraint to the configuration space. In non-limiting embodiments or aspects, each set of contact regions in the configuration space is evaluated as valid or invalid. In non-limiting embodiments or aspects, disjoint and continuous valid regions in the configuration space form the set of contact regions. In non-limiting embodiments or aspects, the configuration space is partitioned with the Delaunay Triangulation method. In non-limiting embodiments or aspects, wherein a circle encloses all simplex partitions created from the Delaunay Triangulation method. In non-limiting embodiments or aspects, the method includes, for each iteration, computing a box (e.g., a maximum sized box that fits within the circle and / or other constraints) until an invalid point is found, the set of contact points is based on a valid box from multiple iterations.

[0008] According to non-limiting embodiments or aspects, provided is a system for identifying contact regions on an object for interaction by a robotic device, comprising at least one computing device configured to: determine a configuration space within the boundary of an object; determine a plurality of segments from partitioning the configuration space; determine a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and control a robotic device to interact with the object based on the set of contact regions.6B54195.DOCX Page 2 of 22Attorney Docket No. 08993-2600638

[0009] In non-limiting embodiments or aspects, the boundary of the object is mapped to a mathematical representation according to the dimensionality of the object. In non-limiting embodiments or aspects, wherein a n-dimensional object is mapped to a (n-l)-dimensional interval. In non-limiting embodiments or aspects, the at least one computing device is further configured to: determine the configuration space, the configuration space is represented by a d-fold cartesian product of the boundary of the object. In non-limiting embodiments or aspects, the at least one computing device is further configured to: create an order simplex by applying an ordering constraint to the configuration space. In non-limiting embodiments or aspects, each contact region in the configuration space is evaluated as valid or invalid. In nonlimiting embodiments or aspects, disjoint and continuous valid regions in the configuration space form the set of contact regions. In non-limiting embodiments or aspects, the configuration space is partitioned with the Delaunay Triangulation method. In non-limiting embodiments or aspects, a circle encloses all simplex partitions created from the Delaunay Triangulation method. In non-limiting embodiments or aspects, the at least one computing device is further configured to: for each iteration, computing a box that fits within the circle (e.g., a maximum sized box for the size of the circle and / or other constraints) until an invalid point is found, the set of contact points is based on a valid box from multiple iterations. According to nonlimiting embodiments or aspects, provided is a computer program product for identifying contact regions on an object for interaction by a robotic device, comprising at least one non-transitory computer-readable medium including instructions that, when executed by at least one computing device, cause the at least one computing device to: determine a configuration space within the boundary of an object; determine a plurality of segments from partitioning the configuration space; determine a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and control a robotic device to interact with the object based on the at least one contact region.

[0010] Other non-limiting embodiments or aspects will be set forth in the following numbered clauses:

[0011] Clause 1 : A method for identifying contact regions on an object for interaction by a robotic device, comprising: determining, with at least one computing device, a configuration space within the boundary of an object; determining, with at least one computing device, a plurality of segments from partitioning the configuration6B54195.DOCX Page 3 of 22Attorney Docket No. 08993-2600638space; determining, with at least one computing device, a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and controlling a robotic device to interact with the object based on the set of contact regions.

[0012] Clause 2: The method of clause 1 , wherein the boundary of the object is mapped to a mathematical representation according to the dimensionality of the object.

[0013] Clause 3: The method of any of clauses 1 -2, wherein a n-dimensional object is mapped to a (n-l)-dimensional interval.

[0014] Clause 4: The method of any of clauses 1 -3, further comprising: determining the configuration space, wherein the configuration space is represented by a d-fold cartesian product of the boundary of the object.

[0015] Clause 5: The method of any of clauses 1 -4, further comprising creating an order simplex by applying an ordering constraint to the configuration space.

[0016] Clause 6: The method of any of clauses 1-5, wherein each set of contact regions in the configuration space is evaluated as valid or invalid.

[0017] Clause 7: The method of any of clauses 1 -6, wherein disjoint and continuous valid regions in the configuration space form the set of contact regions.

[0018] Clause 8: The method of any of clauses 1-7, wherein the configuration space is partitioned with the Delaunay Triangulation method.

[0019] Clause 9: The method of any of clauses 1-8, wherein a circle encloses all simplex partitions created from the Delaunay Triangulation method.

[0020] Clause 10: The method of any of clauses 1-9, further comprising: for each iteration, computing a maximum sized box that fits within the circle until an invalid point is found, wherein the set of contact points is based on a valid box from multiple iterations.

[0021] Clause 11 : A system for identifying contact regions on an object for interaction by a robotic device, comprising at least one computing device configured to: determine a configuration space within the boundary of an object; determine a plurality of segments from partitioning the configuration space; determine a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and control a robotic device to interact with the object based on the set of contact regions.6B54195.DOCX Page 4 of 22Attorney Docket No. 08993-2600638

[0022] Clause 12: The system of clause 11 , wherein the boundary of the object is mapped to a mathematical representation according to the dimensionality of the object.

[0023] Clause 13: The system of any of clauses 11-12, wherein a n-dimensional object is mapped to a (n-l)-dimensional interval.

[0024] Clause 14: The system of any of clauses 11-13, wherein the at least one computing device is further configured to: determine the configuration space, wherein the configuration space is represented by a d-fold cartesian product of the boundary of the object.

[0025] Clause 15: The system of any of clauses 11-14, wherein the at least one computing device is further configured to: create an order simplex by applying an ordering constraint to the configuration space.

[0026] Clause 16: The system of any of clauses 11-15, wherein each contact region in the configuration space is evaluated as valid or invalid.

[0027] Clause 17: The system of any of clauses 11-16, wherein disjoint and continuous valid regions in the configuration space form the set of contact regions.

[0028] Clause 18: The system of any of clauses 11-17, wherein the configuration space is partitioned with the Delaunay Triangulation method.

[0029] Clause 19: The system of any of clauses 11-18, wherein a circle encloses all simplex partitions created from the Delaunay Triangulation method.

[0030] Clause 20: The system of any of clauses 11-19, wherein the at least one computing device is further configured to: for each iteration, computing a maximum sized box that fits within the circle until an invalid point is found, wherein the set of contact points is based on a valid box from multiple iterations.

[0031] Clause 21 : A computer program product for identifying contact regions on an object for interaction by a robotic device, comprising at least one non-transitory computer-readable medium including instructions that, when executed by at least one computing device, cause the at least one computing device to: determine a configuration space within the boundary of an object; determine a plurality of segments from partitioning the configuration space; determine a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and control a robotic device to interact with the object based on the at least one contact region.6B54195.DOCX Page 5 of 22Attorney Docket No. 08993-2600638

[0032] These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structures and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the invention.BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Additional advantages and details are explained in greater detail below with reference to the non-limiting, exemplary embodiments that are illustrated in the accompanying schematic figures, in which:

[0034] FIG. 1 illustrates a schematic diagram for a system for interacting with an object by a robotic device according to non-limiting embodiments or aspects;

[0035] FIG. 2 illustrates a flow diagram for a method of interacting with an object by a robotic device according to non-limiting embodiments or aspects;

[0036] FIG. 3 illustrates a configuration space for a system for interacting with an object by a robotic device according to non-limiting embodiments or aspects;

[0037] FIG. 4 illustrates example contact regions for multiple dimensions for a system for interacting with an object by a robotic device according to non-limiting embodiments or aspects;

[0038] FIG. 5 illustrates a diagram of geometries used in an algorithm for identifying contact regions for a system for interacting with an object by a robotic device according to non-limiting embodiments or aspects;

[0039] FIG. 6 illustrates an algorithm for identifying contact regions for a system for interacting with an object by a robotic device according to non-limiting embodiments or aspects;

[0040] FIG. 7 illustrates program code for an algorithm for identifying contact regions for a system for interacting with an object by a robotic device according to nonlimiting embodiments or aspects; and

[0041] FIG. 8 is a diagram of components of one or more devices of FIG. 1 according to non-limiting aspects or embodiments.6B54195.DOCX Page 6 of 22Attorney Docket No. 08993-2600638DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0042] For purposes of the description hereinafter, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and derivatives thereof shall relate to the embodiments as they are oriented in the drawing figures. However, it is to be understood that the embodiments may assume various alternative variations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary embodiments or aspects of the invention. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects disclosed herein are not to be considered as limiting.

[0043] No aspect, component, element, structure, act, step, function, instruction, and / or the like used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more” and “at least one.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, and / or the like) and may be used interchangeably with “one or more” or “at least one.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise.

[0044] As used herein, the term “communication” may refer to the reception, receipt, transmission, transfer, provision, and / or the like, of data (e.g., information, signals, messages, instructions, commands, and / or the like). For one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to be in communication with another unit means that the one unit is able to directly or indirectly receive information from and / or transmit information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. Additionally, two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives6B54195.DOCX Page 7 of 22Attorney Docket No. 08993-2600638information and does not actively transmit information to the second unit. As another example, a first unit may be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit.

[0045] As used herein, the term “computing device” may refer to one or more electronic devices configured to process data. A computing device may, in some examples, include the necessary components to receive, process, and output data, such as a display, a processor, a memory, an input device, and a network interface. A computing device may be a mobile device. The computing device may also be a desktop computer or other form of non-mobile computer. An “interface” refers to a generated display, such as one or more graphical user interfaces (GUIs) with which a user may interact, either directly or indirectly (e.g., through a keyboard, mouse, touchscreen, etc.).

[0046] Reference to “a computing device” or “a processor,” as used herein, may refer to a previously-recited computing device and / or processor that is recited as performing a previous step or function, a different computing device and / or processor, and / or a combination of computing devices and / or processors. For example, as used in the specification and the claims, a first computing device and / or a first processor that is recited as performing a first step or function may refer to the same or different computing device and / or a processor recited as performing a second step or function.

[0047] As used herein, the term “application programming interface” (API) may refer to computer code that allows communication between different systems or (hardware and / or software) components of systems. For example, an API may include function calls, functions, subroutines, communication protocols, fields, and / or the like usable and / or accessible by other systems or other (hardware and / or software) components of systems.

[0048] Non-limiting embodiments improve upon existing approaches of identifying contact regions, such as brute force approaches, by providing computational efficiencies and improved processing speeds, allowing for real-time interaction between a robotic device and an object. Non-limiting embodiments improve upon existing methods by reducing latency and the use of computational resources, allowing for real-time applications with an efficient use of available computing power. For example, using an existing brute force technique, determining four (4) or more contacts may result in memory errors and a failure to compute. Through6B54195.DOCX Page 8 of 22Attorney Docket No. 08993-2600638implementation of non-limiting embodiments described herein, four (4) or more contact points may be advantageously determined using high-dimensionality techniques without resulting in memory or other computational errors.

[0049] Further, non-limiting embodiments described herein provide for improved real-world outcomes (e.g., less errors manipulating / grasping objects) by using contact regions, which are a volume-based quality metric, rather than contact points. Using contact regions results in faster processing and better results with real-world constraints, dynamics, and uncertainties. By identifying contact regions (e.g., independent contact regions (ICRs)) on an object through fast, low-latency techniques, constraints and uncertainties of real-world objects can be overcome. In simulated experiments using five (5) variations of shaped objects to introduce uncertainties with respect to different sized / scaled objects, contact regions determined using non-limiting embodiments described herein had an average success rate of 92% as compared to a success rate of 26% for point-based grasps. In real-world, physical experiments, contact regions determined using non-limiting embodiments described herein had a 100% success rate when uncertainties were introduced and point-based approaches had a 0% success rate.

[0050] Non-limiting embodiments described herein provide for an incremental discovery of invalid contact points to guide the identification of valid contact regions, allowing for object manipulation when invalid points are not known in advance. Using contact regions provides guidance for hand-object contact placement in languagebased interfaces, improves the ability to interpret and retarget human demonstrations, results in more physically informed grasp filters, and enhances the ability to co-design hand shapes and product geometries. Further benefits and advantages will be understood by those skilled in the art.

[0051] Referring to FIG. 1 , a system 1000 for interacting with an object 106 is shown according to non-limiting embodiments. The system 1000 includes a computing device 100 in communication with a robotic device 102. The robotic device 102 may include, for example, a gripping and / or manipulation device such as a robotic hand and / or the like. A manipulation device may include, for example, a device with two or more appendages (e.g., such as fingers) and / or end effectors capable of grasping and / or manipulating an object 106 from one or more contact points. Any number of appendages and contact points may be used in non-limiting embodiments. As a nonlimiting example, the robotic device 102 may include the manipulation device6B54195.DOCX Page 9 of 22Attorney Docket No. 08993-2600638described in U.S. Patent No. 12,282,710, titled “Flexible manipulation device and method for fabricating the same.” It will be appreciated that various types of robotic devices may be used with non-limiting embodiments. In non-limiting embodiments, the computing device 100 may autonomously control the robotic device 102 and / or may control the robotic device with input from a user, such as input through the computing device 100 and / or another computing device in communication with the computing device 100.

[0052] With continued reference to FIG. 1 , the computing device 100 may be in communication with image data 104 stored in a data storage device (e.g., internal memory of the computing device 100 and / or any other form of data storage). The image data 104 may include one or more images and / or representations of one of more objects 106. For example, the image data 104 may include boundary data, 3D graphical data (e.g., point cloud data or the like), 2D graphical data, vector representations, and / or the like. In some non-limiting embodiments, the image data 104 may be obtained in real-time or near-real-time (e.g., while the robotic device 102 is being controlled) with one or more sensors 108 (e.g., cameras, optical sensors, LiDAR sensors, and / or the like) in communication with the computing device. In some non-limiting embodiments, the image data 104 may be predefined and known by the system. The object 106 may include any physical object capable of being picked up and / or manipulated by the robotic device 102.

[0053] As used herein, the term “contact region” refers to an area on an object that includes multiple potential points (e.g., contact points). A contact region may be discrete (e.g., an ICR). The term “optimal contact region” may refer to a contact region that has been determined to be better suited for a task (e.g., such as grasping) than at least some other contact regions. There may exist multiple optimal contact regions out of a set of possible contact regions in some non-limiting embodiments.

[0054] Referring to FIG. 3, diagrams for determining contact regions of an object to interact with are shown according to non-limiting embodiments. In the example in FIG. 3, for purpose of explanation, it is assumed that the object is 2D with a contact on the perimeter of the boundary 302. FIG. 3 shows a determination of two contact regions as a grasp, although it will be appreciated that any number of contact regions may be determined and that the object may be 3D in a real-world implementation. The shape of the boundary 302 of the object is shown as a curve parametrized on [0, L] with frictional contacts and contact regions. FIG. 3 also shows the configuration space6B54195.DOCX Page 10 of 22Attorney Docket No. 08993-2600638304 with “Contact 1” on the x-axis and “Contact 2” on the y-axis. The configuration space 304 is shown as an order simplex with force closure grasps (e.g., regions) shown as clusters of points. FIG. 3 also shows themetric distance transformation 306, which computes the furthest interior point in the Chebyshev norm, corresponding to the largest square (e.g., hypercube with additional dimensions) contained in the force closure space. The box 312 corresponds to a set of contact regions (e.g., 314, 316). As used herein, the terms “square” and “box” may refer to a 2D square, a 3D cube, and / or a hypercube of n-dimensions. The dimensionality may depend on the number of contact regions to be identified in some non-limiting embodiments.

[0055] With continued reference to FIG. 3, in non-limiting embodiments, the object perimeter may be discretized into L intervals. Contact 1 may be placed anywhere along the perimeter (e.g., boundary 302). In non-limiting embodiments, the contacts may be assumed to be ordered and non-overlapping. However, in some non-limiting embodiments the contact regions may overlap. Contact 2 may be placed at a location with greater index than Contact 1 , resulting in a simplex 304 of possible pairs of contact points. Each pair of contact points may be evaluated based on any desired quality metric to determine whether it is “valid” or “invalid.” Valid contacts in FIG. 3 are shown in the simplex 304 as filled-in cells. Once all possible pairs of contacts are classified, a set of optimal contact regions are found as the largest axis aligned box (e.g., 312) having all valid grasps and contained within the simplex 304. Themetric distance transform may be utilized to identify the optimal result. In non-limiting embodiments, the optimal contact regions may be highlighted, along with the corresponding contact regions 314, 316 on the object perimeter 302. In non-limiting embodiments, each box may correspond to a set of contact regions.

[0056] FIG. 4 illustrates examples of contact regions 422, 424, 426 on different objects where dimensionality (e.g., 2D, 3D, 4D, 5D, 6D, 7D) increases with the number of contact regions 422, 424, 426. For example, two contact regions may be approached as a 2D problem, three contact regions may be approached as a 3D problem, and so on. Non-limiting embodiments may be used to determine any number of contact regions to suit any type of robotic device that interacts with the object. For example, non-limiting embodiments may determine two, three, four, five, six, seven, and / or more contact regions in different implementations.6B54195.DOCX Page 11 of 22Attorney Docket No. 08993-2600638

[0057] In non-limiting embodiments, a contact may lie on the parameterized object boundary dB according to c e [0.L]. A grasp g consists of d non-coincidental contacts. Contact regions cannot overlap, resulting in a grasp configuration space Cdwritten:[0, L]do,L]Cd' Sym(d)Starting with the d-fold Cartesian product of the contact manifold [0, L] and itself, the fat diagonal A is removed, and the quotient space by the symmetric group "Sym"(d) is taken to remove permutations in contact ordering. This is the space of unordered and unlabeled configurations of d contacts. Geometrically, Cdcan be represented with an order simplex by symmetrically dissecting the box (hypercube), reducing the volume by a factor of dl. An order simplex for d-contacts may be written as:< < < < <

[0058] In non-limiting embodiments, the independent contact regions may be assumed to be disjoint continuous regions on the object boundary dB such that any grasp g (e.g., two or more contact points) with contacts in the same region may be classified as a valid grasp for interaction. Within the grasp configuration space, Cdmay be an independent contact region identified as an axis aligned box that contains only valid contact points (e.g., grasps).

[0059] In non-limiting embodiments, once a set of valid and a set of invalid contact points (e.g., grasps) have been identified within Cd, these sets may be further analyzed. For the invalid grasp set V, the Delaunay Triangulation DT = DELAUNAYTRIANGULATION(7) may be considered. The Delaunay property guarantees an empty circumcircle (circumsphere) for every simplex, a e DT, in the triangulation. In non-limiting embodiments, the algorithm searches within the largest empty circumcircle until an invalid grasp is found then updates the triangulation, repeating the process until there can be no further independent contact regions remaining that are larger than the largest already found within some bound. As used herein, the term “circumcircle” may refer to a circle, sphere, or hypersphere that passes through all vertices of a triangle (simplex).

[0060] Referring to FIG. 5, an example of Delaunay Triangulation and enclosing circumcircles is shown according to a non-limiting embodiment. Delaunay Triangulation partitions the order simplex into smaller simplices 500, each having an enclosing circumcircle (e.g., 502, 504) which has either already been explored or is6B54195.DOCX Page 12 of 22Attorney Docket No. 08993-2600638empty. Enclosing circumcircles may be used to bound the largest possible empty axis aligned boxes, and hence the largest possible independent contract regions. As an example, for any small and positive er, bounds for a box (hypercube) placed at the center S of an empty circumsphere can be expressed as:(r~ — er) < rb< r+or alternatively:e = Vd — 1r~ * (1 + e) = r+wherefcis the radius of the largest possible empty axis aligned box, expressed as its half-edge length, and r+is the radius of the largest empty circumsphere.

[0061] In non-limiting embodiments, each simplex in the Delaunay Triangulation may be provided with a candidate score. When the circumcenter is in the interior of the order simplex (e.g., circumcircles 502 and 504), the score may be the circumradius. Circumcenters outside the order simplex may undergo further processing. As an example, for a given hypersphere C* with center S, the smallest hypercube C~ centered at S is the inscribed hypercube with half-edge length r and the largest hypercube C+centered at S is the circumscribed hypercube with half-edge length r.

[0062] Referring now to FIG. 2, a flow diagram for a method for interacting with an object is shown according to a non-limiting embodiment. It will be appreciated that the steps shown in FIG. 2 are for exemplary purposes only and that fewer, additional, and / or different steps, and / or a different order of steps, may be used in non-limiting embodiments. The steps may be performed by a computing device (e.g., computing device 100 in FIG. 1). In non-limiting embodiments, a step may be automatically performed in response to performance of a previous step. The method shown in FIG.2 starts at step 200 in which a boundary of an object is determined. The boundary may be determined based on sensor data (e.g., such as an image from a camera) relating to the object, as an example, and / or based on predetermined data.

[0063] At step 202, a configuration space of contact points may be determined based on the boundary. The configuration space of contacts may include, for example, an nth dimensional space including all possible empty points within the boundary6B54195.DOCX Page 13 of 22Attorney Docket No. 08993-2600638based on the number of desired contact regions. For example, to identify two contact regions, the configuration space may be two dimensions. To identify three contact regions, the configuration space may be three dimensions. For six contact regions, the configuration space may be six dimensions.

[0064] At step 204, a plurality of segments may be determined. The plurality of segments may include simplices (e.g., triangles) including all possible contact regions in the configuration space. As used herein, the term “triangle” may refer to a simplex of any dimensions, such as a 2D triangle, a 3D tetrahedron, and the like. In non-limiting embodiments, the vertices of the triangles may be formed from invalid points in the configuration space. At step 206, the largest empty circle (e.g., circumcircle) may be identified within the triangulated segments. For example, in non-limiting embodiments, for each iteration, the simplex with the highest candidate score is found (e.g., the simplex having the largest empty circumcircle). A circumcircle may be identified as the largest circle within a triangle (simplex).

[0065] At step 208, in response to identifying the simplex with the highest candidate score, a square (e.g., hypercube in multi-dimensional space) may be computed and / or estimated to its maximum size that fits in the center of the largest empty circle and / or satisfies other constraints. For example, a square may be placed at the center and enlarged (e.g., grown) from the center of the largest empty circumcircle identified at step 206 until an invalid contact point is encountered at step 210 (e.g., steps 208 and 210 may repeat until step 210 results in an invalid finding). At step 212, after hypercube computation, the optimal contact regions in the configuration space may be updated (e.g., a list of different sets of contact regions may be updated) and the invalid contact points may be inserted into the Delaunay Triangulation at step 214. The invalid contact points may form new vertices in the configuration space, increasing the number of triangles (simplices). Updating the contact regions may include adding the identified set of contact regions to a ranked list of identified sets of contact regions. The sets of contact regions identified in each iteration may be continually ranked based on the candidate score to identify the most optimal set(s) of contact regions.

[0066] At step 216, it may be determined whether the algorithm to identify contact regions should terminate or continue to process additional iterations by looping back to step 206. In non-limiting embodiments, the process may continue and proceed to step 206 if the configuration space includes additional triangles (simplices) in which a circumcircle can be drawn that is larger than the largest empty circumcircle previously6B54195.DOCX Page 14 of 22Attorney Docket No. 08993-2600638found. In circumstances where the algorithm is unable to generate a larger circumcircle, the process may end and proceed to step 218.

[0067] In non-limiting embodiments, several different options may be pursued at step 216. For example, in a first case, the radius of the best candidate (C*. radius) may be less than or equal to the radius of the largest empty axis aligned box (hypercube) found (I*. radius). In such a case, the algorithm will not find a better set of independent contact regions by proceeding than what has already been found, so the algorithm terminates at step 216 by proceeding to step 218. In a second example, an invalid grasp G may be found which is already contained in the point set of Delaunay Triangulation at step 214. In such an example, the candidate C* may be marked as explored and the method may proceed to the next iteration at step 216 by continuing back to step 206. In another example, an invalid grasp G may be found at step 214 that is not contained in the point set of Delaunay Triangulation. This grasp may be added to the point set of Delaunay Triangulation and the point set may be updated. C* may then be marked as explored, the next best candidate may be identified, and the algorithm may proceed to the next iteration by continuing to step 206. Once the contact regions are identified, the method may proceed to step 218 to control the robotic device to interact with the object (e.g., grasping the object with a gripping mechanism or the like) based on the contact regions.

[0068] Referring to FIG. 6, shown is a diagram of an iterative algorithm for identifying contact regions for interaction with a robotic device according to a nonlimiting embodiment. A single iteration 602 of the algorithm includes identifying the best next candidate (e.g., the largest current empty region) and attempting to compute (e.g., grow) a hypercube within that region. Once an invalid contact point 606 is found, the triangulation may be updated, and the next iteration may begin. The vertices 604 in FIG. 6 represent invalid contact points. In the triangulation of candidate regions 608, some triangles (simplices) are shown darker (e.g., triangles 610) to represent higher candidate scores. The circumcircle 612 is of the highest scoring candidate region. The darker regions in the search space 608 are valid contact regions already discovered by the algorithm. The lighter regions (e.g., triangles 616) are other valid contact regions not yet discovered for visualization purposes. The box (hypercube) 618 is the largest box that could be computed (e.g., grown) at the current candidate center 612. The point 620 is the first invalid contact point found during hypercube computation, which stops the hypercube computation. FIG. 6 also shows example6B54195.DOCX Page 15 of 22Attorney Docket No. 08993-2600638snapshots from different iteration examples (e.g., Iteration 1 , 7, 13, 19, 25, 31 , 37, and 43) with increasing triangulation for purposes of illustration. Example program code (e.g., pseudocode) for an algorithm for identifying contact regions according to nonlimiting embodiments is shown in FIG. 7.

[0069] Although non-limiting embodiments described herein relate to identifying independent contact regions for interacting with a robotic device, it will be appreciated that the iterative algorithm and process described herein may be applied to other uses and domains. For example, for any multi-dimensional optimization problem in which the candidate space can be represented, the techniques described herein may be applied to identify optimal candidates.

[0070] Referring now to FIG. 8, shown is a diagram of example components of a computing device 900 for implementing and performing the systems and methods described herein according to non-limiting embodiments. The computing device 900 may represent, for example, the computing device 100 from FIG. 1. In some nonlimiting embodiments, device 900 may include additional components, fewer components, different components, or differently arranged components than those shown in FIG. 8. Device 900 may include a bus 902, a processor 904, memory 906, a storage component 908, an input component 910, an output component 912, and a communication interface 914. Bus 902 may include a component that permits communication among the components of device 900. In some non-limiting embodiments, processor 904 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 904 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), virtual or augmented reality depicting systems and devices, etc.) that can be programmed to perform a function. Memory 906 may include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 904.

[0071] With continued reference to FIG. 8, storage component 908 may store information and / or software related to the operation and use of device 900. For example, storage component 908 may include a hard disk (e.g., a magnetic disk, an6B54195.DOCX Page 16 of 22Attorney Docket No. 08993-2600638optical disk, a magneto-optic disk, a solid-state disk, etc.) and / or another type of computer-readable medium. Input component 910 may include a component that permits device 900 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally, or alternatively, input component 910 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output component 912 may include a component that provides output information from device 900 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.). Communication interface 914 may include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables device 900 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 914 may permit device 900 to receive information from another device and / or provide information to another device. For example, communication interface 914 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0072] Device 900 may perform one or more processes described herein. Device 900 may perform these processes based on processor 904 executing software instructions stored by a computer-readable medium, such as memory 906 and / or storage component 908. A computer-readable medium may include any non-transitory memory device. A memory device includes memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices. Software instructions may be read into memory 906 and / or storage component 908 from another computer-readable medium or from another device via communication interface 914. When executed, software instructions stored in memory 906 and / or storage component 908 may cause processor 904 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software. The term “programmed or configured,” as used herein, refers to an arrangement of software, hardware circuitry, or any combination thereof on one or more devices.6B54195.DOCX Page 17 of 22Attorney Docket No. 08993-2600638

[0073] Although embodiments have been described in detail for the purpose of illustration, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed embodiments, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment can be combined with one or more features of any other embodiment.6B54195.DOCX Page 18 of 22

Claims

Attorney Docket No. 08993-2600638THE INVENTION CLAIMED IS1. A method for identifying contact regions on an object for interaction by a robotic device, comprising:determining, with at least one computing device, a configuration space within the boundary of an object;determining, with at least one computing device, a plurality of segments from partitioning the configuration space;determining, with at least one computing device, a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; andcontrolling a robotic device to interact with the object based on the set of contact regions.

2. The method of claim 1 , wherein the boundary of the object is mapped to a mathematical representation according to the dimensionality of the object.

3. The method of claim 2, wherein a n-dimensional object is mapped to a (n-l)-dimensional interval.

4. The method of claim 1 , further comprising:determining the configuration space, wherein the configuration space is represented by a d-fold cartesian product of the boundary of the object.

5. The method of claim 4, further comprising creating an order simplex by applying an ordering constraint to the configuration space.

6. The method of claim 1 , wherein each set of contact regions in the configuration space is evaluated as valid or invalid.

7. The method of claim 6, wherein disjoint and continuous valid regions in the configuration space form the set of contact regions.6B54195.DOCX Page 19 of 22Attorney Docket No. 08993-26006388. The method of claim 1 , wherein the configuration space is partitioned with the Delaunay Triangulation method.

9. The method of claim 8, wherein a circle encloses all simplex partitions created from the Delaunay Triangulation method.

10. The method of claim 9, further comprising:for each iteration, computing a box that fits within the circle until an invalid point is found, wherein the set of contact points is based on a valid box from multiple iterations.

11. A system for identifying contact regions on an object for interaction by a robotic device, comprising at least one computing device configured to:determine a configuration space within the boundary of an object; determine a plurality of segments from partitioning the configuration space; determine a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and control a robotic device to interact with the object based on the set of contact regions.

12. The system of claim 11 , wherein the boundary of the object is mapped to a mathematical representation according to the dimensionality of the object.

13. The system of claim 12, wherein a n-dimensional object is mapped to a (n-l)-dimensional interval.

14. The system of claim 11 , wherein the at least one computing device is further configured to:determine the configuration space, wherein the configuration space is represented by a d-fold cartesian product of the boundary of the object.

15. The system of claim 14, wherein the at least one computing device is further configured to:6B54195.DOCX Page 20 of 22Attorney Docket No. 08993-2600638create an order simplex by applying an ordering constraint to the configuration space.

16. The system of claim 11 , wherein each contact region in the configuration space is evaluated as valid or invalid.

17. The system of claim 16, wherein disjoint and continuous valid regions in the configuration space form the set of contact regions.

18. The system of claim 11 , wherein the configuration space is partitioned with the Delaunay Triangulation method.

19. The system of claim 18, wherein a circle encloses all simplex partitions created from the Delaunay Triangulation method.

20. The system of claim 19, wherein the at least one computing device is further configured to:for each iteration, computing a box that fits within the circle until an invalid point is found, wherein the set of contact points is based on a valid box from multiple iterations.

21. A computer program product for identifying contact regions on an object for interaction by a robotic device, comprising at least one non-transitory computer-readable medium including instructions that, when executed by at least one computing device, cause the at least one computing device to:determine a configuration space within the boundary of an object; determine a plurality of segments from partitioning the configuration space; determine a set of contact regions by iteratively processing the plurality of segments based on identifying invalid points in the configuration space; and control a robotic device to interact with the object based on the at least one contact region.6B54195.DOCX Page 21 of 22

Images