Systems and methods for robust in-hand robot manipulation

Abstract

A controller is provided for manipulating an object with at least one external contact using a gripper with a robotic arm having an actuator. The controller includes a signal interface configured to receive contact signals from the gripper and send control signals to the actuator, a memory configured to store a computer-implemented program including a gripper internal force model and a robust tuning framework, and a processor configured to execute instructions of the computer program. The instructions include the steps of calculating an object hand internal slip of the object relative to the gripper based on the contact signals, calculating a naive motion cone that maintains a desired contact pattern assuming contact parameters between the object and the gripper, refining the naive motion cone based on uncertainty ranges of the respective contact parameters, generating a manipulator position trajectory that minimizes the internal slip of the gripper according to the refined motion cone, and controlling the gripper according to the generated manipulator position trajectory by sending gripper trajectory position signals to the actuator of the robotic arm to re-orient the object in the gripper with the at least one external contact.

Description

Technical Field

[0001] The present invention relates to a controller and method for robust, contact-rich in-hand manipulation, and more particularly, to a method and apparatus for manipulating an object having at least one external contact using a gripper of a robotic arm with actuators. Background Technology

[0002] Humans are highly adept at dexterity. We can skillfully utilize various forms of contact (e.g., contact with the environment, our own bodies, etc.) to perform complex manipulations. In stark contrast, enabling robots to achieve this dexterity remains extremely challenging. Effectively utilizing environmental contact can provide robots with additional dexterity when performing complex manipulations. However, most current-generation robotic systems avoid contact with their environment. Contact interaction leads to discontinuous dynamics, making the planning, estimation, and control of contact-rich tasks challenging. Furthermore, uncertainties in relevant parameters such as the coefficient of friction or the grasping position of an object often cause complex manipulation actions to fail.

[0003] In the real world, uncertainty is the norm, not the exception. Uncertainty has multiple causes. Sensing and actuation errors of components are common in the real world. Despite significant advancements in perception and learning algorithms, these errors remain highly relevant to failures during execution. Therefore, understanding the mechanics of the relevant manipulation tasks and designing robust controllers to various parameter uncertainties are crucial for designing robotic systems that can be deployed in the real world and employ realistic sensor feedback.

[0004] This disclosure proposes a method for performing complex in-hand manipulation of objects using external contact. In these methods, a robot grasps an object using a simple parallel gripper and manipulates the object's orientation. The aim of the manipulation task is to reorient the object while maintaining the grasp. Since the gripper is a simple parallel gripper, the robot performs manipulation using contact between the object and the environment. The proposed manipulation involves multiple contact configurations, making the execution of the manipulation plan susceptible to various uncertainties. However, existing manipulation techniques cannot incorporate uncertainties during the planning and control of these complex manipulation actions, and therefore fail under unavoidable uncertainties. Therefore, there is a need for techniques capable of planning and implementing robust manipulation behaviors under various uncertainties. Summary of the Invention

[0005] In recent years, artificial intelligence (AI) technology has made tremendous progress in computer vision and speech. However, robotics has not improved at the same pace, and designing robotic systems that can operate reliably and perform robustly in the real world remains challenging.

[0006] Some embodiments of this disclosure are based on the understanding that many complex manipulation tasks generate multiple contact configurations. For example, consider a task where a robot equipped with a simple parallel gripper has grasped an object. The purpose of the manipulation task is to change the grasp of the object into a posture that cannot be directly obtained in free space; that is, the robot cannot change the grasping posture simply by re-grasping the object.

[0007] Some embodiments of this disclosure are based on the understanding that, for most objects, manipulating an object grasped by a simple parallel gripper is an underactuated task. A robot can manipulate the object's orientation while it is being grasped by utilizing contact with the environment (e.g., at least one external contact). However, manipulating the object's orientation using environmental contact (e.g., at least one external contact) introduces an additional contact configuration between the object and the environment. This contact configuration can be highly susceptible to uncertainties in various contact-related parameters.

[0008] Some embodiments of this disclosure are based on the understanding that perception errors can lead to various parameter errors in manipulation, which may cause the controller to fail during the execution of various complex manipulation actions. It is important to consider these types of uncertainties when designing controllers for the execution of various complex actions. These parameter uncertainties can cause undesirable perturbations to the object relative to the planned trajectory, resulting in failures during execution. Therefore, it is desirable for the planning algorithm to be robust to various uncertainties such as the grasping center and external contact position.

[0009] Some embodiments of this disclosure are based on the understanding that maintaining external contact patterns is crucial for the success of contact-rich in-hand manipulation tasks. This disclosure proposes a system and method for designing a manipulation controller that fully considers object in-hand slippery and parametric uncertainties. We focus on two-finger gripper grasping and the case where only translational motion is allowed. The proposed robust manipulation framework also identifies gripper motions that minimize object in-hand translation to prevent the robot from losing its grip on the object. The proposed method focuses on two-finger gripper grasping and the case where only translational motion is allowed. The proposed robust manipulation framework also identifies gripper motions that minimize object in-hand translation to prevent the robot from losing its grip on the object.

[0010] Some embodiments of this disclosure are based on the understanding that in-hand manipulation tasks involving external contact typically present two main challenges. In response to receiving manipulation task commands (task commands or task signals), contact parameters, and gripper motion via a signal interface (hardware interface), the system must predict object motion in both the world coordinate system and the gripper coordinate system. This is not easy, as the object simultaneously undergoes contact-rich motion in the environment and slides within the gripper. The generated gripper trajectory must prevent unexpected contact mode transitions under parameter uncertainties, particularly uncertainties in kinematic parameters such as contact position and object size.

[0011] The method proposed in this disclosure addresses the two challenges mentioned above and provides a robust controller design scheme for robust in-hand manipulation with external contact. This disclosure proposes a robust planning method for in-hand manipulation tasks while maintaining external contact. Given the desired contact pattern and the uncertainty range of each parameter, the method first calculates the motion cone for each combination of vertex parameters within the uncertainty range using an in-gripper mechanical model, and then takes the intersection of these motion cones to obtain the robust motion cone. The controller obtained by the proposed method can be implemented on physical robotic systems with available onboard sensing devices (such as vision, tactile, field-to-ground, etc.).

[0012] According to some embodiments of this disclosure, a controller is provided for manipulating an object having at least one external contact using a gripper of a robotic arm with an actuator, comprising: a signal interface configured to receive task commands and contact signals from the manipulator and to send control signals to the actuator; a memory configured to store a computer-implemented program including a mechanical model of the gripper, a parametric model of the manipulation task, and a robust tuning framework; and a processor configured to execute instructions of the computer program associated with the memory, wherein the instructions include the step of using the gripper... The in-machine mechanical model calculates the in-hand slippage of the object in the gripper; assuming contact parameters between the object and the gripper, a motion cone is calculated to maintain the desired contact pattern; the motion cone is refined based on the uncertainty range of each contact parameter; a manipulator position trajectory that minimizes the in-hand slippage of the object is generated based on the refined motion cone; and the manipulator is controlled according to the generated manipulator position trajectory by sending the manipulator position trajectory signal to the actuator of the robot arm, thereby reorienting the object in the gripper using the at least one external contact.

[0013] Furthermore, some embodiments of this disclosure provide a non-transitory computer-readable medium for manipulating an object having at least one external contact using a manipulator and gripper of a robotic arm with actuators. The non-transitory computer-readable medium includes instructions stored thereon that, when executed on a processor, perform the following steps: receiving a task command and a contact signal from the manipulator using a signal interface and sending a control signal to the actuator; calculating the in-hand slippage of the object in the gripper using a mechanical model within the gripper; calculating a motion cone that maintains a desired contact pattern, assuming contact parameters between the object and the gripper; refining the motion cone based on the uncertainty range of each of the contact parameters; generating a manipulator position trajectory that minimizes the in-hand slippage of the object based on the refined motion cone; and controlling the manipulator according to the generated manipulator position trajectory by sending a manipulator position trajectory signal of the manipulator position trajectory to the actuator of the robotic arm, thereby reorienting the object in the gripper using the at least one external contact.

[0014] The present disclosure is further described below by way of non-limiting example of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein the same reference numerals denote similar parts in the various views of the drawings. The drawings shown are not necessarily drawn to scale, but generally focus on illustrating the principles of the embodiments currently disclosed.

[0015] Although the following figures illustrate the embodiments currently disclosed, other embodiments are also contemplated as noted in the discussion. Exemplary embodiments are given by way of illustration and not limitation. Many other modifications and embodiments falling within the scope and spirit of the principles of the embodiments currently disclosed will be conceived by those skilled in the art. Attached Figure Description

[0016] [ Figure 1 ]

[0017] Figure 1 The illustration schematically demonstrates typical failures of in-hand manipulation due to loss of external contact during manipulation, while simultaneously showcasing the success of the proposed robust manipulation controller.

[0018] [ Figure 2 ]

[0019] Figure 2 The illustration schematically depicts in-hand gesture manipulation using external contact in a complete three-dimensional setup, as well as the two-dimensional simplification employed in this disclosure.

[0020] [ Figure 3 ]

[0021] Figure 3The manual manipulation task, along with the associated model and control parameters considered in this disclosure, are shown.

[0022] [ Figure 4A ]

[0023] Figure 4A The limit surface is shown for the use of the manipulation model considered in this disclosure for a feasible manipulation task.

[0024] [ Figure 4B ]

[0025] Figure 4B The feasible range of motion for in-hand manipulation tasks using the manipulation model employed in this disclosure is shown.

[0026] [ Figure 5A ]

[0027] Figure 5A The limit surface is shown for the manipulation model considered in this disclosure for an infeasible manipulation task.

[0028] [ Figure 5B ]

[0029] Figure 5B The feasible range of motion is shown using the manipulation model employed in this disclosure for tasks that are otherwise impossible to perform in-hand manipulation.

[0030] [ Figure 6 ]

[0031] Figure 6 The proposed robust manipulation framework for in-hand gesture manipulation described in this disclosure is shown.

[0032] [ Figure 7 ]

[0033] Figure 7 An example of a robust gripper motion trajectory calculated using the method described in this disclosure is shown.

[0034] [ Figure 8 ]

[0035] Figure 8 A robot system designed according to this disclosure is shown, which has different sensing modes that can be used in the system.

[0036] [ Figure 9 ]

[0037] Figure 9 Examples of manipulation tasks according to some embodiments of the present disclosure are shown, in which a robot performs a hand-held manipulation task of a bottle by contacting its external environment.

[0038] [ Figure 10 ]

[0039] Figure 10 An example manipulation task according to some embodiments of this disclosure is shown, in which a robot must create a structure with an object in an undesirable initial posture.

[0040] [ Figure 11A ]

[0041] Figure 11A This illustrates a hand manipulation task using a single point of contact according to an embodiment of the present disclosure.

[0042] [ Figure 11B ]

[0043] Figure 11B The measurement results of the sliding distance of the contact point during operation are shown according to an embodiment of the present disclosure. Detailed Implementation

[0044] In the following description, numerous specific details are set forth for illustrative purposes in order to provide a full understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without these specific details. In other instances, apparatus and methods are shown only in block diagram form to avoid obscuring this disclosure.

[0045] When used in this specification and claims, the terms “for example,” “for instance,” and “such as,” as well as the verbs “comprising,” “having,” “including,” and other verb forms thereof, when used in conjunction with an enumeration of one or more components or other items, shall be construed as an open-ended expression, meaning that the enumeration shall not be considered as excluding other additional components or items. The term “based on” means at least partially based on. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. Any headings used in this specification are for convenience only and have no legal effect or limitation.

[0046] Some embodiments of this disclosure are based on the understanding that contact is central to most manipulation tasks. The desired goal is to endow robots with human-like manipulation capabilities. Humans can utilize their fingers (e.g., two fingers), body, and environment to support the mechanical processes of manipulation and react to unpredictable disturbances, thereby performing highly complex manipulations. These tasks far exceed traditional pick-and-place tasks in terms of the complexity of planning and control. Designing controllers that enable robots to achieve human-like physical intelligence has long been one of the most challenging problems in robotics and artificial intelligence.

[0047] However, robots generally struggle to effectively utilize contact, resulting in poor maneuverability. These contacts during manipulation can occur between the object and the robot, or between the object and its environment as the robot attempts to manipulate it. Typically, these tasks are underactuated, meaning certain contact configurations cannot be directly controlled by the robot. This makes manipulation tasks highly prone to failure.

[0048] Some embodiments of this disclosure are based on the understanding that one of the main reasons for poor performance of manipulation controllers when dealing with novel objects (i.e., objects for which the robot has no prior model) is the uncertainty of kinematic and physical parameters. Most existing vision techniques used to estimate the position or orientation of an object may not provide sufficient accuracy to perform the task accurately. However, through testing and validation, it may be possible to estimate the uncertainty of the estimates of different parameters relevant to a particular manipulation task.

[0049] Figure 3 The manual manipulation tasks, along with related models and control parameters, are shown as some embodiments of this disclosure. Figure 3 An example manipulation task 301 is shown as one implementation. The manipulation task (manipulation task command) has several sets of parameters 302 that need to be considered in the planning of the task. In some implementations, the state of the manipulated task can be represented by the object's orientation (object angle) and the desired object angle 302a. Similarly, there are many modeling parameters 302b that may affect the manipulation task, such as gripping position, gripping force, external contact position, object mass, center of mass position, and coefficients of friction between surfaces 302b. Robot control actions can be movements within the gripper and the forces 302c that it can apply to perform the desired manipulation.

[0050] Some embodiments of this disclosure are based on the understanding that manipulation planning frameworks can utilize uncertainty estimates of relevant parameters to design and compute controllers robust to these uncertainties. In real-world manipulation, these parameters need to be estimated for new objects, and these parameters may be uncertain; these controllers can provide robustness against parameter uncertainty. Considering these uncertainties is particularly important when manipulating small parts that are typically difficult to observe using general-purpose vision systems.

[0051] Figure 1 An example scenario is shown where the robot attempts to manipulate the orientation of pin 113, which must be inserted to perform assembly task 103. The current gripping of pin 113 is not suitable for downstream assembly task 102.

[0052] Some embodiments of this disclosure are based on the understanding that robots with simple dexterity have limited maneuverability because the gripper may not be fully actuated or have a very limited range of motion. However, in some cases, the robot can utilize the environment to compensate for limited dexterity and perform maneuvers. Because the robot is equipped with a simple gripper 111, which has single-degree-of-freedom parallel fingers 112. The parallel fingers 112 can constitute a two-finger gripper, or the number of fingers in the gripper can be two or more. In some cases, the number of fingers can be two or more. This makes the task of reorienting the pin difficult, and the robot must utilize the external contact 110 between the pin 113 and the environment to reorient the pin.

[0053] Figure 1 The diagram schematically illustrates a typical failure of in-hand manipulation due to loss of external contact during manipulation, while simultaneously demonstrating the success of the proposed robust manipulation controller. One scenario is shown where a controller that does not account for uncertainties in the pin's gripping position eventually loses contact with the external environment 104, thus failing to redirect the pin for insertion. However, a controller designed to account for potential uncertainties in the gripping position is able to maintain contact between the pin and the environment. The robot successfully redirects the pin to 102, thereby enabling it to complete the downstream insertion task 103.

[0054] Some embodiments of this disclosure are based on the understanding that it is crucial to establish and create reliable models for contact-rich manipulation techniques. This is because the mechanism of the manipulation task can provide information about various relevant parameters and help identify the parameters most likely to cause the manipulation task to fail.

[0055] Some embodiments of this disclosure are based on the understanding that, in order to obtain a robust controller for in-hand manipulation, an accurate parametric model of the object's motion within the gripper is needed, and this parametric model can be robustly refined using uncertainty estimation of the parameters. The complete model can be very complex, requiring a very detailed analysis of the friction cone of the manipulation task. Therefore, we consider a simplified two-dimensional model of the task, which can be easily analyzed without including predictive capabilities.

[0056] Figure 2 An example scenario of an in-hand manipulation task in the full three-dimensional scenario 201 and its two-dimensional simplification 211 are shown. In the original configuration, the object 203 grasped by the finger 202 forms an external line contact 204 with the environment surface 205. In the two-dimensional simplification 211, the full line contact 204 is replaced with a point contact 213. The grasping contact surface is simplified to a contact surface 212.

[0057] Some embodiments of this disclosure are based on the understanding that, since we are concerned with parallel gripper holders, the system can be approximated as a two-dimensional contact model (mechanical model within the gripper), and the gripper forms a surface contact with the object. Assuming the object maintains N contact points with the environment, the contact positions and contact normals of these contact points in the gripper coordinate system are denoted as follows: and Boolean variables Indicates whether the object rotates counterclockwise around the point of contact. Integer. This indicates that contact point i is sliding to the left, sticking, or sliding to the right. Disconnected contact points are not considered contact points to be maintained during manipulation. We use a common quantitative representation of the contact mode. We define the generalized velocity of objects in the world coordinate system. Decompose it into its motion in the gripper coordinate system and the motion of the gripper in the world coordinate system Because the robotic hand has a fixed orientation, therefore... The mechanical model of the gripper aims to obtain... Feasible motion cone To maintain the desired contact pattern and predict and .

[0058] Some embodiments of this disclosure are based on the understanding that, since the gripper forms a sheet contact with the object, a second-order limit surface (LS) is used as the load-motion mapping for slippage within the gripper, wherein the available frictional spinor applied from the object to the gripper... It is defined by an ellipsoid.

[0059] (1)

[0060] in, and Let be the coefficient of friction and the gripping force between the object and the gripper fingers, respectively. κ is the integration constant. For a uniformly distributed circular contact surface, κ≈0.6r, where r is the radius of the contact surface.

[0061] Some embodiments of this disclosure are based on the understanding that in-hand slippage only occurs under the equality constraint of equation (1). Since our goal is to adjust in-hand posture, it can be assumed that the equality constraint of equation (1) is always activated throughout the motion trajectory. According to the maximum dissipation principle, in-hand motion... It should be perpendicular to the limiting surface LS at the point of force spinor.

[0062] (2)

[0063] α≥0 is a proportionality constant. Given the desired contact pattern, and The possible ranges are respectively called the Force Spin Motion Set (WMS) and the Environmental Motion Set (EMS).

[0064] Some embodiments of this disclosure are based on the understanding that load-motion mapping (2) provides a range of possible... The set of forces spinors (WS) serves as a bridge for deriving the WMS. Each contact point contributes one or two force spinor rays to the moving cone, and the non-negative linear combination of these force spinor rays, along with the gravitational force spinor, constitutes the force spinor set.

[0065] Some embodiments of this disclosure are based on the understanding that, due to the linear complementarity condition of the Coulomb friction model in two-dimensional space, there can only be one or two force spinor rays in total, including all contact points. If there is only one force spinor ray, the contact mode has two degrees of freedom (DoF); if there are two force spinor rays, the contact mode has only one degree of freedom. This paper mainly discusses the case with two force spinor rays; the case with only one force spinor ray can be considered a degenerate special case. The gravitational force spinor in the gripper coordinate system is denoted as... And the force spinor ray provided by the contact point in the clamp coordinate system is denoted as and Then the set of force spinors can be mathematically represented as: (3) Since we assume that the friction of the clamp can lift the pin, therefore It must fall within LS. Therefore, WS and LS must have a non-empty intersection. Since LS is ellipsoidal, the intersection of WS and LS in force spinor space will be a three-dimensional elliptical arc, which can be parametrically represented as follows: (4) Combining equation (2), WMS can be expressed as: (5) The above division is an element-wise division. For convenience, the right side of equation (5) is denoted as... .

[0066] Figure 4A The limiting surface is shown for a feasible manipulation task using the manipulation model considered in this disclosure. The figure illustrates an example of visualizing the WS and WMS. The robot needs to rotate a pin clockwise around the line contact while maintaining contact line adhesion, while only performing translational motion. Assume the coefficient of friction of the contact plane is 0.2. The center of mass is located at... , And the grab point is located at , The friction coefficient of the gripper is 0.4, and the gripping force is 20 N. The gripping is approximated as a contact of a uniform circular surface with a radius of 0.01 m. The ellipsoid 401 is the limit surface (LS) in equation (1). The two rays 402 and 404 and the force spin they span are respectively the friction force spin. , The force spinor set (WS) in equation (3). The intersection between WS and LS is in Figure 4A The solid line represents arc 403. Figure 4B This illustrates the feasible range of motion obtained using the manipulation model employed in this disclosure for in-hand manipulation tasks. The force-spinometer motion set (force-spinometer motion space) (WMS) is obtained by taking the normal ray at each point of the solid-line arc, as shown... Figure 4B The curved surface 412 is shown in the figure. To derive the motion cone... Furthermore, EMS needs to be calculated, which will be explained below.

[0067] Some embodiments of this disclosure are based on the understanding that calculating EMS from contact kinematic constraints is relatively straightforward. These constraint equations depend on the contact mode representation. .For example, Figure 1 The adhesive line contact in the middle has the following contact modes:

[0068] It has only one degree of freedom, therefore EMS becomes a single ray, which is composed of... Figure 4B Line 415 is shown in the diagram.

[0069]

[0070] In some cases, it can have two degrees of freedom, in which case the EMS will become a linear cone stretched by two moving rays.

[0071] Some embodiments of this disclosure are based on the understanding that: the feasible motion cone for maintaining the desired contact mode is the Minkowski and... The Minkowski sum is a cone spanned by two planes 414 and 417 and a surface bounded by 412. 414 represents the spinor of the gravitational force. Since the gripper only allows translational motion, the intersection of this Minkowski sum and the ω=0 plane is taken, thus forming the moving cone. The moving cone is composed of Figure 4B The surface 416 in the image is visualized. We will... Quantitatively described as angular range Given the movement of the gripper in the motion cone. It can be uniquely decomposed into motion in WMS and motion in EMS, thereby predicting the motion of the object within the gripper.

[0072] (6)

[0073] Some embodiments of this disclosure are based on the understanding that there are still cases where the WMS and EMS do not intersect the ω=0 plane. This occurs when the robot cannot perform manipulation while maintaining the desired contact pattern using only translational motion. Figure 5A and Figure 5B An example of this situation is given, where =90° and μ=0.1. Since μ is small, every translational motion will cause the contact point to slide, therefore .

[0074] Figure 5A The LS 501 in this case is shown. Figure 5B The feasible range of motion obtained by using the manipulation model in this disclosure for an infeasible in-hand manipulation task is shown. Figure 5B The 513 in the equation represents the gravitational spinor. For example... Figure 5B As shown, the intersection of surfaces 511, 512, 514, and 515 is empty, indicating that the desired manipulation task is infeasible. This geometrically represents... The intersection with the plane of ω=0 is an empty set, thus indicating that... .

[0075] Some embodiments of this disclosure are based on the understanding that the motion cone calculated by equations (1) to (5) can accurately estimate each contact parameter. The motion cone can be refined to be robust to parameter uncertainties. Using ξ∈R n This represents the set of parameters whose uncertainties need to be considered. For (Uncertainty parameter) Each component Let its uncertainty range be And let Δξ denote the set of Δξ_i. Each different combination of parameters in the n-dimensional cuboid ξ∈ξ±Δξ produces a different motion cone. By definition, a robust motion cone Should be by all possible corresponding The intersection is determined. When Compared to When the size is small, with other parameters fixed, the moving cone With each individual parameter It approximately changes monotonicly within the range of uncertainty. Therefore, it can be... Approximately composed of an n-dimensional cuboid The intersection of the moving cones generated by each vertex.

[0076] (7)

[0077] Using the calculated robust motion cone Many search-based planning methods, such as RRT search, can be directly adopted.

[0078] Some embodiments of this disclosure are based on the understanding that: the desired object in a given EMS undergoes single-step world motion. In this case, it can be estimated that Minimize the translation part We found that, due to nonlinear constraints, the optimizer easily gets trapped in infeasible local minima. Therefore, we adopted a sampling-based method. We used intervals... exist Inner uniform sampling Then solve for α≥0, β≥0 and that satisfy the following equality constraints. : (8) For a given θ, if α, β, and θ cannot be determined... The solution obtained does not satisfy the robustness condition. The choice of θ should be skipped. The θ that minimizes α should be chosen. For forward kinematics, the obtained β and α are sufficient to predict the world motion of the object and the slippage within the gripper. The sample-based optimization for θ is repeated in the next time step until the target pose is reached.

[0079] Figure 6 The proposed robust manipulation framework for in-hand gesture manipulation described in this disclosure is shown. Figure 6 This provides an overview of robust planning methods and... =Example of a robust trajectory planning from 38° to 90°. The robot needs to pivot the pin while ensuring that the two contact points do not separate 601. The mass and center of mass positions are shown in 602. The coefficients of friction for the horizontal contact plane and the vertical direct contact plane are 0.1 and 0.12, respectively, and the gripper is in and The object is captured at its position, as shown in Figure 602. The bottom length of the object is 0.027 m. It can be seen that... yes A subset of, thus further ensuring the contact mode under parameter uncertainty. 603 shows the subset of, thus further ensuring the contact mode under parameter uncertainty. Each vertex 604 in the equation generates a motion cone. Robust motion cones are obtained by taking the intersection; wedge region 605 is the naive motion cone assuming accurate parameters, while wedge region 606 is the robust motion cone. Control actions are generated using forward motion students to minimize object in-hand slippery 608.

[0080] Figure 7 This is an example of a robust manipulator motion trajectory calculated using the method described in this disclosure. Figure 7 This demonstrates the hand manipulation task from =40 to 85 is the calculated trajectory (gripper position trajectory or manipulator position trajectory) 701. The robust motion cone is shown as 703, which is smaller than the basic motion cone 702 and begins to contract as the task progresses, indicating that no additional robustness can be provided after the grasped object has gone beyond a certain orientation.

[0081] It should be noted that the control signal output by the robust programming method can be represented as the gripper trajectory of the manipulator end effector trajectory. Since the manipulator trajectory can be easily converted into the gripper trajectory by performing a rigid transformation between the two coordinates, the two are often used interchangeably.

[0082] Next, the effectiveness of the robust programming method is verified by comparing it with naive programming. (Using again...) Figure 1 The example is used as the first test case. We consider the crawl point ( and ) and the location of the center of mass ( and The uncertainty of the gripper position is assumed to be 3 mm in both the x and y directions. A robust programming method is used to generate the gripper position trajectory (manipulator position trajectory). The baseline trajectory is generated by a naive programming method. We intentionally add ±2 mm displacements to the x and y coordinates of the gripper point respectively and compare the performance of robust programming and naive programming. The results are summarized in Table 1 (…). Figure 11B In the above calculations, robust planning ensures contact point adhesion in almost all cases, while naive planning exhibits detectable contact point slippage (slippage of the object within the hand). Robust planning also shows a slippage of 1.7 mm when both the x and y coordinates of the gripping point have a displacement of -2 mm. This is likely because the contact point falls outside the range of uncertainty in the planning at this setting. Figure 1 Snapshots of naive and robust planning are shown.

[0083] Figure 8A robot system 800 designed according to this disclosure is shown, which has different sensing modes 803 (e.g., vision 803a, tactile 803b, force sensor 803c, etc.) that can be used in the system. The robot system 800 may include a controller 820, which has at least a signal interface 821, a processor 822, and a memory 823. The controller 820 is configured to be connected to an operator terminal 835 via a network 830. The controller 820 can receive manipulation task commands including parameters via the signal interface. In this case, the parameters may include task status, model parameters, and control actions, wherein the manipulation task commands are input by an operator using the operator terminal 835 via the network 830. The network 830 may be part of a wireless communication link, a wired link, an optical fiber communication link, or a combination thereof. In response to receiving a manipulation task command via the signal interface 821, the controller 820 may use the processor 822 to begin controlling the robot arm 810, the processor 822 beginning to execute steps of computer-implemented instructions stored in the memory / storage device 823. The robotic arm 810 includes joints 811 with actuators, links, and manipulator units 812 and 813, which include grippers 814 with fingers. Circuitry may be arranged separately from or included within the robotic arm 810 and is configured to operate the actuators, manipulator units, and gripper 814 to perform tasks provided by a controller 820 operated by an operator using an operating terminal 835. The figure illustrates the gripper's fingers grasping an object 817.

[0084] Figure 9 An example of another in-hand manipulation task according to some embodiments of the present disclosure is shown, in which an object 901 is shown in a series of poses as it is manipulated by a robot (manipulator) 904. The robot 904 uses the external environment 902 to manipulate the pose of the object 901 from an initial pose 901 to a desired pose 903.

[0085] Figure 10 An example of another possible application according to some embodiments of the present disclosure is shown. The task is to create a structure 1004 on a desktop surface 1003. A robot needs to manipulate an object 1002 from an initial configuration 1002 to a desired configuration 1005. The desktop surface also has a manipulation station 1001 that the robot may potentially use during the task. The robot can use some embodiments of the present disclosure to redirect the object 1002 from its initial configuration to the desired configuration 1005 to complete the desired structure 1004.

[0086] Figure 11AAnother in-hand manipulation task according to some embodiments of this disclosure is illustrated. This task involves changing the orientation of a pin from 1101 to 1103 using an external line contact 1102. To illustrate the robustness of the proposed controller, we performed a series of experiments by varying the gripping position 1104 relative to a known ideal position 1104. We approximate the slippage produced at 1102 during the manipulation task. Figure 11B The measurement results of the sliding distance of the contact point during operation are shown according to an embodiment of the present disclosure. Figure 11B The results are shown when various noise values ​​are added to the ideal gripping position. The goal here is to minimize the slippage occurring at the external contact position 1102. Figure 11B It is evident that the proposed robust controller outperforms the benchmark controller in minimizing slippage during processing.

[0087] The embodiments described above can be implemented in any of a variety of ways. For example, these embodiments can be implemented using hardware, software, or a combination thereof. When implemented in software, the software code can execute on any suitable processor or set of processors, whether these processors are located in a single computer or distributed among multiple computers. Such a processor can be implemented as an integrated circuit, wherein one or more processors are included in one integrated circuit component. However, the processor can also be implemented using any suitable form of circuitry.

[0088] Furthermore, embodiments of this disclosure can be implemented as methods, examples of which have been given above. Actions performed as part of this method can be arranged in any suitable order. Therefore, embodiments can be constructed in which actions are performed in a sequence different from the order shown, including the simultaneous execution of certain actions even if they are shown as sequential in the exemplary embodiments.

[0089] The use of ordinal numbers such as "first" and "second" to modify claim elements in claims does not in itself indicate any priority, order of precedence or sequence of one claim element relative to another claim element, nor does it indicate the chronological order of the execution of method actions. Rather, it serves only as a marker to distinguish one claim element with a certain name from another element with the same name (if no ordinal number is used), thereby differentiating the claim elements.

[0090] Although this disclosure has been described by way of examples of preferred embodiments, it should be understood that various other adjustments and modifications can be made within the spirit and scope of the invention.

[0091] Therefore, the appended claims are intended to cover all such variations and modifications that fall within the true spirit and scope of the invention.

Claims

1. A controller for manipulating an object having at least one external contact using a gripper of a robotic arm with actuators, the controller comprising: A signal interface configured to receive task commands and contact signals from the manipulator and to send control signals to the actuator; The memory is configured to store a computer-implemented program, the program including a mechanical model of the gripper, a parametric model of the manipulation task, and a robust tuning framework. A processor configured to execute instructions of a computer program in association with the memory, wherein the instructions include the following steps: The hand slippage of the object in the gripper is calculated using the mechanical model within the gripper. Calculate the motion cone that maintains the desired contact pattern, assuming the contact parameters between the object and the gripper. The motion cone is refined based on the uncertainty range of each of the aforementioned contact parameters; Based on the refined motion cone, a manipulator position trajectory is generated that minimizes the in-hand slippage of the object; and By sending the manipulator position trajectory signal of the manipulator position trajectory to the actuator of the robot arm, the manipulator is controlled according to the generated manipulator position trajectory, thereby reorienting the object in the gripper using the at least one external contact.

2. The controller according to claim 1, wherein, The motion cone used for the movement within the gripper of an object is represented by the Minkowski representation of the force spinor motion space (WMS) and the environmental motion set (EMS).

3. The controller according to claim 1, wherein, The robust motion cone for robust hand manipulation is obtained by taking the intersection of the motion cones calculated for each of the contact parameters within the range of uncertainty.

4. The controller according to claim 1, wherein, Using an internal mechanical model of the gripper, a feasible motion cone for the grasped object is calculated while maintaining the desired contact pattern with at least one external contact with the environment.

5. The controller according to claim 1, wherein, For each combination of vertex parameters within the range of uncertainty, the motion cone is calculated based on the mechanical model within the gripper.

6. The controller according to claim 1, wherein, The robust motion cone is determined by the intersection of different motion cones.

7. The controller according to claim 1, wherein, The processor begins executing the steps in response to receiving a manipulation task command including parameters via the signal interface, wherein the parameters include task status, model parameters, and control actions, and wherein the manipulation task command is input to the signal interface by an operator using an operating terminal connected to the controller via a network.

8. The controller according to claim 1, wherein, The gripper is a two-finger gripper.

9. The controller according to claim 1, wherein, While controlling the gripper, the manipulator position trajectory remains in contact with at least one external part of the object.

10. The controller according to claim 1, wherein, The processor begins executing the instructions in response to the task command.

11. A non-transitory computer-readable medium for manipulating an object having at least one external contact using a manipulator and gripper of a robotic arm with actuators, the non-transitory computer-readable medium comprising instructions stored thereon that, when executed on a processor, perform the following steps: The signal interface is used to receive task commands and contact signals from the manipulator and to send control signals to the actuator. The hand slippage of the object in the gripper was calculated using an internal mechanical model of the gripper. Calculate the motion cone that maintains the desired contact pattern, assuming the contact parameters between the object and the gripper. The motion cone is refined based on the uncertainty range of each of the aforementioned contact parameters; Based on the refined motion cone, a manipulator position trajectory is generated that minimizes the in-hand slippage of the object; as well as By sending the manipulator position trajectory signal of the manipulator position trajectory to the actuator of the robot arm, the manipulator is controlled according to the generated manipulator position trajectory, thereby reorienting the object in the gripper using the at least one external contact.

12. The non-transitory computer-readable medium according to claim 11, wherein, The motion cone used for the movement within the gripper of an object is represented by the Minkowski representation of the force spinor motion space (WMS) and the environmental motion set (EMS).

13. The non-transitory computer-readable medium according to claim 11, wherein, The robust motion cone for robust hand manipulation is obtained by taking the intersection of the motion cones calculated for each of the contact parameters within the range of uncertainty.

14. The non-transitory computer-readable medium according to claim 11, wherein, Using an internal mechanical model of the gripper, a feasible motion cone for the grasped object is calculated while maintaining the desired contact pattern with at least one external contact with the environment.

15. The non-transitory computer-readable medium according to claim 11, wherein, For each combination of vertex parameters within the range of uncertainty, the motion cone is calculated based on the mechanical model within the gripper.

16. The non-transitory computer-readable medium according to claim 11, wherein, The robust motion cone is determined by the intersection of different motion cones.

17. The non-transitory computer-readable medium according to claim 11, wherein, The processor begins executing the steps in response to receiving a manipulation task command including parameters via the signal interface, wherein the parameters include task status, model parameters, and control actions, and wherein the manipulation task command is input to the signal interface by an operator using an operating terminal connected to the controller via a network.

18. The non-transitory computer-readable medium according to claim 11, wherein, The gripper is a two-finger gripper.

19. The non-transitory computer-readable medium according to claim 11, wherein, While controlling the gripper, the manipulator position trajectory maintains the object in at least one external contact.

20. The non-transitory computer-readable medium according to claim 11, wherein, The processor begins executing the instructions in response to the task command.

Images