Method, device and computer readable storage medium for generating a grasping pose

Abstract

Embodiments of the present application disclose a method and device for generating a grasping pose, an apparatus and a computer readable storage medium. The method comprises: obtaining a target contact point on a target object, and obtaining a template contact point corresponding to a dexterous hand; constructing a pose optimization objective function for the dexterous hand grasping the target object according to the target contact point and the template contact point; performing inverse kinematics solving on a robot arm to which the dexterous hand belongs based on the target contact point, obtaining wrist reachability results of the robot arm, and calculating a wrist disturbance rotation amount corresponding to the dexterous hand; adjusting a wrist disturbance rotation parameter in the pose optimization objective function according to the wrist disturbance rotation amount, and obtaining an adjusted pose optimization objective function; and determining a target grasping pose of the dexterous hand grasping the target object according to the adjusted pose optimization objective function. Through the method, the generation accuracy of the grasping pose when the dexterous hand grasps an object can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and specifically to a method, apparatus, device, and computer-readable storage medium for generating grasping postures. Background Technology

[0002] In the process of a robot grasping an object, it usually needs to use a dexterous hand to grasp the object. Therefore, it is necessary to generate a corresponding grasping posture for the dexterous hand so that it can complete the corresponding grasping action.

[0003] In related technologies, high-degree-of-freedom dexterous hands are typically used to perform grasping tasks, and methods such as geometric matching and deep learning are employed to generate grasping postures in a high-dimensional action space. However, this method of generating grasping postures requires a high degree of freedom from the dexterous hand. When applied to low-degree-of-freedom dexterous hands, the generated grasping postures may not be suitable for the low-degree-of-freedom dexterous hand, leading to a decrease in the accuracy of the grasping posture when the dexterous hand grasps objects.

[0004] Therefore, improving the accuracy of the grasping posture when a dexterous hand grasps an object is an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of this application provide a method, apparatus, device, and computer medium for generating grasping postures.

[0006] The technical solution adopted in this application is a method for generating a grasping posture, comprising:

[0007] Obtain the target contact point on the target object, and obtain the template contact point corresponding to the dexterous hand; Based on the target contact point and the template contact point, construct an objective function for posture optimization of the dexterous hand grasping the target object; Based on the target contact point, inverse kinematics is performed on the robotic arm to which the dexterous hand belongs to obtain the wrist reachability result of the robotic arm; if the wrist reachability result indicates that the wrist of the robotic arm cannot reach the target contact point, then the wrist perturbation rotation amount corresponding to the dexterous hand is calculated according to the principal axis of the target object and the preset perturbation angle range; based on the wrist perturbation rotation amount, the wrist perturbation rotation parameter in the posture optimization objective function is adjusted to obtain the adjusted posture optimization objective function; Based on the adjusted posture optimization objective function, the target grasping posture for the dexterous hand to grasp the target object is determined.

[0008] A device for generating a grasping posture, comprising: The acquisition unit is used to acquire the target contact point on the target object and the template contact point corresponding to the dexterous hand; The construction unit is used to construct an attitude optimization objective function for the dexterous hand grasping the target object based on the target contact point and the template contact point; An adjustment unit is used to perform inverse kinematics calculation on the robotic arm to which the dexterous hand belongs, based on the target contact point, to obtain the wrist reachability result of the robotic arm; if the wrist reachability result indicates that the wrist of the robotic arm cannot reach the target contact point, then the wrist perturbation rotation amount corresponding to the dexterous hand is calculated according to the principal axis of the target object and a preset perturbation angle range; the wrist perturbation rotation parameter in the posture optimization objective function is adjusted according to the wrist perturbation rotation amount to obtain the adjusted posture optimization objective function; The processing unit is used to determine the target grasping posture of the dexterous hand grasping the target object based on the adjusted posture optimization objective function.

[0009] In one embodiment of this application, based on the foregoing scheme, before determining the target grasping posture of the dexterous hand grasping the target object according to the adjusted posture optimization objective function, the processing unit is further configured to determine a contact matching result based on the target contact point and the template contact point; if the contact matching result indicates that a single contact point fails to cover the stable contact area of ​​the target object, then the contact point matching weight parameter is reduced to obtain an adjusted contact point matching weight parameter; the posture optimization objective function is adjusted according to the adjusted contact point matching weight parameter to obtain the adjusted posture optimization objective function.

[0010] In one embodiment of this application, based on the foregoing scheme, before determining the target grasping posture of the dexterous hand grasping the target object according to the adjusted posture optimization objective function, the acquisition unit is further configured to acquire the joint angles of the dexterous hand and the joint angle boundaries corresponding to the joint angles; determine joint angle constraint parameters according to the deviation between the joint angles and the joint angle boundaries; and adjust the joint angle constraint terms in the posture optimization objective function according to the joint angle constraint parameters to obtain the adjusted posture optimization objective function.

[0011] In one embodiment of this application, based on the foregoing scheme, the processing unit is further configured to: solve for the posture parameters of the dexterous hand in the current iteration based on the adjusted posture optimization objective function, and obtain multiple candidate grasping postures corresponding to the current iteration; simulate grasping the target object based on the multiple candidate grasping postures, and obtain simulation results corresponding to the multiple candidate grasping postures respectively; update the target contact point based on the simulation results, and reconstruct the posture optimization objective function based on the updated target contact point and the template contact point, until the iteration termination condition is reached; when the iteration termination condition is reached, determine the target grasping posture based on the multiple candidate grasping postures corresponding to the terminated iteration.

[0012] In one embodiment of this application, based on the foregoing scheme, the acquisition unit is further configured to, when the iteration termination condition is reached, acquire the termination score results corresponding to the multiple candidate grasping postures corresponding to the termination iteration round; and determine the candidate grasping posture with the highest termination score result as the target grasping posture.

[0013] In one embodiment of this application, based on the foregoing scheme, the acquisition unit is further configured to acquire an object mesh model corresponding to the target object; perform surface sampling on the object mesh model to obtain an object point cloud corresponding to the target object, the object point cloud including multiple object points; determine a stable contact region from the object point cloud according to the geometric attribute information corresponding to the object point cloud; and select a target contact point on the target object from the multiple object points in the stable contact region.

[0014] In one embodiment of this application, based on the foregoing scheme, the acquisition unit is further configured to acquire a structural model corresponding to the dexterous hand; determine, based on the structural model, a candidate contact area for contacting the target object from the dexterous hand, the candidate contact area including at least one of the fingertip area and the fingertip area of ​​the dexterous hand; and select a template contact point corresponding to the dexterous hand from a plurality of contact points in the candidate contact area based on the contact attribute information of each contact point in the candidate contact area.

[0015] A device for generating grasping postures includes a memory storing computer-readable instructions and a processor for reading the computer-readable instructions stored in the memory to implement the grasping posture generation method described above.

[0016] A computer-readable storage medium having stored computer-readable instructions thereon, which, when executed by a computer's processor, cause the processor to perform the above-described method for generating a grasping posture.

[0017] A computer program product comprising computer-readable instructions that, when executed by a processor, implement the above-described method for generating a grasping posture.

[0018] In the above technical solution: This application's technical solution constructs a posture optimization objective function based on the target contact points on the target object and the corresponding template contact points of the dexterous hand. Then, it adjusts the target parameters in the posture optimization objective function according to the dexterous hand's degree-of-freedom compensation strategy. Specifically, compensation can be made from dimensions such as contact point matching weights, wrist perturbation rotation of the dexterous hand, and joint angle constraints. This satisfies the degree-of-freedom compensation requirements of different dimensions, helping to compensate for the problem of limited degrees of freedom in the dexterous hand and reducing the difficulty in generating accurate grasping postures due to limited degrees of freedom. Therefore, this application can improve the accuracy of generating grasping postures when a dexterous hand grasps an object.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings…

[0021] Figure 1 This is a schematic diagram of a system architecture for generating a grasping posture, as described in this application.

[0022] Figure 2 This is a schematic diagram illustrating an application scenario of a grasping posture generation method involved in this application.

[0023] Figure 3 This is a flowchart illustrating a method for generating a grasping posture according to an exemplary embodiment.

[0024] Figure 4 This is a flowchart illustrating a method for generating a grasping posture according to another exemplary embodiment.

[0025] Figure 5 This is a schematic diagram of a stable contact area involved in this application.

[0026] Figure 6 This is a flowchart illustrating a method for generating a grasping posture according to another exemplary embodiment.

[0027] Figure 7This is a flowchart illustrating a method for generating a grasping posture according to another exemplary embodiment.

[0028] Figure 8 This is a block diagram illustrating a grasping posture generation apparatus according to an exemplary embodiment.

[0029] Figure 9 This is a schematic diagram of the structure of a computer system for generating a grasping posture according to an exemplary embodiment. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments identical to those described in this application. Rather, they are merely examples of apparatuses and methods identical to some aspects of this application as detailed in the appended claims.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily need to be performed in the described order. For example, some operations / steps can be broken down, while others can be integrated or partially integrated; therefore, the actual execution order may change depending on the specific circumstances.

[0033] It should be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0034] Before introducing the technical solutions of the embodiments of this application, let's first introduce the technical terms involved in the embodiments of this application.

[0035] A dexterous hand can be a mechanical component with multiple degrees of freedom and multiple joints, and its structure and function are quite similar to those of a human hand. A dexterous hand typically includes a palm, fingers, joints, drive, transmission, and control components. By controlling the movement of different joints, the fingers can perform actions such as bending, extending, or rotating relative to the palm. In the embodiments of this application, the dexterous hand can serve as a component for performing grasping actions, and a target grasping posture for grasping a target object can be generated for the dexterous hand.

[0036] A low-DOF dexterity hand can be a dexterity hand with a relatively small number of degrees of freedom. For example, a low-DOF dexterity hand can be a dexterity hand with no more than 2 degrees of freedom for a single finger and / or no more than 10 degrees of freedom in total. Compared to a real human hand or a high-DOF dexterity hand, a low-DOF dexterity hand has weaker posture adjustment capabilities, but its structure is relatively simple, its reliability is higher, and its manufacturing and maintenance costs are also lower. In the embodiments of this application, a low-DOF dexterity hand can be an example of a dexterity hand, and the grasping posture generation method in the embodiments of this application can be used to improve the accuracy of grasping posture generation when such a dexterity hand grasps a target object.

[0037] A stable contact area can be a region on the surface of a target object suitable for contact with a dexterous hand. Stable contact areas are typically located on relatively smooth parts of the target object's surface, such as locations with minimal changes in normal vectors, curvature, or the absence of obvious sharp protrusions. Stable contact areas are more suitable contact positions for a dexterous hand to grasp the target object. In this embodiment, stable contact areas can be determined from the object point cloud of the target object, and target contact points on the target object can be selected from the object points within the stable contact area.

[0038] In related technologies, high-degree-of-freedom dexterous hands are typically used to perform grasping tasks, and methods such as geometric matching and deep learning are employed to generate grasping postures in a high-dimensional action space. However, this method of generating grasping postures requires a high degree of freedom from the dexterous hand. When applied to low-degree-of-freedom dexterous hands, the generated grasping postures may not be suitable for the low-degree-of-freedom dexterous hand, leading to a decrease in the accuracy of the grasping posture when the dexterous hand grasps objects.

[0039] Based on this, embodiments of this application propose a method for generating a grasping posture, a device for generating a grasping posture, an apparatus for generating a grasping posture, a computer-readable storage medium, and a computer program product. In these embodiments, after constructing a posture optimization objective function based on the target contact point on the target object and the template contact point corresponding to the dexterous hand, the target parameters in the posture optimization objective function are compensated and adjusted according to the degree-of-freedom compensation strategy corresponding to the dexterous hand. Specifically, during the compensation adjustment, compensation can be made from dimensions such as contact point matching weights, wrist perturbation rotation of the dexterous hand, and joint angle constraints, thereby meeting the degree-of-freedom compensation requirements of different dimensions. This helps to compensate for the problem of limited degrees of freedom of the dexterous hand and reduces the difficulty in generating accurate grasping postures due to limited degrees of freedom. Therefore, this application can improve the accuracy of generating grasping postures when a dexterous hand grasps an object.

[0040] Please see Figure 1 , Figure 1 This is a schematic diagram of a system architecture for generating a grasping posture, as described in this application.

[0041] exist Figure 1 In this system architecture, the generation of the grasping posture may include a contact point acquisition module 110, a posture optimization objective function construction module 120, a degree of freedom compensation module 130, a calculation module 140, a simulation module 150, a scoring module 160, and a target grasping posture determination module 170.

[0042] The contact point acquisition module 110 can be used to acquire target contact points on the target object and template contact points corresponding to the dexterous hand. The target contact point can be a point on the surface of the target object that is suitable for contact with the dexterous hand, and the template contact point can be a contact point corresponding to a candidate contact area such as the fingertip area or fingertip area of ​​the dexterous hand.

[0043] In one embodiment of this application, the contact point acquisition module 110 can first generate an object point cloud based on the object mesh model corresponding to the target object, then determine a stable contact area based on the geometric attribute information corresponding to each object point in the object point cloud, and select a target contact point from the stable contact area. The contact point acquisition module 110 can also determine candidate contact areas from the dexterous hand based on the structural model corresponding to the dexterous hand, and select template contact points from the candidate contact areas.

[0044] The posture optimization objective function construction module 120 can be used to construct a posture optimization objective function for dexterous hand grasping a target object based on the target contact point and the template contact point.

[0045] The degree-of-freedom compensation module 130 can be used to compensate and adjust the target parameters in the posture optimization objective function according to the degree-of-freedom compensation strategy corresponding to the dexterous hand. The target parameters may include at least one of the contact point matching weight parameters, wrist perturbation rotation parameters of the dexterous hand, and joint angle constraint parameters of the dexterous hand.

[0046] The calculation module 140 can be used to optimize the objective function based on the adjusted posture, solve for the posture parameters of the dexterous hand, and obtain multiple candidate grasping postures. Different candidate grasping postures can correspond to different combinations of hand rotation matrix (R), hand position (t), and joint angle (θ).

[0047] The simulation module 150 can be used to simulate grasping based on multiple candidate grasping postures. Specifically, the simulation module 150 can control a dexterous hand to grasp a target object according to the candidate grasping postures in a simulation environment, and record the simulation results corresponding to each candidate grasping posture. The simulation results can include information such as whether the target object was successfully grasped, whether slippage occurred during the grasping process, the pose change of the target object, the contact force distribution, and the joint output torque.

[0048] The scoring module 160 can be used to score multiple candidate grasping postures based on the simulation results output by the simulation module 150. The scoring module 160 can determine the score for each candidate grasping posture based on information such as contact stability, slippage, and contact force distribution. Furthermore, the scoring module 160 can update the target contact point based on the simulation results of the current iteration and return the updated target contact point to the contact point acquisition module 110. This allows subsequent iterations to reconstruct the posture optimization objective function based on the updated target contact point and accordingly activate each module for the next iteration.

[0049] It should be noted that the process of returning from the scoring module 160 to the contact point acquisition module 110, the attitude optimization objective function construction module 120, the degree of freedom compensation module 130, and the calculation module 140 can be executed M times, where M can be a natural number of 0, 1, 2, 3, etc.

[0050] Therefore, in the process of generating a target grasping posture, multiple rounds of candidate grasping posture calculation, simulated grasping, and scoring updates can be performed. In each iteration, the system can update the target contact point based on the simulation results of the previous round and reconstruct the posture optimization objective function. Then, the degree of freedom compensation module 130 compensates and adjusts the reconstructed posture optimization objective function so that the candidate grasping postures calculated in the next round are more suitable for dexterous hand grasping the target object.

[0051] The target grasping posture determination module 170 can be used to determine the target grasping posture based on multiple candidate grasping postures corresponding to the termination iteration when the iteration termination condition is met.

[0052] Therefore, this grasping posture generation system can adjust the posture optimization objective function based on the contact relationship between the target object and the dexterous hand, combined with the degree of freedom compensation method, so as to generate a target grasping posture that is more suitable for the dexterous hand to grasp the target object through iteration.

[0053] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of a grasping posture generation method involved in this application.

[0054] like Figure 2 As shown, this application scenario can include a dexterous hand and a target object. The dexterous hand can be an actuator used to grasp the target object. It can be located at the end of a robot's robotic arm and move to the vicinity of the target object under the actuation of the robotic arm. The dexterous hand can include a palm, multiple fingers, and corresponding joint structures for each finger, enabling it to grasp the target object by adjusting the palm posture and finger joint angles.

[0055] In one embodiment of this application, the dexterous hand can be a low-degree-of-freedom dexterous hand. For example, a low-degree-of-freedom dexterous hand can be a dexterous hand with no more than 2 degrees of freedom for a single finger and / or no more than 10 degrees of freedom in total. Low-degree-of-freedom dexterous hands have relatively simple structures and high reliability, but their posture adjustment capabilities are relatively limited. Therefore, when grasping a target object, it is necessary to generate a grasping posture corresponding to its structure in order to accurately grasp the target object. It should be noted that the grasping posture generation method provided in this application embodiment can also be applied to high-degree-of-freedom dexterous hands, and this application embodiment does not limit this application.

[0056] The target object can be an object that the dexterous hand needs to grasp. For example, the target object can be an object with a touchable surface, such as a cup, bottle, or box. The system can generate a target grasping posture for the dexterous hand to grasp the target object based on the target contact points on the target object and the corresponding template contact points of the dexterous hand, enabling the dexterous hand to grasp the target object according to the target grasping posture.

[0057] It should be noted that the grasping posture generation system in this application can be used in both simulation and real-world grasping scenarios. In simulation scenarios, the grasping posture generation system can be used to generate a target grasping posture for a dexterous hand to grasp a target object within the simulation environment, and to simulate the process of the dexterous hand grasping the target object based on this target grasping posture. In real-world grasping scenarios, the grasping posture generation system can be used to generate a target grasping posture for a robot's corresponding dexterous hand to grasp a target object, enabling the robot to control the dexterous hand to grasp the target object according to this target grasping posture.

[0058] Furthermore, the grasping posture generation system of this application can be deployed on a server or a terminal device. When deployed on a server, the terminal device can access the server and perform processes such as grasping posture generation and simulated grasping through the server. When deployed on a terminal device, the terminal device can perform processes such as grasping posture generation and simulated grasping locally, or it can control the robot to perform a real grasping task based on the generated target grasping posture.

[0059] In the embodiments of this application, terminal devices include, but are not limited to, extended reality devices (virtual reality devices, augmented reality devices, mixed reality devices, etc.), mobile phones, computers (tablets, laptops, desktop computers, etc.), smart home devices (televisions, refrigerators, air conditioners, washing machines, robot vacuums, etc.), smart wearable devices (bracelets, watches, etc.).

[0060] In the embodiments of this application, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The server cluster or distributed system includes cloud servers for providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0061] It is understood that a communication connection is established between the terminal device and the server via a wired or wireless network. Exemplarily, the wireless or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any other network, including but not limited to a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), mobile, wired or wireless networks, private networks, or any combination of virtual private networks.

[0062] In the specific implementation of this application, user-related data is involved, such as the pinyin sequence, target text, and input process data entered by the user. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0063] Please see Figure 3 , Figure 3 This is a flowchart illustrating a method for generating a grasping pose according to an exemplary embodiment. This method can be applied to... Figure 1In the grasping posture generation system (hereinafter referred to as the system), the system can be deployed on a server. Of course, this method can also be applied to other implementation environments, and there are no restrictions on the subject executing this method.

[0064] like Figure 3 As shown, in an exemplary embodiment, the method includes at least the following steps.

[0065] S310. Obtain the target contact point on the target object and the template contact point corresponding to the dexterous hand.

[0066] The target contact point can be a point on the surface of the target object that is used to contact the dexterous hand, and the template contact point can be a point on the dexterous hand that is used to contact the target object.

[0067] It should be noted that this application can calculate the target grasping posture through an iterative process, so at this point, we can enter the first iteration round.

[0068] In one embodiment of this application, "acquiring the target contact point on the target object" in S310 may specifically include, for example: Figure 4 S410 to S440 are shown in detail below.

[0069] S410. Obtain the object mesh model corresponding to the target object.

[0070] The target object can be the object that a dexterous hand needs to grasp. The object mesh model can be used to represent the three-dimensional surface shape of the target object. This object mesh model can include multiple mesh patches, which can characterize information such as the outer contour and surface structure of the target object. This object mesh model can be denoted as... For example, the object mesh model can be a triangular mesh model corresponding to the target object; of course, the object mesh model can also be other mesh models that can represent the surface shape of the target object, and the embodiments of this application do not limit this.

[0071] In one embodiment of this application, the object mesh model can be constructed by an external modeling tool or generated based on target object data collected by a sensor. For example, surface data of the target object can be collected by an external 3D acquisition device, and then an object mesh model corresponding to the target object can be generated based on the collected surface data.

[0072] S420. Perform surface sampling on the object mesh model to obtain the object point cloud corresponding to the target object. The object point cloud includes multiple object points.

[0073] After obtaining the object mesh model corresponding to the target object, the surface of the object mesh model can be sampled to obtain the object point cloud corresponding to the target object. This object point cloud is denoted as... An object point cloud can include multiple object points distributed on the surface of a target object, with each object point corresponding to a sampling location on the target object's surface. By converting the object mesh model into an object point cloud, the continuous surface of the target object can be discretized into multiple object points, making it easier to determine which regions are more suitable as contact areas for a dexterous hand based on the geometric attribute information of each object point.

[0074] Optionally, the system can use methods such as Poisson disk sampling or uniform sampling to perform surface sampling on the object mesh model to obtain the object point cloud. Of course, in other embodiments, other surface sampling methods can also be used to obtain the object point cloud, and this application embodiment does not limit this.

[0075] S430. Determine the stable contact region from the object point cloud based on the geometric attribute information corresponding to the object point cloud.

[0076] The geometric attribute information can be used to represent the geometric information of the object's point cloud. Optionally, the geometric attribute information may include the principal axis direction of the object's point cloud. With the position of the center of gravity The system can use specific geometric algorithms to extract geometric attribute information from the point cloud of the object, such as Principal Component Analysis (PCA).

[0077] After determining the geometric attribute information corresponding to the object point cloud, a stable contact region can be determined from the object point cloud by combining the distribution of each object point in the object point cloud on the surface of the target object. This stable contact region can be denoted as... ,satisfy .

[0078] like Figure 5 The diagram shown is a schematic representation of a stable contact region according to this application. Figure 5 In this context, the object point cloud can include multiple object points distributed on the surface of the target object, and the stable contact region can be a region in the object point cloud located in a relatively smooth position on the surface of the target object. For example, based on the principal axis direction and the position of the center of gravity corresponding to the object point cloud, candidate regions can be determined on the surface of the target object, and regions with small changes in normal vector or curvature can be selected from the candidate regions as stable contact regions.

[0079] It should be noted that the stable contact area can be any region on the surface of the target object suitable for contact with a dexterous hand. Compared to sharp corners, edges, or local protrusions, the surface variation of the stable contact area is smaller, making it easier for a dexterous hand to form a stable contact with this area. This stable contact area can be a continuous region on the surface of the target object, or it can be a region composed of multiple adjacent or nearby object points. The embodiments of this application do not limit the specific shape of the stable contact area.

[0080] Optionally, when determining a stable contact region from the object point cloud, the system can determine the local neighborhood corresponding to each object point based on the three-dimensional coordinates of each object point in the object point cloud, and perform planar feature matching based on multiple object points in the local neighborhood. Specifically, for any object point in the object point cloud, the system can obtain multiple neighboring object points whose distance from the object point is less than a preset neighborhood distance threshold based on the object point's three-dimensional coordinates.

[0081] After determining the local neighborhood corresponding to the object point, the system can fit a local reference plane based on the object point and multiple neighboring object points, and determine the distances from the multiple neighboring object points to the local reference plane. If the distances from the multiple neighboring object points to the local reference plane are all less than a preset plane distance threshold, then the local region where the object point is located can be considered to have planar features.

[0082] Optionally, the system can also determine the changes in the normal vectors of the local neighborhood based on the normal vectors corresponding to multiple object points in the local neighborhood. For example, it can determine the angle between the normal vectors of multiple object points, or the angle between the normal vectors of multiple object points and the normal vector of the local reference plane. If the angle is less than a preset angle threshold, the local neighborhood can be considered to have planar features.

[0083] After determining the planar features and normal vector changes corresponding to each object point, the system can identify local regions with planar features as stable contact regions.

[0084] Optionally, if there are stable contact areas that are close together, the system can merge at least two close stable contact areas. This allows different stable contact areas to be connected together, avoiding a scattered distribution of stable contact areas, which helps to improve the accuracy of subsequent generation of grasping posture.

[0085] S440. Select the target contact point on the target object from multiple object points in the stable contact area.

[0086] After determining the stable contact area, the system can select the target contact point on the target object from multiple object points within the stable contact area. The target contact point can be used to represent the position on the surface of the target object where it will make contact with the dexterous hand. Subsequently, the target contact point can be matched with the template contact point corresponding to the dexterous hand to generate the target grasping posture when the dexterous hand grasps the target object.

[0087] Optionally, the system can select a target contact point from multiple object points in the stable contact area based on information such as the position of the object point in the stable contact area, the spacing between different object points, and the surface smoothness of the area where the object point is located. For example, multiple object points that are relatively evenly distributed can be selected as target contact points from the stable contact area, or object points with smoother local surfaces can be selected as target contact points.

[0088] Through the optional embodiments described in S410 to S440, an object point cloud can be obtained based on the object mesh model of the target object, a stable contact area can be determined based on the geometric attribute information corresponding to the object point cloud, and a target contact point can be selected from the stable contact area. This makes the area corresponding to the target contact point more suitable for contact, reducing the likelihood of selecting the target contact point in areas with significant surface variations on the target object, thereby helping to improve the accuracy of subsequent grasping posture generation.

[0089] In one embodiment of this application, the step of "obtaining the template contact point corresponding to the dexterous hand" in S310 may specifically include, for example: Figure 6 S610 to S630 are shown in detail below.

[0090] S610. Obtain the structural model corresponding to the dexterous hand.

[0091] A structural model corresponding to a dexterous hand can be used to represent the structural information of the dexterous hand. This structural model can include the palm, fingers, joints, and geometric information of the surface of each finger. Using this structural model, the system can determine which areas are suitable for contact with the target object when grasping it.

[0092] S620. Based on the structural model, determine candidate contact areas from the dexterous hand for contacting the target object, the candidate contact areas including at least one of the fingertip area and the fingertip area of ​​the dexterous hand.

[0093] After obtaining the structural model of the dexterous hand, candidate contact areas for contacting the target object can be determined from the dexterous hand based on this model. Candidate contact areas can be typical contact areas in the dexterous hand that are relatively easy to contact the surface of the target object during grasping. For example, candidate contact areas can include at least one of the fingertip area and the fingertip area. The fingertip area can be the area near the fingertip, and the fingertip area can be the area on the inside of the finger used to contact the surface of the target object.

[0094] S630. Based on the contact attribute information of each contact point in the candidate contact area, select the template contact point corresponding to the dexterous hand from multiple contact points in the candidate contact area.

[0095] After determining the candidate contact areas, the system can identify multiple contact points within these areas and, based on the contact attribute information of each contact point, select the template contact point corresponding to the dexterous hand. The contact attribute information can include the position of the contact point, the corresponding normal vector, and other information. Specifically, the position of the contact point indicates its location within the dexterous hand's structural model, and the normal vector indicates the orientation of the surface on which the contact point is located.

[0096] Optionally, surface points that may come into contact with the target object, such as the fingertip area and fingertip area, can be designated as template contact points. These template contact points can be denoted as... .

[0097] Through the optional embodiments described in S610 to S630, template contact points can be selected from areas more suitable for grasping contact, such as the fingertip area and the fingertip area, based on the structural model of the dexterous hand. This eliminates unsuitable dexterous hand surface areas, making the template contact points more consistent with the actual contact position when the dexterous hand grasps the target object, thereby helping to improve the accuracy of subsequent grasping posture generation.

[0098] S320. Based on the target contact point and the template contact point, construct an objective function for posture optimization of the dexterous hand grasping the target object.

[0099] In this embodiment, after determining the target contact point on the target object and the template contact point corresponding to the dexterous hand, an attitude optimization objective function can be constructed based on the residual between the target contact point and the template contact point.

[0100] The posture optimization objective function can include the posture parameters of the dexterous hand, which can include the rigid body pose of the hand and the joint angle vectors. (Hand rigid body pose) This can include a hand rotation matrix (R) and a hand translation vector (t). The hand rotation matrix represents the orientation of the dexterous hand's palm, and the hand translation vector represents the position of the dexterous hand's palm in three-dimensional space, which can be represented using (x, y, z) spatial coordinates. Joint angle vector ( () can be used to represent the joint angles corresponding to each joint in a dexterous hand.

[0101] Alternatively, the attitude optimization objective function can be constructed by minimizing the contact point residual. This attitude optimization objective function can be expressed as:

[0102] Where N can represent the total number of object points in the stable contact region, j can represent the j-th object point, and j is a positive integer less than or equal to N. This can represent the j-th target contact point. Indicates a stable contact area; This can represent the j-th template contact point; It can represent the positive kinematic mapping of a dexterous hand, and is used to determine the spatial position of the template contact point based on the template contact point, the rigid body pose of the palm, and the joint angle vector; This can represent the contact point matching weight parameter between the j-th target contact point and the j-th template contact point; λ can represent the joint angle constraint regularization term, used to constrain the joint angles of a dexterous hand and prevent them from going out of bounds; λ can represent the weight factor of the regularization term corresponding to the joint angle constraint term.

[0103] It should be noted that the number of object points in the stable contact region may differ from the number of template contact points corresponding to the dexterous hand. To facilitate the establishment of the correspondence between target contact points and template contact points, the object points and template contact points in the stable contact region can be downsampled to make the number of target contact points participating in posture optimization consistent with the number of template contact points. For example, after downsampling, N target contact points and N template contact points can be obtained, and the j-th target contact point and the j-th template contact point can be considered as a pair of contact points.

[0104] S330. Based on the target contact point, perform inverse kinematics solution on the robotic arm to which the dexterous hand belongs to obtain the wrist reachability result of the robotic arm; if the wrist reachability result indicates that the wrist of the robotic arm cannot reach the target contact point, then calculate the wrist perturbation rotation amount corresponding to the dexterous hand according to the principal axis of the target object and the preset perturbation angle range; according to the wrist perturbation rotation amount, adjust the wrist perturbation rotation parameter in the posture optimization objective function to obtain the adjusted posture optimization objective function.

[0105] Based on the degree-of-freedom compensation strategy corresponding to the dexterous hand, the objective parameters in the posture optimization objective function are compensated and adjusted to obtain the adjusted posture optimization objective function. The objective parameters include at least one of the contact point matching weight parameters, the wrist perturbation rotation parameters of the dexterous hand, and the joint angle constraint parameters of the dexterous hand.

[0106] In this embodiment, the system can set corresponding degree-of-freedom compensation strategies from the dimensions of contact point matching weight, wrist perturbation rotation of the dexterous hand, and joint angle constraint of the dexterous hand, respectively, so as to compensate and adjust the corresponding parameters in the posture optimization objective function.

[0107] Specifically, for the contact point matching weight dimension, the system can compensate for the degree of freedom in the following way: determine the contact matching result based on the target contact point and the template contact point; if the contact matching result indicates that a single contact point fails to cover the stable contact area of ​​the target object, then reduce the contact point matching weight parameter to obtain the adjusted contact point matching weight parameter; adjust the posture optimization objective function based on the adjusted contact point matching weight parameter to obtain the adjusted posture optimization objective function.

[0108] The contact matching result can be used to represent the coverage relationship between the target contact point and the template contact point. Specifically, for a template contact point, if it does not fall within the stable contact area of ​​the target object in the current grasping posture, it can be considered that the template contact point does not cover the stable contact area. For a target contact point, if it is located within the stable contact area but is not covered by the template contact point on the dexterous hand side in the current contact matching result, it can be considered that the target contact point is an uncovered contact point. Therefore, when the target contact point or template contact point participating in the contact matching exhibits the above-mentioned conditions, the contact matching result can be considered to indicate that a single contact point fails to cover the stable contact area of ​​the target object.

[0109] If the contact matching result indicates that a single contact point fails to cover the stable contact area of ​​the target object, the system can attenuate the contact point matching weight parameter corresponding to that single contact point. For example, the adjusted contact point matching weight parameter can satisfy the following relationship: .

[0110] in, This represents the unadjusted matching weight parameter corresponding to the j-th contact point. This represents the adjusted matching weight parameter corresponding to the j-th contact point. This represents the weight attenuation coefficient corresponding to the j-th contact point. Optionally, It can be a value greater than 0 and less than 1, for example... .

[0111] In this way, after adjusting the parameters as described above for each individual contact point that fails to cover the stable contact area of ​​the target object, the adjusted contact point matching weight parameters can be obtained. The system can then adjust the attitude optimization objective function based on the adjusted contact point matching weight parameters to obtain the adjusted attitude optimization objective function.

[0112] By using the degree-of-freedom compensation method of the contact point matching weight dimension, the influence of a single contact point on the posture optimization objective function can be reduced when a single contact point fails to cover the stable contact area. This reduces the situation where the grasping posture is difficult to generate due to some contact points failing to cover the stable contact area. This can improve the tolerance of contact points in the case of mismatch, and does not require each contact point to strictly cover the stable contact area, thereby helping to improve the generation stability of the grasping posture when the dexterous hand grasps the target object.

[0113] For the wrist perturbation rotation dimension of the dexterous hand, the system can compensate for the degree of freedom in the following way: Based on the target contact point, perform inverse kinematics solution on the robotic arm to which the dexterous hand belongs to obtain the wrist reachability result of the robotic arm; if the wrist reachability result indicates that the wrist of the robotic arm cannot reach the target contact point, then calculate the wrist perturbation rotation amount corresponding to the dexterous hand according to the principal axis of the target object and the preset perturbation angle range; based on the wrist perturbation rotation amount, adjust the wrist perturbation rotation parameter in the posture optimization objective function to obtain the adjusted posture optimization objective function.

[0114] The wrist reachability result can be used to indicate whether the wrist of the robotic arm to which the dexterous hand belongs can reach the position corresponding to the target contact point. The system can perform inverse kinematics solution on the robotic arm based on the target contact point. If the inverse kinematics solution result shows that the wrist of the robotic arm cannot reach the target contact point under the current rotation matrix R, then the target contact point can be considered unreachable.

[0115] At this point, the system can apply a perturbation rotation to the wrist posture of the dexterous hand based on the principal axis of the target object and a preset perturbation angle range. The wrist perturbation rotation parameter in the posture optimization objective function can be the hand rotation matrix R, which can be updated according to the following relationship: .

[0116] Where R represents the palm rotation matrix before adjustment. This represents the adjusted hand rotation matrix. Indicates along the principal axis of the object The applied perturbation rotation, Indicates the angle of the applied disturbance. Optionally, There is a corresponding preset disturbance angle range, for example .For example, It can be set to 20°, and the sampling step size of the perturbation angle can be 5°.

[0117] After calculating the wrist perturbation rotation amount, the system can update the wrist perturbation rotation parameter in the attitude optimization objective function based on the wrist perturbation rotation amount, and obtain the adjusted attitude optimization objective function.

[0118] By using this wrist perturbation rotation dimension degree of freedom compensation method, when the robotic arm's wrist cannot directly reach the target contact point, the dexterous hand's wrist posture can be adjusted along the main axis of the target object. This allows the posture optimization process to search under more wrist postures, reducing the situation where the grasping posture cannot be generated due to wrist posture limitations. This helps to improve the accuracy of the grasping posture generation when the dexterous hand grasps the target object.

[0119] For the joint angle constraint dimension of the dexterous hand, the system can compensate for the degree of freedom in the following way: obtain the joint angles of the dexterous hand and the joint angle boundaries corresponding to the joint angles; determine the joint angle constraint parameters based on the deviation between the joint angles and the joint angle boundaries; adjust the joint angle constraint terms in the posture optimization objective function based on the joint angle constraint parameters to obtain the adjusted posture optimization objective function.

[0120] The joint angle boundary can include the upper boundary and the lower boundary of the joint angle. For the k-th joint in a dexterous hand, the joint angle corresponding to that joint can be denoted as... The upper boundary of the joint angle corresponding to this joint is denoted as , where represents the maximum allowed joint angle for the k-th joint. The lower boundary of the joint angle corresponding to this joint can be denoted as . , where represents the minimum joint angle that the k-th joint is allowed to reach.

[0121] After obtaining the joint angles and corresponding joint angle boundaries of the dexterous hand, the system can determine the joint angle constraint parameters based on the deviation between the joint angles and the joint angle boundaries. The joint angle constraint terms in the posture optimization objective function can satisfy the following relationship:

[0122] in, This indicates the joint angle constraint term. This represents the joint penalty coefficient corresponding to the k-th joint.

[0123] Based on the above relationship, it can be seen that when the joint angle of the k-th joint does not exceed the joint angle boundary, the penalty value corresponding to that joint in the joint angle constraint term is 0; when the joint angle of the k-th joint exceeds the upper or lower boundary of the joint angle, the penalty value corresponding to that joint in the joint angle constraint term can increase as the extent of the exceedance increases.

[0124] After determining the joint angle constraint parameters, the system can adjust the joint angle constraint terms in the attitude optimization objective function based on these parameters, resulting in an adjusted attitude optimization objective function. This adjustment allows for minor out-of-bounds movement of the joint angles near the joint critical region.

[0125] By using this joint angle constraint dimension degree-of-freedom compensation method, minor overshoots of the joint angle can be allowed near the joint critical region, which helps to improve the tolerance for such errors, improves the robustness of candidate grasping posture generation, and thus helps to improve the accuracy of grasping posture generation when a dexterous hand grasps a target object.

[0126] S340. Based on the adjusted posture, optimize the objective function to determine the target grasping posture for the dexterous hand to grasp the target object.

[0127] The system can solve the adjusted posture optimization objective function to obtain multiple candidate grasping postures.

[0128] In one embodiment of this application, S340 may specifically include, as follows: Figure 7 S710 to S740 are shown in detail below.

[0129] S710. In the current iteration, the posture parameters of the dexterous hand are solved based on the adjusted posture optimization objective function to obtain multiple candidate grasping postures corresponding to the current iteration.

[0130] In the current iteration, the system can solve for the hand rotation matrix R, hand position t, and joint angle vector θ based on the adjusted posture optimization objective function. Since there can be multiple possible solutions in the posture optimization objective function, the system can obtain multiple candidate grasping postures corresponding to the current iteration.

[0131] Optionally, each candidate grasping posture can be denoted as .in, Indicates the i-th candidate grasping posture. This represents the hand rotation matrix corresponding to the i-th candidate grasping posture. This represents the hand position corresponding to the i-th candidate grasping posture. This represents the joint angle vector corresponding to the i-th candidate grasping posture. Different candidate grasping postures can correspond to different... , and The combination of .

[0132] S720. Simulate grasping the target object based on multiple candidate grasping postures, and obtain the simulation results corresponding to the multiple candidate grasping postures respectively.

[0133] In the current iteration, the system can input each candidate grasping posture into the simulation platform, and control the dexterous hand to grasp the target object according to the candidate grasping posture. For the i-th candidate grasping posture... The system can operate in a simulation environment based on... , and Control the movement of the dexterous hand and record the simulation results corresponding to the candidate grasping posture.

[0134] The simulation results can include information such as whether the target object was successfully grasped, whether slippage occurred during the grasping process, changes in the target object's pose, contact force distribution, and joint output torque. The system can then score each candidate grasping posture based on this information. The corresponding scoring function can be expressed as:

[0135] The meanings of each indicator are as follows: : Represents the standard deviation of the contact force distribution, which can be used to measure whether the force is balanced during the grasping process; : Represents the pose drift of the target object from its initial state to its release state; : Indicates the contact area between the dexterous hand and the target object; The penalty for whether slippage occurred indicates whether slippage occurred during the grabbing process; The sum of squares of the joint output torques can measure the stability of energy consumption during the grasping process; (s=1, 2, 3, 4, 5): These represent the weights of the corresponding indicators, which can be set based on experience or obtained through calibration.

[0136] Among them, the scoring results The lower the value, the worse the grasping effect corresponding to the candidate grasping posture.

[0137] In one embodiment of this application, the simulation results may further include failure modes corresponding to candidate grasping postures. For example, if the target object is not stably grasped during the simulation, insufficient grasping force can be recorded; if the contact area between the dexterous hand and the target object is too small, a small contact area can be recorded; if the target object deviates significantly during the grasping process, poor posture stability can be recorded; if the target object slides relative to the dexterous hand, slippage can be recorded. In this way, the system can determine whether a candidate grasping posture is suitable for grasping a target object based on the simulation results of each candidate grasping posture.

[0138] S730. Based on the simulation results, update the target contact point and reconstruct the attitude optimization objective function based on the updated target contact point and template contact point until the iteration termination condition is met.

[0139] After obtaining the simulation results corresponding to the current iteration, the system can update the target contact point based on the simulation results and reconstruct the attitude optimization objective function based on the updated target contact point.

[0140] In one embodiment of this application, the target contact point can be updated in the following manner.

[0141] The system can first generate a point cloud of objects. Set contact importance weights for each object point p in the data. This contact importance weight can be used to represent the degree to which a corresponding object point is suitable as a target contact point. Optionally, the initial contact importance weight for each object point p... It can be set to 1.

[0142] Next, after completing the simulation and obtaining the scoring results in the current iteration, the system can update the contact importance weights of each object point in the object point cloud based on the scoring results. Optionally, after the k-th iteration, the contact importance weights corresponding to object point p can be updated according to the following relationship:

[0143] in, This represents the contact importance weight of object point p in the k-th iteration (k is a positive integer greater than 0). This represents the contact importance weight of object point p in the (k+1)th iteration. Indicates the feedback update rate. Indicates the candidate grasping posture The relationship between the scoring results and the contact importance weight of object point p. Optionally, It can satisfy .

[0144] After updating the contact importance weights, the system can analyze the object point cloud. The system selects object points whose contact importance weight is greater than a preset weight threshold (denoted as τ), and obtains the stable contact region corresponding to the next iteration based on the selected object points. For example, the stable contact region corresponding to the next iteration can be represented as:

[0145] in, This represents the stable contact region corresponding to the (k+1)th iteration. The system can reselect the target contact point from this stable contact region and reconstruct the attitude optimization objective function based on the reselected target contact point and the template contact point.

[0146] In one embodiment of this application, if a wrist perturbation rotation degree-of-freedom compensation strategy is used in the current iteration, the system can also adjust the perturbation angle range based on the simulation results of the current iteration. Optionally, the perturbation angle range can be updated according to the following relationship:

[0147] in, This represents the maximum perturbation angle corresponding to the k-th iteration. This represents the maximum perturbation angle corresponding to the (k+1)th iteration. This indicates the expansion step size of the disturbance angle range. This indicates whether a capture failed in the current iteration. For example, in... When this happens, it indicates that the current iteration failed to capture the target, and the wrist perturbation rotation range can be expanded in the next iteration; When the current iteration fails to capture, the wrist perturbation rotation range in the next iteration can be left unexpanded.

[0148] In one embodiment of this application, the system can also update the contact point matching weights based on simulation results. Optionally, the contact point matching weights corresponding to the j-th contact point can be updated according to the following relationship:

[0149] in, This represents the contact point matching weight corresponding to the j-th contact point in the k-th iteration. This represents the contact point matching weight corresponding to the j-th contact point in the (k+1)-th iteration. This represents the target matching weights determined based on the simulation results. This represents the smoothing update coefficient. For contact points with severe slippage in the simulation results, their corresponding... Set to a lower value; for contact points that show relatively stable contact in the simulation results, their corresponding values ​​can be adjusted. Set to a higher value.

[0150] After updating at least one of the target contact point, the perturbation angle range, and the contact point matching weights, the system can reconstruct the attitude optimization objective function based on the updated target contact point and template contact point, and proceed to the next iteration. This process is repeated until the iteration termination condition is met.

[0151] The iteration termination condition can be set by those skilled in the art and is not limited here. For example, it could be that the iteration stops when a preset iteration number threshold is reached. Alternatively, it could detect whether the pose difference between the rigid body pose (R, t) of the hand of the candidate grasping posture with the highest score in each iteration and the rigid body pose (R, t) of the candidate grasping posture with the highest score in the previous iteration is less than or equal to a preset pose difference threshold. If so, the iteration can stop.

[0152] S740. When the iteration termination condition is met, the target grasping posture is determined based on the multiple candidate grasping postures corresponding to the termination iteration round.

[0153] When the iteration termination condition is met, the system can use the iteration round that reached the termination condition as the termination iteration round, and determine the target grasping posture that the dexterous hand will finally use when grasping the target object based on the multiple candidate grasping postures obtained in the termination iteration round.

[0154] In one embodiment of this application, S740 may include S741 to S742, as detailed below.

[0155] S741. When the iteration termination condition is met, obtain the termination score results corresponding to the multiple candidate grasping postures for the termination iteration round.

[0156] When the iteration termination condition is met, the system can obtain the score results corresponding to each candidate grasping posture in the termination iteration round. At this time, there are still multiple candidate grasping postures.

[0157] The termination score can be obtained from the simulation results in the termination iteration round.

[0158] S742. The candidate grasping posture with the highest termination score is determined as the target grasping posture.

[0159] The system can compare multiple termination scores and determine the candidate grasping posture with the highest termination score as the target grasping posture.

[0160] Through the optional embodiments described in S741 to S742, after the iteration process ends, the candidate grasping posture with the highest score can be selected as the target grasping posture from among the multiple candidate grasping postures corresponding to the terminating iteration round. In this way, through multiple simulation iterations and multiple optimizations of the posture optimization objective function, the final target grasping posture is more accurate.

[0161] The method of this application constructs a posture optimization objective function based on the target contact points on the target object and the template contact points corresponding to the dexterous hand. Then, it adjusts the target parameters in the posture optimization objective function according to the dexterous hand's degree-of-freedom compensation strategy. Specifically, compensation can be made from dimensions such as contact point matching weights, wrist perturbation rotation of the dexterous hand, and joint angle constraints. This satisfies the degree-of-freedom compensation requirements of different dimensions, helping to compensate for the problem of limited degrees of freedom in the dexterous hand and reducing the difficulty in generating accurate grasping postures due to limited degrees of freedom. Therefore, this application can improve the accuracy of generating grasping postures when a dexterous hand grasps an object.

[0162] Figure 8 This is a block diagram illustrating a grasping posture generation apparatus according to one embodiment of this application. Figure 8 The diagram shows a device for generating a grasping posture, the device comprising: A device for generating a grasping posture, comprising: The acquisition unit 810 is used to acquire the target contact point on the target object and the template contact point corresponding to the dexterous hand; The building unit 820 is used to build a posture optimization objective function for the dexterous hand grasping the target object based on the target contact point and the template contact point; The adjustment unit 830 is used to perform inverse kinematics solution on the robotic arm to which the dexterous hand belongs based on the target contact point to obtain the wrist reachability result of the robotic arm; if the wrist reachability result indicates that the wrist of the robotic arm cannot reach the target contact point, then the wrist perturbation rotation amount corresponding to the dexterous hand is calculated according to the principal axis of the target object and the preset perturbation angle range; according to the wrist perturbation rotation amount, the wrist perturbation rotation parameter in the posture optimization objective function is adjusted to obtain the adjusted posture optimization objective function; The processing unit 840 is used to optimize the objective function based on the adjusted posture and determine the target grasping posture for the dexterous hand to grasp the target object.

[0163] In one embodiment of this application, based on the aforementioned scheme, before determining the target grasping posture of the dexterous hand grasping the target object according to the adjusted posture optimization objective function, the processing unit 840 is further configured to determine the contact matching result based on the target contact point and the template contact point; if the contact matching result indicates that a single contact point fails to cover the stable contact area of ​​the target object, the contact point matching weight parameter is reduced to obtain the adjusted contact point matching weight parameter; the posture optimization objective function is adjusted according to the adjusted contact point matching weight parameter to obtain the adjusted posture optimization objective function.

[0164] In one embodiment of this application, based on the aforementioned scheme, before determining the target grasping posture of the dexterous hand grasping the target object according to the adjusted posture optimization objective function, the acquisition unit 810 is further used to acquire the joint angles of the dexterous hand and the joint angle boundaries corresponding to the joint angles; determine the joint angle constraint parameters according to the deviation between the joint angles and the joint angle boundaries; and adjust the joint angle constraint terms in the posture optimization objective function according to the joint angle constraint parameters to obtain the adjusted posture optimization objective function.

[0165] In one embodiment of this application, based on the aforementioned scheme, the processing unit 840 is further configured to: solve for the posture parameters of the dexterous hand in the current iteration based on the adjusted posture optimization objective function, and obtain multiple candidate grasping postures corresponding to the current iteration; simulate grasping a target object based on the multiple candidate grasping postures, and obtain simulation results corresponding to the multiple candidate grasping postures respectively; update the target contact point based on the simulation results, and reconstruct the posture optimization objective function based on the updated target contact point and template contact point, until the iteration termination condition is reached; when the iteration termination condition is reached, determine the target grasping posture based on the multiple candidate grasping postures corresponding to the terminated iteration.

[0166] In one embodiment of this application, based on the aforementioned scheme, the acquisition unit 810 is further configured to acquire the termination score results corresponding to the multiple candidate grasping postures corresponding to the termination iteration round when the iteration termination condition is reached; and determine the candidate grasping posture with the highest termination score as the target grasping posture.

[0167] In one embodiment of this application, based on the aforementioned scheme, the acquisition unit 810 is further configured to acquire an object mesh model corresponding to the target object; perform surface sampling on the object mesh model to obtain an object point cloud corresponding to the target object, the object point cloud including multiple object points; determine a stable contact region from the object point cloud based on the geometric attribute information corresponding to the object point cloud; and select the target contact point on the target object from the multiple object points in the stable contact region.

[0168] In one embodiment of this application, based on the aforementioned scheme, the acquisition unit 810 is further configured to acquire a structural model corresponding to the dexterous hand; determine candidate contact areas for contacting the target object from the dexterous hand according to the structural model, the candidate contact areas including at least one of the fingertip area and the fingertip area of ​​the dexterous hand; and select the template contact point corresponding to the dexterous hand from multiple contact points in the candidate contact areas according to the contact attribute information of each contact point in the candidate contact areas.

[0169] It should be noted that the apparatus provided in the foregoing embodiments and the method provided in the foregoing embodiments belong to the same concept, and the specific way in which each module and unit performs operations has been described in detail in the method embodiments.

[0170] Embodiments of this application also provide a grasping posture generation device, including: a memory storing computer-readable instructions; and a processor reading the computer-readable instructions stored in the memory to execute the grasping posture generation method as described above.

[0171] Figure 9 This is a schematic diagram of the structure of a computer system suitable for generating a grasping posture to implement the embodiments of this application.

[0172] It should be noted that, Figure 9 The computer system 900 of the grasping posture generation device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0173] like Figure 9 As shown, the computer system 900 includes a Central Processing Unit (CPU) 901, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 902 or programs loaded from storage portion 908 into Random Access Memory (RAM) 903. The RAM 903 also stores various programs and data required for system operation. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An Input / Output (I / O) interface 905 is also connected to the bus 904.

[0174] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. Removable media 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 910 as needed so that computer programs read from them can be installed into storage section 908 as needed.

[0175] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising computer-readable instructions that, when executed by a processor, implement the aforementioned method for generating grasping postures. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable media 911. When the computer program is executed by central processing unit (CPU) 901, it performs various functions defined in the system of this application.

[0176] It should be noted that the computer-readable storage medium shown in the embodiments of this application can also be referred to as a computer medium, which can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above. A computer-readable medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable medium may include, but are not limited to: an electrical connection having at least one wire, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains at least one executable instruction for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer-readable instructions.

[0178] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0179] Another aspect of this application provides a computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a processor, cause the processor to implement the aforementioned method for generating a grasping posture. This computer-readable medium may be included in the grasping posture generation device described in the above embodiments, or it may exist independently and not incorporated into the grasping posture generation device.

[0180] Another aspect of this application provides a computer program product including computer-readable instructions stored in a computer-readable medium. A processor of a computer device reads the computer-readable instructions from the computer-readable medium and executes the instructions, causing the computer device to perform the grasping gesture generation method provided in the various embodiments described above.

[0181] The above content is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A method for generating a grasping posture, characterized in that, include: The target contact point on the target object and the template contact point corresponding to the dexterous hand are obtained. The target contact point is located within the stable contact area of ​​the target object. The template contact point is included in a plurality of contact points in the candidate contact area. The candidate contact area includes at least one of the fingertip area and the fingertip area of ​​the dexterous hand. Based on the target contact point and the template contact point, construct an objective function for posture optimization of the dexterous hand grasping the target object; Based on the target contact point, the inverse kinematics of the robotic arm to which the dexterous hand belongs is solved to obtain the wrist accessibility result of the robotic arm; If the wrist reachability result indicates that the robotic arm's wrist cannot reach the target contact point, then the wrist perturbation rotation amount corresponding to the dexterous hand is calculated based on the object's principal axis and the preset perturbation angle range. Based on the wrist perturbation rotation amount, the wrist perturbation rotation parameter in the posture optimization objective function is adjusted to obtain the adjusted posture optimization objective function. The wrist perturbation rotation parameter includes the palm rotation matrix. Obtain the joint angles of the dexterous hand and the joint angle boundaries corresponding to the joint angles; determine the joint angle constraint parameters based on the deviation between the joint angles and the joint angle boundaries; adjust the joint angle constraint terms in the posture optimization objective function based on the joint angle constraint parameters to obtain the adjusted posture optimization objective function; Based on the adjusted posture optimization objective function, the target grasping posture for the dexterous hand to grasp the target object is determined, including: Based on the adjusted posture optimization objective function, candidate grasping postures are determined, and the target object is simulated to grasp the object based on the candidate grasping postures, thereby obtaining the scoring results corresponding to the candidate grasping postures. The contact importance weight of each object point in the object point cloud corresponding to the target object is updated according to the scoring result, and object points with updated contact importance weights greater than a preset weight threshold are selected from the object point cloud to obtain the stable contact region corresponding to the next iteration. The target contact point is re-determined from the stable contact region corresponding to the next iteration round, and the posture optimization objective function is reconstructed based on the re-determined target contact point and the template contact point to determine the target grasping posture.

2. The method according to claim 1, characterized in that, Before determining the target grasping posture for the dexterous hand to grasp the target object based on the adjusted posture optimization objective function, the method further includes: The contact matching result is determined based on the target contact point and the template contact point; If the contact matching result indicates that a single contact point fails to cover the stable contact area of ​​the target object, then the contact point matching weight parameter is reduced to obtain the adjusted contact point matching weight parameter. The attitude optimization objective function is adjusted based on the adjusted contact point matching weight parameters to obtain the adjusted attitude optimization objective function.

3. The method according to claim 1, characterized in that, The step of determining the target grasping posture for the dexterous hand to grasp the target object based on the adjusted posture optimization objective function includes: In the current iteration, the posture parameters of the dexterous hand are solved according to the adjusted posture optimization objective function to obtain multiple candidate grasping postures corresponding to the current iteration. Simulate grasping the target object based on the multiple candidate grasping postures, and obtain simulation results corresponding to the multiple candidate grasping postures respectively; Based on the simulation results, the target contact point is updated, and the attitude optimization objective function is reconstructed based on the updated target contact point and the template contact point until the iteration termination condition is met. When the iteration termination condition is met, the target grasping posture is determined based on multiple candidate grasping postures corresponding to the termination iteration round.

4. The method according to claim 3, characterized in that, When the iteration termination condition is met, determining the target grasping posture based on multiple candidate grasping postures corresponding to the termination iteration round includes: When the iteration termination condition is met, obtain the termination score results corresponding to the multiple candidate grasping postures corresponding to the termination iteration round. The candidate grasping posture with the highest termination score is determined as the target grasping posture.

5. The method according to claim 1, characterized in that, The acquisition of the target contact point on the target object includes: Obtain the object mesh model corresponding to the target object; Surface sampling is performed on the object mesh model to obtain the object point cloud corresponding to the target object, and the object point cloud includes multiple object points; Based on the geometric attribute information corresponding to the object point cloud, a stable contact region is determined from the object point cloud; Select the target contact point on the target object from among multiple object points in the stable contact area.

6. The method according to claim 1, characterized in that, The step of obtaining the template contact point corresponding to the dexterous hand includes: Obtain the structural model corresponding to the dexterous hand; Based on the structural model, candidate contact areas for contacting the target object are determined from the dexterous hand; Based on the contact attribute information of each contact point in the candidate contact area, the template contact point corresponding to the dexterous hand is selected from multiple contact points in the candidate contact area.

7. A device for generating a grasping posture, characterized in that, include: The acquisition unit is used to acquire target contact points on a target object and acquire template contact points corresponding to a dexterous hand. The target contact points are located within a stable contact area of ​​the target object, and the template contact points are included in a plurality of contact points in a candidate contact area. The candidate contact area includes at least one of the fingertip area and the fingertip area of ​​the dexterous hand. The construction unit is used to construct an attitude optimization objective function for the dexterous hand grasping the target object based on the target contact point and the template contact point; An adjustment unit is used to perform inverse kinematics solution on the robotic arm to which the dexterous hand belongs based on the target contact point, so as to obtain the wrist reachability result of the robotic arm; If the wrist reachability result indicates that the robotic arm's wrist cannot reach the target contact point, then the wrist perturbation rotation amount corresponding to the dexterous hand is calculated based on the object's principal axis and a preset perturbation angle range. Based on the wrist perturbation rotation amount, the wrist perturbation rotation parameter in the posture optimization objective function is adjusted to obtain the adjusted posture optimization objective function. The wrist perturbation rotation parameter includes the palm rotation matrix. The joint angles of the dexterous hand and the joint angle boundaries corresponding to the joint angles are obtained. Based on the deviation between the joint angles and the joint angle boundaries, joint angle constraint parameters are determined. Based on the joint angle constraint parameters, the joint angle constraint term in the posture optimization objective function is adjusted to obtain the adjusted posture optimization objective function. The processing unit is configured to determine the target grasping posture for the dexterous hand to grasp the target object based on the adjusted posture optimization objective function, including: Based on the adjusted posture optimization objective function, candidate grasping postures are determined, and the target object is simulated to grasp the object based on the candidate grasping postures, thereby obtaining the scoring results corresponding to the candidate grasping postures. The contact importance weight of each object point in the object point cloud corresponding to the target object is updated according to the scoring result, and object points with updated contact importance weights greater than a preset weight threshold are selected from the object point cloud to obtain the stable contact region corresponding to the next iteration. The target contact point is re-determined from the stable contact region corresponding to the next iteration round, and the posture optimization objective function is reconstructed based on the re-determined target contact point and the template contact point to determine the target grasping posture.

8. A device for generating a grasping posture, characterized in that, include: Memory, which stores computer-readable instructions; A processor reads computer-readable instructions stored in memory to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 6.

10. A computer program product comprising computer-readable instructions, characterized in that, When the computer-readable instructions are executed by a processor, they implement the method as described in any one of claims 1 to 6.

Images