A method and system for estimating the pose of an object in a robot's hand

Abstract

The application discloses a robot palm-in-object pose estimation method and system, and belongs to the technical field of robot tactile perception. The method is as follows: obtaining a plurality of standard force value data, an effective contact point set, a target zero force set, an object initial pose, a target prediction particle set, an enhanced particle observation log-likelihood value set and a pose estimation index based on a robot full palm and a robot hand joint state; if the pose estimation index does not satisfy a preset condition, obtaining an object updated pose as the object initial pose, obtaining the enhanced particle observation log-likelihood value set based on the object initial pose again, and until the pose estimation index satisfies the preset condition, then determining an object target pose mode based on the above data. The robot palm-in-object pose estimation method provided by the application can realize quick and stable estimation of the palm-in-object pose only by relying on the robot full palm tactile information and the hand joint state.

Description

Technical Field

[0001] This invention relates to the field of robot tactile perception technology, and in particular to a method and system for estimating the pose of an object in the hand of a robot. Background Technology

[0002] Currently, a certain research foundation has been established for pose estimation technology for objects within the robotic hand. Among them, the technologies closest to practical application needs mainly fall into three categories: First, collecting relevant data through vision, depth cameras, or external motion capture systems to track the pose of objects within the hand; Second, using traditional geometric optimization or ICP-type methods to fit the pose of a small number of contact points with the object model to achieve pose estimation; Third, employing data-driven or deep learning methods to directly regress the object's pose or pose increment based on a large number of tactile samples to complete the pose calculation.

[0003] Under the current technological background, although various implementation paths have been developed, there are still many shortcomings in practical application scenarios: First, vision or motion capture solutions are easily affected by severe occlusion from fingers, palm, and the object itself when operating in the palm, making it difficult to provide a stable and continuous true pose, thus affecting the accuracy of pose estimation; Second, traditional geometric optimization methods are prone to convergence to incorrect solutions, mode collapse, or insufficient real-time performance when tactile contact points are sparse, force directions are complex, and noise is high, failing to meet the real-time and reliability requirements of practical operations; Third, for symmetrical objects such as cylinders and prisms, existing methods often cannot stably represent... The naturally occurring multimodal pose solutions are prone to averaging or mode jumps in pose estimation results, reducing estimation accuracy. In addition, many methods lack explicit constraints on physical feasibility such as friction cones, zero-force constraints, static equilibrium, and hand-object penetration, resulting in inconsistencies between pose estimation results and real contact relationships, and a disconnect from actual scenarios. Finally, learning-based methods usually rely on large-scale training data with ground truth values, but in hand-occluded scenarios, high-quality pose ground truth values ​​are difficult to collect. Furthermore, these methods have limitations in generalization ability across objects, materials, hand shapes, and haptic arrays, making it difficult to adapt to diverse real-world application scenarios. Summary of the Invention

[0004] The present invention aims to provide a method and system for estimating the pose of an object in the palm of a robot, so as to solve the above-mentioned technical problems, avoid the inability to continuously and stably obtain the pose of the object due to occlusion caused by palm operation, and realize the fast and stable estimation of the pose of the object in the palm by relying solely on the tactile information of the robot's entire palm and the state of the hand joints.

[0005] To address the aforementioned technical problems, this invention provides a method for estimating the pose of an object within the hand of a robot, comprising: Based on the robot's entire hand and the state of the robot's hand joints, several standard force values, effective contact point sets, and target zero force sets are obtained. The initial pose of the object is obtained based on the effective contact point sets, and the target predicted particle set is obtained based on the initial pose of the object. Based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, an enhanced particle observation log likelihood value set is obtained; Pose estimation metrics are obtained based on the target prediction particle set and the enhanced particle observation log likelihood value set. If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object. The updated pose of the object is then used as the initial pose of the object. The target prediction particle set is then re-obtained based on the initial pose of the object. The enhanced particle observation log-likelihood value set is then re-obtained based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions. Finally, the target pose pattern of the object is determined based on the enhanced particle observation log-likelihood value set and the target prediction particle set.

[0006] In the above scheme, tactile contact information that allows effective contact between the robot's entire hand and the object can be filtered out by analyzing the robot's hand joint states. Non-contact areas are marked, resulting in several standard force values, a set of effective contact points, and a set of target zero forces. Then, the initial pose of the object can be obtained from the set of effective contact points, generating a target prediction particle set, providing basic observation data and initial pose basis for subsequent pose estimation. Next, an enhanced particle observation log-likelihood set is obtained using the target prediction particle set, the target zero force set, the effective contact point set, and several effective force values. This set measures the degree of matching between each particle and the actual contact situation of the robot's hand, improving the accuracy of object pose estimation. Finally, a pose estimation index is calculated using the target prediction particle set and the enhanced particle observation log-likelihood set. This index determines the reliability of the current object pose estimation result. If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object can be fine-tuned, and the estimation of the object pose pattern can be continuously optimized, avoiding the inability to continuously and stably obtain the object pose due to occlusion caused by in-hand operations. Finally, when the pose estimation index meets the preset pose qualification conditions, the target pose pattern of the object is determined, so as to realize the fast and stable estimation of the pose of the object in the palm by relying only on the tactile information of the robot's whole palm and the state of the hand joints.

[0007] Furthermore, the process of acquiring several effective force value data, effective contact point set, and target zero force set based on the robot's entire hand and robot hand joint states, and acquiring the object's initial pose based on the effective contact point set, to acquire the target predicted particle set based on the object's initial pose, includes: Based on the robot's entire hand, several standard force values ​​were obtained, and based on the robot's hand joint state, several standard force values, and a preset tactile unit coordinate library, several contact candidate point coordinate data were obtained. Based on a preset rejection force value threshold, several standard force value data are filtered to obtain several effective force value data; Based on several effective power value data, obtain the number of activations in several neighborhoods corresponding to several effective power value data; Based on preset sensor partitioning rules, several effective force value data are divided into several sensor partition subsets; Based on several effective force value data and several neighborhood activation numbers, a comprehensive score for several force values ​​corresponding to several effective force value data is calculated. Based on the comprehensive score of several force values ​​and the preset first retention quantity, several effective force value data in several sensor partition subsets are filtered to obtain several effective sensor partition subsets and an initial zero force set; Based on preset sorting conditions, preset second retention quantity, several effective force value data from several effective sensor partition subsets and initial zero force set, obtain target zero force set and target force value data, and obtain effective contact point set based on the target force value data and several contact candidate point coordinate data; The initial pose of the object is obtained based on the set of effective contact points, and the set of object localization particles is obtained based on the initial pose of the object. The set of object localization particles is randomly perturbed to obtain the set of target prediction particles.

[0008] In the above scheme, several standard force values ​​and several contact candidate point coordinates are obtained by using the robot's entire palm, the robot's hand joint states, and a preset tactile unit coordinate library, providing basic data for subsequent screening. Next, the standard force values ​​are screened using a preset rejection threshold to eliminate noise and invalid force values, resulting in several effective force values. Then, several neighborhood activation counts are obtained from these effective force values, which can measure the density of the contact area between the object and the robot's palm, facilitating the priority retention of core contact points with more concentrated and stable contact. Subsequently, the effective force values ​​are divided into several sensor partition subsets using preset sensor partitioning rules, preventing a single sensor partition from dominating the estimation results. Finally, a comprehensive force score is calculated using the effective force values ​​and the number of neighborhood activations, objectively assessing the importance of each effective force value. The comprehensive force score, along with a preset first retention count, is used to screen each sensor partition subset, obtaining effective sensor partition subsets and an initial zero-force set to ensure a balanced distribution of contact points. Next, by using preset sorting conditions, a preset second retention quantity, several effective force value data from several effective sensor partition subsets, and an initial zero force set, the target zero force set and target force value data are obtained. Based on the target force value data and several contact candidate point coordinate data, an effective contact point set is obtained, which can filter out tactile contact information that generates effective contact with the object within the robot's entire palm and mark non-contact areas. Finally, the initial pose of the object is obtained through the effective contact point set, and then the target prediction particle set is obtained through the initial pose of the object, which can provide reliable initial particle samples for subsequent object pose estimation.

[0009] Further, the step of obtaining the initial pose of the object based on the effective contact point set, and obtaining the object localization particle set based on the initial pose of the object, and then randomly perturbing the object localization particle set to obtain the target prediction particle set, includes: Calculate sublinear weights for several contact points based on the effective set of contact points; The centroid and average force direction of the contact points are calculated based on the effective set of contact points and the sublinear weights of several contact points. A coarse initial pose is obtained based on the centroid of the contact point and the average force direction; Fine-tune the coarse initial pose based on a preset offset to obtain the object's initial pose. Based on the initial pose of the object, a set of object positioning particles is obtained. The set of object positioning particles is randomly perturbed based on a preset zero-mean three-dimensional Gaussian distribution function and a preset zero-mean three-dimensional angular axis perturbation distribution function to obtain a set of target predicted particle translations. Pose mapping is performed on the target predicted particle translation set to obtain the target predicted particle pose set; The target predicted particle set is obtained based on the target predicted particle translation set and the target predicted particle posture set.

[0010] In the above scheme, calculating several sublinear weights for contact points using the effective contact point set can suppress the excessive influence of a single abnormally large contact force, making the acquisition of the object's initial pose more reliable. Next, using the effective contact point set and several sublinear weights, the centroid of the contact points and the average force direction can be calculated, obtaining the core position and force direction of the contact between the object and the robot's hand. Then, a coarse initial pose is obtained using the centroid of the contact points and the average force direction, and fine-tuned based on a preset offset to obtain the object's initial pose, improving its accuracy. Next, a set of object positioning particles is obtained from the object's initial pose, and then randomly perturbed according to a preset zero-mean three-dimensional Gaussian distribution function and a preset zero-mean three-dimensional angular axis perturbation distribution function to generate candidate samples for pose translation directions, obtaining the target predicted particle translation set. Finally, by performing pose mapping on the target predicted particle translation set, a set of target predicted particle postures can be obtained, generating candidate samples for pose rotation directions. Finally, the target predicted particle translation set and the target predicted particle pose set can provide complete candidate particles for subsequent object pose estimation, thus obtaining the target predicted particle set.

[0011] Furthermore, the step of obtaining the enhanced particle observation log-likelihood value set based on the target predicted particle set, the target zero-force set, the effective contact point set, and several effective force value data includes: Based on the target predicted particle set and the pre-acquired 3D model of the object, the surface data of the object is determined; Based on the target prediction particle set, target zero force set, effective contact point set, and several effective force value data, calculate the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set; The enhanced particle observation log-likelihood set is obtained based on the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set.

[0012] In the above scheme, the object surface data is determined by predicting the target particle set and pre-acquiring the object's 3D model, providing a geometric reference for subsequent particle scoring. Next, the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set are calculated using the target predicted particle set, target zero-force set, effective contact point set, and several effective force value data, enabling the evaluation of particle rationality from multiple dimensions. Then, the enhanced particle observation log-likelihood value set is obtained using the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set, quantifying the degree of matching between each particle and the actual contact situation.

[0013] Furthermore, the calculation of the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set based on the target predicted particle set, target zero-force set, effective contact point set, and several effective force value data includes: Based on the target prediction particle set, the effective contact point set, and the object surface data, several effective contact distance residuals are calculated, and based on several effective contact distance residuals, a preset effective contact point weight set, and a preset distance residual standardization coefficient, a geometric consistency score set is calculated. Based on the target prediction particle set and the effective contact point set, determine the surface normals of several objects corresponding to several effective contact points, and calculate the direction consistency score set based on the effective contact point set, the preset effective contact point weight set and the surface normals of several objects. Based on the effective contact point set, obtain several normal force components and several tangential force components corresponding to the effective contact point set, and calculate the friction cone score set based on the preset effective contact point weight set, several normal force components, several tangential force components and preset friction cone violation standardization coefficient. Based on the target predicted particle set, the target zero force set, and the object surface data, calculate several zero force contact distance residuals, and calculate the zero force constraint score set based on the several zero force contact distance residuals, the preset zero force constraint standardization coefficient, and the preset safety constraint. The half-space constraint score set and static equilibrium score set are calculated based on the target prediction particle set, several effective force value data and effective contact point set.

[0014] In the above scheme, the effective contact distance residual is calculated using the target predicted particle set, the effective contact point set, and the object surface data. A geometric consistency score set is then calculated by combining the effective contact distance residual, a preset effective contact point weight set, and a preset distance residual standardization coefficient. This geometric consistency score can determine whether the particle pose matches the effective contact point geometrically. Next, the object surface normal corresponding to the effective contact point is determined using the target predicted particle set and the effective contact point set. A direction consistency score set is then calculated by combining the effective contact point set, the preset effective contact point weight set, and the object surface normal. This direction consistency score can determine whether the contact force direction is consistent with the object surface normal direction. Finally, several normal force components and several tangential force components are obtained from the effective contact point set. A friction cone score set is then calculated by combining the preset effective contact point weight set, several normal force components, several tangential force components, and a preset friction cone violation standardization coefficient. This friction cone score can determine whether the contact state conforms to the laws of physical friction. Subsequently, several zero-force contact distance residuals are calculated using the target prediction particle set, the target zero-force set, and object surface data. These residuals, along with preset zero-force constraint standardization coefficients and preset safety constraints, are then used to calculate a zero-force constraint score set. This score allows for the assessment of whether there is a conflict between the non-contact region and the particle hypothesis, improving the physical plausibility of the object's pose estimation. Finally, a half-space constraint score set and a static equilibrium score set are calculated using the target prediction particle set, several effective force value data, and the effective contact point set. The half-space constraint score determines whether the object's spatial position is reasonable, and the static equilibrium score determines whether the contact mechanics meets the equilibrium requirements.

[0015] Furthermore, the calculation of the half-space constraint score set and the static equilibrium score set based on the target predicted particle set, several effective force value data, and the effective contact point set includes: Based on the target prediction particle set, obtain the positions of several object reference points corresponding to several target prediction particles; Several sensitive out-of-plane normals are obtained based on several effective force values, and the half-space constraint score set is calculated based on several sensitive out-of-plane normals, several object reference point positions, and preset half-space constraint standardization coefficients. Several relative moments are calculated based on the effective contact point set and the coordinate positions of the preset reference points. The static equilibrium score set is then calculated based on the effective contact point set, the preset equivalent external load, the preset resultant force residual standardization coefficient, the preset resultant moment residual standardization coefficient, the preset equivalent moment term, and several relative moments.

[0016] In the above scheme, the positions of several object reference points corresponding to several target predicted particles are obtained through the target predicted particle set, which provides a positional basis for calculating the half-space constraint score. Next, several out-of-plane normals are obtained through several effective force value data. Based on these out-of-plane normals, the positions of several object reference points, and preset half-space constraint standardization coefficients, a half-space constraint score set is calculated. This score can be used to determine whether the object is within the effective sensing range of the sensor. Then, several relative moments are calculated using the effective contact point set and preset reference point coordinates, providing basic data for mechanical equilibrium assessment. Finally, a static equilibrium score set is calculated using the effective contact point set, preset equivalent external load, preset resultant force residual standardization coefficient, preset resultant moment residual standardization coefficient, preset equivalent moment term, and several relative moments. This score can be used to determine whether the contact mechanics under the particle assumption meets the quasi-static equilibrium requirements.

[0017] Furthermore, the step of obtaining the enhanced particle observation log-likelihood set based on the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set includes: Based on the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set, calculate the log likelihood values ​​of several real-time particle observations corresponding to several target predicted particles; The log-likelihood values ​​of pre-acquired historical particle observations and several real-time particle observation log-likelihood values ​​are fused to obtain an enhanced set of particle observation log-likelihood values.

[0018] In the above scheme, several real-time particle observation log-likelihood values ​​corresponding to several target predicted particles are calculated using geometric consistency score sets, directional consistency score sets, friction cone score sets, zero-force constraint score sets, half-space constraint score sets, and static equilibrium score sets. This allows for the quantification of the matching degree between each particle and the tactile observation. Then, by fusing a pre-acquired historical particle observation log-likelihood value set with several real-time particle observation log-likelihood values, the stability and anti-interference ability of object pose estimation can be improved, resulting in an enhanced particle observation log-likelihood value set.

[0019] Further, if the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object, and the updated pose of the object is used as the initial pose of the object. The target predicted particle set is then re-obtained based on the initial pose of the object. The enhanced particle observation log-likelihood value set is then re-obtained based on the target predicted particle set, the target zero-force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions. The target pose mode of the object is then determined based on the enhanced particle observation log-likelihood value set and the target predicted particle set, including: If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object. The updated pose of the object is then used as the initial pose of the object to re-obtain the target prediction particle set based on the initial pose of the object. The enhanced particle observation log likelihood value set is then re-obtained based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data until the pose estimation index meets the preset pose qualification conditions. Then, the target prediction particle set is normalized in the number domain based on the enhanced particle observation log likelihood value set to obtain the target prediction particle weight set. The number of valid samples is calculated based on the target predicted particle weight set. If the number of valid samples is less than a preset sample threshold, a set of retained particles is obtained based on the target predicted particle weight set and the target predicted particle set. Based on a preset object pose parameter library, the reserved particle set is divided into modes to obtain several object pose modes and several reserved particles corresponding to several object pose modes. The target pose pattern of an object is determined based on several retained particles corresponding to several object pose patterns and several object pose patterns.

[0020] In the above scheme, when the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned and the matching degree between particles and observations is continuously optimized until the pose estimation index meets the preset pose qualification conditions and the surface estimation result is stable and reliable. At this time, the target prediction particle set is normalized in the number domain to obtain the target prediction particle weight set, which can clarify the reliability of each particle. Then, the number of effective samples is calculated through the target prediction particle weight set. The number of effective samples can be used to determine whether there is a degradation situation of excessive concentration of particles. If the number of effective samples is less than the preset sample threshold, high-reliability particles are retained and invalid particles are removed based on the target prediction particle weight set and the target prediction particle set, resulting in a retained particle set. Next, the retained particle set is divided into patterns through a preset object pose parameter library to obtain multiple object pose patterns and several particles corresponding to several object pose patterns. Finally, through several object pose patterns and the corresponding several retained particles, the target pose pattern of the object can be determined, thereby obtaining the final target pose pattern of the object in the robot's hand.

[0021] This invention provides a pose estimation system for an object in the hand of a robot, comprising an initial pose generation module, a particle observation log-likelihood calculation module, a pose estimation index evaluation module, and a target pose determination module, specifically: The initial pose generation module is used to obtain several effective force value data, effective contact point set and target zero force set based on the robot's full palm and robot hand joint state, and obtain the object's initial pose based on the effective contact point set, so as to obtain the target predicted particle set based on the object's initial pose. The particle observation log likelihood calculation module is used to obtain an enhanced particle observation log likelihood value set based on the target predicted particle set, the target zero force set, the effective contact point set, and several effective force value data. The pose estimation index evaluation module is used to obtain pose estimation indexes based on the target prediction particle set and the enhanced particle observation log likelihood value set. The target pose determination module is used to fine-tune the initial pose of the object if the pose estimation index does not meet the preset pose qualification conditions, obtain the updated pose of the object, and use the updated pose of the object as the initial pose of the object, so as to re-obtain the target prediction particle set based on the initial pose of the object, and re-obtain the enhanced particle observation log likelihood value set based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions, and determine the target pose pattern of the object based on the enhanced particle observation log likelihood value set and the target prediction particle set.

[0022] This invention provides a pose estimation system for objects within a robot's palm. In practical applications, only an initial pose generation module is needed. By analyzing the robot's entire palm and the states of its hand joints, tactile contact information that creates effective contact with the object within the robot's palm can be filtered out, and non-contact areas are marked, resulting in several standard force values, a set of effective contact points, and a set of target zero forces. Then, the initial pose of the object can be obtained from the set of effective contact points, and a target prediction particle set can be generated, providing basic observation data and initial pose basis for subsequent pose estimation. Next, a particle observation log-likelihood calculation module is used to obtain an enhanced particle observation log-likelihood value set from the target prediction particle set, the target zero force set, the set of effective contact points, and several effective force values. This set measures the degree of matching between each particle and the actual contact situation of the robot's palm, improving the accuracy of object pose estimation. Then, a pose estimation index evaluation module is employed. This module calculates the pose estimation index using the target prediction particle set and the augmented particle observation log-likelihood set. The index determines the reliability of the current object pose estimation result. If the pose estimation index does not meet the preset pose qualification conditions, the module can fine-tune the initial object pose and continuously optimize the estimation of the object's pose pattern, avoiding the inability to continuously and stably acquire the object pose due to occlusion caused by in-palm operations. Finally, a target pose determination module is used. When the pose estimation index meets the preset pose qualification conditions, the module determines the object's target pose pattern, enabling rapid and stable estimation of the in-palm object pose solely based on the robot's full-palm tactile information and hand joint states.

[0023] Furthermore, the initial pose generation module is used to acquire several effective force value data, effective contact point set, and target zero force set based on the robot's entire hand and robot hand joint states, and to acquire the object's initial pose based on the effective contact point set, so as to acquire the target predicted particle set based on the object's initial pose, including: Based on the robot's entire hand, several standard force values ​​were obtained, and based on the robot's hand joint state, several standard force values, and a preset tactile unit coordinate library, several contact candidate point coordinate data were obtained. Based on a preset rejection force value threshold, several standard force value data are filtered to obtain several effective force value data; Based on several effective power value data, obtain the number of activations in several neighborhoods corresponding to several effective power value data; Based on preset sensor partitioning rules, several effective force value data are divided into several sensor partition subsets; Based on several effective force value data and several neighborhood activation numbers, a comprehensive score for several force values ​​corresponding to several effective force value data is calculated. Based on the comprehensive score of several force values ​​and the preset first retention quantity, several effective force value data in several sensor partition subsets are filtered to obtain several effective sensor partition subsets and an initial zero force set; Based on preset sorting conditions, preset second retention quantity, several effective force value data from several effective sensor partition subsets and initial zero force set, obtain target zero force set and target force value data, and obtain effective contact point set based on the target force value data and several contact candidate point coordinate data; The initial pose of the object is obtained based on the set of effective contact points, and the set of object localization particles is obtained based on the initial pose of the object. The set of object localization particles is randomly perturbed to obtain the set of target prediction particles.

[0024] In the above scheme, several standard force values ​​and several contact candidate point coordinates are obtained by using the robot's entire palm, the robot's hand joint states, and a preset tactile unit coordinate library, providing basic data for subsequent screening. Next, the standard force values ​​are screened using a preset rejection threshold to eliminate noise and invalid force values, resulting in several effective force values. Then, several neighborhood activation counts are obtained from these effective force values, which can measure the density of the contact area between the object and the robot's palm, facilitating the priority retention of core contact points with more concentrated and stable contact. Subsequently, the effective force values ​​are divided into several sensor partition subsets using preset sensor partitioning rules, preventing a single sensor partition from dominating the estimation results. Finally, a comprehensive force score is calculated using the effective force values ​​and the number of neighborhood activations, objectively assessing the importance of each effective force value. The comprehensive force score, along with a preset first retention count, is used to screen each sensor partition subset, obtaining effective sensor partition subsets and an initial zero-force set to ensure a balanced distribution of contact points. Next, by using preset sorting conditions, a preset second retention quantity, several effective force value data from several effective sensor partition subsets, and an initial zero force set, the target zero force set and target force value data are obtained. Based on the target force value data and several contact candidate point coordinate data, an effective contact point set is obtained, which can filter out tactile contact information that generates effective contact with the object within the robot's entire palm and mark non-contact areas. Finally, the initial pose of the object is obtained through the effective contact point set, and then the target prediction particle set is obtained through the initial pose of the object, which can provide reliable initial particle samples for subsequent object pose estimation. Attached Figure Description

[0025] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating a method for estimating the pose of an object in the palm of a robot, as provided in an embodiment of the present invention; Figure 2 This is an architecture diagram of a pose estimation system for an object in the palm of a robot, provided as an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0029] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0030] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0031] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0032] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0033] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0034] See Figure 1 To avoid the inability to continuously and stably acquire the pose of an object due to occlusion caused by in-palm operations, and to achieve fast and stable estimation of the pose of an object in the robot's palm solely based on the robot's full palm tactile information and hand joint states, this embodiment provides a method for estimating the pose of an object in the robot's palm. The flowchart of this method can be found in [link to flowchart]. Figure 1 ,include: Step S1: Based on the robot's entire hand and the state of the robot's hand joints, obtain several standard force values, effective contact point sets, and target zero force sets, and obtain the object's initial pose based on the effective contact point sets, and obtain the target predicted particle set based on the object's initial pose; Step S2: Obtain the enhanced particle observation log-likelihood set based on the target predicted particle set, the target zero force set, the effective contact point set, and several effective force value data; Step S3: Obtain pose estimation indices based on the target prediction particle set and the enhanced particle observation log likelihood set; Step S4: If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object, and the updated pose of the object is used as the initial pose of the object. The target prediction particle set is then obtained again based on the initial pose of the object. The enhanced particle observation log likelihood value set is then obtained again based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions. The target pose mode of the object is then determined based on the enhanced particle observation log likelihood value set and the target prediction particle set.

[0035] In this embodiment, tactile contact information that effectively contacts the object within the robot's entire palm and the robot's hand joint states can be filtered out, and non-contact areas are marked, resulting in several standard force value data, a set of effective contact points, and a set of target zero forces. Then, the initial pose of the object can be obtained from the set of effective contact points, and a target prediction particle set can be generated, providing basic observation data and initial pose basis for subsequent pose estimation. Next, an enhanced particle observation log-likelihood value set is obtained using the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data. This can measure the degree of matching between each particle and the actual contact situation of the robot's palm, improving the accuracy of object pose estimation. Then, a pose estimation index is calculated using the target prediction particle set and the enhanced particle observation log-likelihood value set. This index can be used to determine whether the current object pose estimation result is reliable. When the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object can be fine-tuned, and the estimation of the object pose pattern can be continuously optimized, avoiding the inability to continuously and stably obtain the object pose due to occlusion caused by palm operation. Finally, when the pose estimation index meets the preset pose qualification conditions, the target pose pattern of the object is determined, so as to realize the fast and stable estimation of the pose of the object in the palm by relying only on the tactile information of the robot's whole palm and the state of the hand joints.

[0036] Furthermore, the process of acquiring several effective force value data, effective contact point set, and target zero force set based on the robot's entire hand and robot hand joint states, and acquiring the object's initial pose based on the effective contact point set, to acquire the target predicted particle set based on the object's initial pose, includes: Based on the robot's entire hand, several standard force values ​​were obtained, and based on the robot's hand joint state, several standard force values, and a preset tactile unit coordinate library, several contact candidate point coordinate data were obtained. Based on a preset rejection force value threshold, several standard force value data are filtered to obtain several effective force value data; Based on several effective power value data, obtain the number of activations in several neighborhoods corresponding to several effective power value data; Based on preset sensor partitioning rules, several effective force value data are divided into several sensor partition subsets; Based on several effective force value data and several neighborhood activation numbers, a comprehensive score for several force values ​​corresponding to several effective force value data is calculated. Based on the comprehensive score of several force values ​​and the preset first retention quantity, several effective force value data in several sensor partition subsets are filtered to obtain several effective sensor partition subsets and an initial zero force set; Based on preset sorting conditions, preset second retention quantity, several effective force value data from several effective sensor partition subsets and initial zero force set, obtain target zero force set and target force value data, and obtain effective contact point set based on the target force value data and several contact candidate point coordinate data; The initial pose of the object is obtained based on the set of effective contact points, and the set of object localization particles is obtained based on the initial pose of the object. The set of object localization particles is randomly perturbed to obtain the set of target prediction particles.

[0037] In this embodiment, multiple full-palm tactile sensor arrays on the robot's entire palm are read via a UART (Universal Asynchronous Receiver / Transmitter) serial port at a baud rate of 921600. Several readings from the sensor arrays are converted into raw force values ​​using LSB (Low Substances of Sensors) coefficients. The readings recorded when the full-palm tactile sensor arrays are activated or manually triggered are used as a preset zero-point baseline. This baseline is used to compensate for zero-point drift in the raw force values, obtaining several standard force values. Then, based on the robot's hand joint state and the standard force values, several candidate contact point coordinates are loaded from a preset tactile unit coordinate library, providing foundational data for subsequent filtering. In this embodiment, the standard force values ​​can also be visualized, mapped to colors, and the resultant force of each sensor mapped to a resultant force arrow. Next, a preset rejection threshold is used to filter the standard force values, eliminating noise and invalid force values, resulting in several valid force values. Then, by obtaining several effective force value data points, the corresponding number of neighborhood activations is obtained. These activations can be used to measure the density of the contact area between the object and the robot's hand, facilitating the priority retention of core contact points with more concentrated and stable contact. Subsequently, the effective force value data points are divided into several sensor partition subsets using preset sensor partitioning rules, preventing a single sensor partition from dominating the estimation results. Next, a comprehensive score for several force values ​​is calculated using the effective force value data points and the number of neighborhood activations, specifically: ; in, The comprehensive score for the force value of the i-th valid force value data. , and These are the preset first comprehensive coefficient, the preset second comprehensive coefficient, and the preset third comprehensive coefficient, respectively. For the i-th effective force value, For the normal component of the i-th effective force value data; The out-of-plane normal for the i-th effective force value data can be directly obtained from several effective force value data. This represents the number of neighborhood activations corresponding to the i-th effective force value data, used to prioritize the retention of the center contact point in the contact area. Then, within each partition, a maximum of a preset first retention number of effective force value data is retained to avoid a single sensor partition dominating the estimation. The resulting comprehensive force value score objectively assesses the importance of each effective force value data. The comprehensive force value scores are used to sort the effective force values ​​in each sensor partition subset according to a preset sorting condition, and a preset first retention number of effective force value data is retained for each sensor partition subset, resulting in an effective sensor partition subset. Effective force value data not included in the effective sensor partition subset is used as the initial zero force, thus obtaining an initial zero force set to ensure a balanced distribution of contact points. The preset sorting condition is descending order. Next, all effective force value data in the effective sensor partition subsets are sorted again according to the comprehensive force value scores and the preset sorting condition. The first preset second retention number of effective force value data is selected as the target force value data, and effective force value data not identified as target force value data is included in the initial zero force set, resulting in the target zero force set. Subsequently, based on the target force data and the coordinate data of several candidate contact points, a set of effective contact points is obtained. This allows for the filtering of tactile contact information that enables effective contact between the robot's entire palm and the object, and the marking of non-contact areas. Finally, the initial pose of the object is obtained from the set of effective contact points, and then the target prediction particle set is obtained from the initial pose of the object, which can provide reliable initial particle samples for subsequent pose estimation of the object.

[0038] Further, the step of obtaining the initial pose of the object based on the effective contact point set, and obtaining the object localization particle set based on the initial pose of the object, and then randomly perturbing the object localization particle set to obtain the target prediction particle set, includes: Calculate sublinear weights for several contact points based on the effective set of contact points; The centroid and average force direction of the contact points are calculated based on the effective set of contact points and the sublinear weights of several contact points. A coarse initial pose is obtained based on the centroid of the contact point and the average force direction; Fine-tune the coarse initial pose based on a preset offset to obtain the object's initial pose. Based on the initial pose of the object, a set of object positioning particles is obtained. The set of object positioning particles is randomly perturbed based on a preset zero-mean three-dimensional Gaussian distribution function and a preset zero-mean three-dimensional angular axis perturbation distribution function to obtain a set of target predicted particle translations. Pose mapping is performed on the target predicted particle translation set to obtain the target predicted particle pose set; The target predicted particle set is obtained based on the target predicted particle translation set and the target predicted particle posture set.

[0039] In this embodiment, by calculating the sublinear weights of several contact points using an effective set of contact points and a preset sublinear function, the excessive influence of a single abnormally large contact force can be suppressed, making the acquisition of the object's initial pose more reliable. The preset sublinear function can be in the form of a power function or a logarithmic function of the contact force amplitude, with its exponent or function parameters being preset algorithm parameters. It satisfies the condition that the larger the contact force, the larger the weight, but the weight growth rate is lower than the linear growth rate, thus avoiding a single abnormally large contact force occupying too high a proportion in the coarse initialization. Next, the centroid of the contact points and the average force direction can be calculated using the effective set of contact points and the sublinear weights of several contact points, obtaining the core position and force direction of the contact between the object and the robot's palm. Then, the coarse initial pose is obtained using the centroid of the contact points and the average force direction. Since tactile force is usually generated by the object pressing against the sensor surface, fine-tuning the coarse initial pose using a preset offset can obtain the object's initial pose, making it more likely that the object's initial pose is located outside the sensor rather than inside the palm, thereby improving its geometric feasibility and enhancing the accuracy of the object's initial pose.

[0040] The fine-tuning of the coarse initial pose can be performed using point cloud registration, signed distance residual minimization, or other existing local geometry optimization methods to make the initial pose of the object more closely resemble the current tactile observation. Next, a set of object positioning particles is obtained from the initial pose. This set is then randomly perturbed according to a preset zero-mean three-dimensional Gaussian distribution function and a preset zero-mean three-dimensional angular axis perturbation distribution function, generating candidate samples for the pose translation direction and obtaining the target prediction particle translation set. Subsequently, the set of object positioning particles is obtained from the initial pose, and the state of the j-th object positioning particle at time t is... ,in To determine the translation of the particle for the j-th object, The quaternion for the j-th object's localized particle can be obtained by mapping the pose of the j-th object's localized particle's translation. Next, the set of object-localized particles is randomly perturbed using a preset zero-mean 3D Gaussian distribution function and a preset zero-mean 3D angular axis perturbation distribution function to obtain the target predicted particle translation set, specifically: ,in, This represents the predicted target particle, This indicates a pose superposition operation. To describe the perturbation of the j-th particle at time t, , For translational disturbance, For rotational disturbance; The preset zero-mean three-dimensional Gaussian distribution function follows a zero-mean three-dimensional Gaussian distribution. ; The preset zero-mean three-dimensional angular axis perturbation distribution function follows a zero-mean three-dimensional angular axis perturbation distribution. .

[0041] Then, by performing pose mapping on the translation set of the target predicted particles, the quaternion of the target predicted particles is obtained. It can obtain the target predicted particle pose set from the current contact distribution principal direction, the object principal axis direction, or the preset symmetry axis direction, and generate candidate samples for pose rotation direction. Finally, the target predicted particle translation set and the target predicted particle pose set can provide complete candidate particles for subsequent object pose estimation, thus obtaining the target predicted particle set.

[0042] In this embodiment, the translational and rotational perturbations employ annealing or adaptive updates: when the number of effective samples is high for several consecutive frames, the number of effective contact points is stable, and the optimal particle score improves, the perturbation is reduced. and To improve convergence accuracy; when the number of effective contact points decreases sharply, the non-contact region contradicts the particle hypothesis, or the number of effective samples decreases, the convergence accuracy is increased. and To broaden the search scope.

[0043] Furthermore, the step of obtaining the enhanced particle observation log-likelihood value set based on the target predicted particle set, the target zero-force set, the effective contact point set, and several effective force value data includes: Based on the target predicted particle set and the pre-acquired 3D model of the object, the surface data of the object is determined; Based on the target prediction particle set, target zero force set, effective contact point set, and several effective force value data, calculate the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set; The enhanced particle observation log-likelihood set is obtained based on the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set.

[0044] In this embodiment, by using the target predicted particle set and the pre-acquired 3D object model, the 3D object model is transformed to the assumed pose of the target predicted particle set, and the object surface data is determined, providing a geometric reference for subsequent particle scoring. Next, using the target predicted particle set, target zero-force set, effective contact point set, and several effective force value data, geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set are calculated, enabling the evaluation of particle rationality from multiple dimensions. Then, using the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set, an enhanced particle observation log-likelihood value set is obtained, quantifying the degree of matching between each particle and the actual contact situation.

[0045] Furthermore, the calculation of the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set based on the target predicted particle set, target zero-force set, effective contact point set, and several effective force value data includes: Based on the target prediction particle set, the effective contact point set, and the object surface data, several effective contact distance residuals are calculated, and based on several effective contact distance residuals, a preset effective contact point weight set, and a preset distance residual standardization coefficient, a geometric consistency score set is calculated. Based on the target prediction particle set and the effective contact point set, determine the surface normals of several objects corresponding to several effective contact points, and calculate the direction consistency score set based on the effective contact point set, the preset effective contact point weight set and the surface normals of several objects. Based on the effective contact point set, obtain several normal force components and several tangential force components corresponding to the effective contact point set, and calculate the friction cone score set based on the preset effective contact point weight set, several normal force components, several tangential force components and preset friction cone violation standardization coefficient. Based on the target predicted particle set, the target zero force set, and the object surface data, calculate several zero force contact distance residuals, and calculate the zero force constraint score set based on the several zero force contact distance residuals, the preset zero force constraint standardization coefficient, and the preset safety constraint. The half-space constraint score set and static equilibrium score set are calculated based on the target prediction particle set, several effective force value data and effective contact point set.

[0046] In this embodiment, the effective contact distance residual is calculated using the target predicted particle set, the effective contact point set, and the object surface data. A geometric consistency score set is then calculated by combining the effective contact distance residual, a preset effective contact point weight set, and a preset distance residual standardization coefficient. This geometric consistency score allows for the determination of whether the target predicted particle pose matches the effective contact point geometrically. Specifically: ,in, The geometric consistency score for the predicted particle of the j-th target increases as the effective contact distance residual decreases, where m represents the total number of effective contact points. The effective contact force weight is the value corresponding to the i-th effective contact point. The effective contact distance residual from the i-th effective contact point to the object surface under the assumption of the j-th target predicted particle can be represented by a signed distance or the nearest point distance. The standardization coefficient for the preset distance residual is used. If a valid contact point is close to the object surface under the assumption of the target prediction particle, it means that the target prediction particle is more geometrically consistent with the current tactile observation and should receive a higher geometric consistency score; conversely, if the contact point is far from the object surface, it means that the target prediction particle assumption is inconsistent with the current observation and its geometric consistency score is reduced. The geometric consistency scores corresponding to several target prediction particles constitute the geometric consistency score set.

[0047] The process of obtaining the preset effective contact point weight set is as follows: ;in, The effective contact force weight is the value corresponding to the i-th effective contact point. To preset the maximum contact force threshold, To preset the reference contact force threshold, i.e. Follow The effective contact force is monotonically increased, and by using a preset reference contact force threshold, it can be normalized, allowing the larger and more stable effective contact force to account for a higher proportion in the likelihood calculation. Next, the surface normal corresponding to the effective contact point is determined using the target prediction particle set and the effective contact point set. Then, a direction consistency score set is calculated by combining the effective contact point set, the preset effective contact point weight set, and the surface normal. This direction consistency score can be used to determine whether the contact force direction is consistent with the surface normal direction. Specifically: ,in, The direction consistency score of the predicted particle for the j-th target is calculated; the direction of the unit force at the i-th effective contact point can be obtained from the effective contact force corresponding to the i-th effective contact point. The direction of the unit force corresponding to the i-th effective contact point. The surface normal of the object corresponding to the i-th effective contact point under the j-th target particle.

[0048] In this embodiment, if the direction of the unit force is opposite to the direction of the normal to the object's surface, the normal to the object's surface is flipped to avoid the angle being... A jump occurs nearby. If the direction of the unit force is more consistent with the direction of the normal to the object's surface, it indicates that the contact relationship corresponding to the predicted particle is more consistent with the actual compressive contact state, and the direction consistency score is higher; conversely, if the two deviate significantly, it indicates that the predicted particle is inconsistent with the current observation in the contact direction, and its direction consistency score decreases. The obtained direction consistency scores corresponding to several predicted particles constitute a direction consistency score set. Then, through the effective contact point set, several normal force components and several tangential force components are obtained, and combined with the preset effective contact point weight set, several normal force components, several tangential force components, and the preset friction cone violation normalization coefficient, the friction cone score set is calculated. The friction cone score can be used to determine whether the contact state conforms to the physical friction law, specifically: ,in, Predict the friction cone score for the particle of the j-th target; Let be the normal force component of the i-th effective contact point under the j-th target predicted particle, representing the force along the normal direction of the object surface; The preset friction coefficient; Let be the tangential force component of the i-th effective contact point under the j-th target predicted particle, and let represent the force component parallel to the object surface. The standardization coefficient for the pre-defined friction cone violation is set.

[0049] If the tangential force component does not exceed a preset tangential force threshold, the contact point is considered to satisfy the friction cone constraint; if the tangential force exceeds the preset tangential force threshold, it indicates that the contact state under the target predicted particle assumption violates frictional feasibility and should be penalized. The preset tangential force threshold can be the product of a preset friction coefficient and the normal force component. The friction cone score can be used to suppress target predicted particles that are geometrically close but do not mechanically satisfy the contact friction law, thereby improving the physical rationality of the object estimation results. The friction cone scores corresponding to several target predicted particles constitute a friction cone score set. Subsequently, several zero-force contact distance residuals are calculated using the target predicted particle set, the target zero-force set, and the object surface data. Combined with several zero-force contact distance residuals, the preset zero-force constraint standardization coefficient, and the preset safety constraint, a zero-force constraint score set is calculated. The zero-force constraint score can be used to determine whether there is a conflict between the non-contact area and the particle assumption, improving the physical rationality of the object pose estimation. Specifically: ,in, Predict the zero-force constraint score for the particle of the j-th target. Indicates the quantity of zero force at the target. This indicates a preset safety constraint, used to reserve a tolerance range for sensor noise, geometric calibration error, and local contact fluctuations for zero force on the target; To pre-set the zero-force constraint standardization coefficient, The residual of the zero-force contact distance between the k-th target and the object surface under the assumption of zero force for the j-th target predicting particle.

[0050] For any target zero force in the target zero force set, if, under the assumption of the j-th target predicted particle, its corresponding spatial position falls near the object surface or even enters the object's interior, it indicates that the target predicted particle assumption is inconsistent with the fact that no contact force was detected at that location, and a penalty should be imposed. Conversely, if the target zero force maintains a sufficient distance from the object surface, it indicates that the particle is compatible with the current zero force observation. When the distance from the target zero force to the object surface is greater than the aforementioned safety margin, it is considered consistent with the target predicted particle assumption. The obtained zero force constraint scores corresponding to several target predicted particles constitute a zero force constraint score set. Finally, by calculating the half-space constraint score set and the static equilibrium score set using the target predicted particle set, several effective force value data, and the effective contact point set, the half-space constraint score can be used to determine whether the object's spatial position is reasonable, and the static equilibrium score can be used to determine whether the contact mechanics meets the equilibrium requirements. The aforementioned preset distance residual standardization coefficient, preset friction cone violation standardization coefficient, and preset zero force constraint standardization coefficient can be determined based on offline calibration experiments.

[0051] Furthermore, the calculation of the half-space constraint score set and the static equilibrium score set based on the target predicted particle set, several effective force value data, and the effective contact point set includes: Based on the target prediction particle set, obtain the positions of several object reference points corresponding to several target prediction particles; Several sensitive out-of-plane normals are obtained based on several effective force values, and the half-space constraint score set is calculated based on several sensitive out-of-plane normals, several object reference point positions, and preset half-space constraint standardization coefficients. Several relative moments are calculated based on the effective contact point set and the coordinate positions of the preset reference points. The static equilibrium score set is then calculated based on the effective contact point set, the preset equivalent external load, the preset resultant force residual standardization coefficient, the preset resultant moment residual standardization coefficient, the preset equivalent moment term, and several relative moments.

[0052] In this embodiment, the positions of several object reference points corresponding to several target prediction particles are obtained through the target prediction particle set, which provides a positional basis for calculating the half-space constraint score. Next, several out-of-plane normals are obtained through several effective force value data. Based on these out-of-plane normals, the positions of several object reference points, and a preset half-space constraint standardization coefficient, a half-space constraint score set is calculated. This half-space constraint score can be used to determine whether an object is within the effective sensing range of the sensor. Specifically: ,in, Predict the half-space constraint score for the particle of the j-th target. The out-of-plane normal for the i-th effective force value data. Predict the object reference point position of the particle for the j-th target. The center position of the i-th effective force value data. The standardization coefficients for the pre-defined half-space constraints.

[0053] Since tactile sensors are typically more sensitive to contact only on the outer side of their sensitive surface, a corresponding half-space constraint can be constructed for each effective contact point based on the normal of its sensitive surface: when the object reference point position assumed for the j-th target prediction particle falls into the half-space behind the effective force value data, it indicates that the spatial position of the object corresponding to the target prediction particle is inconsistent with the effective sensing direction of the tactile sensor, and a penalty should be imposed; when the object reference point position assumed for the j-th target prediction particle falls into the half-space outside the sensitive surface, it is considered that the target prediction particle meets the spatial feasibility requirements, thus eliminating particles that appear geometrically reasonable but whose spatial orientation does not conform to the sensor's sensing mechanism. Then, by calculating several relative torques using the effective contact point set and the preset reference point coordinates, basic data can be provided for mechanical equilibrium assessment.

[0054] Finally, by calculating the static equilibrium score set using the effective contact point set, preset equivalent external load, preset resultant force residual standardization coefficient, preset resultant moment residual standardization coefficient, preset equivalent moment term, and several relative moments, it is possible to determine whether the contact mechanics under the particle assumption meets the quasi-static equilibrium requirements through the static equilibrium score. Specifically: ,in, As the resultant residual term, it can reflect the degree of consistency between each contact force and the preset equivalent external load in the sense of translational equilibrium; The resultant moment residual term reflects the degree of consistency between the moment of each contact force about the coordinate position of the preset reference point and the external moment in the sense of rotational equilibrium. Predict the static equilibrium score of the particle for the j-th target. To preset the first coefficient for static equilibrium, To preset the second coefficient for static equilibrium, The preset equivalent external load can be determined by the object's mass, gravitational acceleration, or the object's center of mass position, as well as known environmental forces. The relative torque of the i-th effective contact point to the coordinate position of the preset reference point can be selected as the centroid of the object, the geometric center of the object, or a preset torque calculation reference point. The preset equivalent torque term can be calculated from the lever arm of the preset equivalent external load relative to the coordinate position of the preset reference point. When no additional preset equivalent external load is set or the external disturbance torque cannot be measured, the preset equivalent torque term can be set to zero or set to a calibrated value. To pre-determine the standardized coefficient of the resultant force residual, The preset normalized coefficients for resultant moment residuals, resultant force residuals, and resultant moment residuals are used to adjust the scale of the influence of the static equilibrium term on the overall observed log-likelihood of the particle. Under quasi-static operating conditions, the resultant force and resultant moment acting on an object should be as close to equilibrium as possible. If the contact mechanics state corresponding to the j-th predicted target particle better matches the actual quasi-static contact conditions, its static equilibrium score is higher; if the j-th predicted target particle is mechanically unreasonable, its static equilibrium score is lower. The aforementioned preset normalized coefficients for half-space constraints, resultant force residuals, and resultant moment residuals can be determined based on offline calibration experiments.

[0055] Furthermore, the step of obtaining the enhanced particle observation log-likelihood set based on the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set includes: Based on the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set, calculate the log likelihood values ​​of several real-time particle observations corresponding to several target predicted particles; The log-likelihood values ​​of pre-acquired historical particle observations and several real-time particle observation log-likelihood values ​​are fused to obtain an enhanced set of particle observation log-likelihood values.

[0056] In this embodiment, the log-likelihood values ​​of several real-time particle observations corresponding to several target predicted particles are calculated using geometric consistency score sets, directional consistency score sets, friction cone score sets, zero-force constraint score sets, half-space constraint score sets, and static equilibrium score sets. This allows for the quantification of the matching degree between each particle and the tactile observation. Specifically: ,in, The real-time particle observation log-likelihood value is used to predict the particle for the j-th target. To predetermine the first log-likelihood coefficient, To pre-determine the second log-likelihood coefficient, To pre-determine the third log-likelihood coefficient, To pre-determine the fourth log-likelihood coefficient, The fifth log-likelihood coefficient is preset. The sixth log-likelihood coefficient is preset. Next, to enhance the estimation stability during in-hand operation, the stability and anti-interference capability of object pose estimation are improved by fusing a pre-acquired historical particle observation log-likelihood value set with several real-time particle observation log-likelihood values. This results in an enhanced particle observation log-likelihood value set, specifically: ,in, To enhance the set of log-likelihood values ​​for particle observations, The preset time decay coefficient is preferably less than 1. The k-th frame's fused particle observation log-likelihood value is formed by the pre-acquired historical particle observation log-likelihood value set and several real-time particle observation log-likelihood values, totaling W frames of fused particle observation log-likelihood value. Since the closer the historical particle observation log-likelihood value is to the current moment, the greater its reference value for the current object estimation result, historical particle observation log-likelihood values ​​closer to the current moment have higher weights, while historical frames farther away from the current moment have lower weights. Through this setting, the historical particle observation log-likelihood value set can be used to alleviate problems such as single-frame haptic jitter and instantaneous loss of local contact, and the adverse effects of changes in physical conditions in historical frames on the current object pose estimation can be reduced.

[0057] Further, if the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object, and the updated pose of the object is used as the initial pose of the object. The target predicted particle set is then re-obtained based on the initial pose of the object. The enhanced particle observation log-likelihood value set is then re-obtained based on the target predicted particle set, the target zero-force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions. The target pose mode of the object is then determined based on the enhanced particle observation log-likelihood value set and the target predicted particle set, including: If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object. The updated pose of the object is then used as the initial pose of the object to re-obtain the target prediction particle set based on the initial pose of the object. The enhanced particle observation log likelihood value set is then re-obtained based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data until the pose estimation index meets the preset pose qualification conditions. Then, the target prediction particle set is normalized in the number domain based on the enhanced particle observation log likelihood value set to obtain the target prediction particle weight set. The number of valid samples is calculated based on the target predicted particle weight set. If the number of valid samples is less than a preset sample threshold, a set of retained particles is obtained based on the target predicted particle weight set and the target predicted particle set. Based on a preset object pose parameter library, the reserved particle set is divided into modes to obtain several object pose modes and several reserved particles corresponding to several object pose modes. The target pose pattern of an object is determined based on several retained particles corresponding to several object pose patterns and several object pose patterns.

[0058] In actual handheld operation, situations may occur such as momentary loss of local contact, excessive concentration of particles, short-term slippage of objects, geometric constraint conflicts, or deterioration of the overall particle score, which may lead to degradation of the filter estimation. Therefore, in this embodiment, when the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned and the matching degree between the particles and the observation is continuously optimized until the pose estimation index meets the preset pose qualification conditions and the surface estimation result is stable and reliable. At this time, the target prediction particle set is normalized in the number domain to obtain the target prediction particle weight set, which can clarify the reliability of each particle, improve numerical stability, and avoid underflow or accuracy loss when the number of particles is large or the log-likelihood difference is large. The pose estimation metrics include the enhanced particle observation log-likelihood value, the first particle average penetration violation, and the number of valid samples. The first particle average penetration violation is obtained by calculating the average penetration violation of the target predicted particle set. Specifically, the pose estimation metrics fail to meet the preset pose qualification conditions if: the enhanced particle observation log-likelihood values ​​in the enhanced particle observation log-likelihood value set are all lower than a preset log-likelihood threshold, the number of valid samples is less than a preset sample threshold, and the first particle average penetration violation is greater than a preset average penetration violation threshold. The number of valid samples is calculated using the target predicted particle weight set.

[0059] Then, the number of effective samples can be used to determine whether the particles are exhibiting excessive concentration and degradation. If the number of effective samples is less than a preset sample threshold, it indicates that the particle degradation phenomenon is aggravated, and low-variance resampling is required based on the target predicted particle weight set and the target predicted particle set to retain high-reliability particles and remove invalid particles, resulting in a retained particle set. After resampling, the weights of each particle can be reset to a uniform distribution for continued propagation and updating in the next time step. In this embodiment, if the number of effective samples is greater than or equal to the preset sample threshold, it indicates that the particle weight distribution is relatively uniform, and the particle set maintains good representativeness, so low-variance resampling of the target predicted particle set is unnecessary. The above processing can suppress particle degradation and maintain the effectiveness, diversity, and real-time performance of the particle set. Next, the retained particle set is divided into patterns using a preset object pose parameter library. Based on the object's principal axis direction, rotation angle around the principal axis, or other pose parameters related to object symmetry in the preset object pose parameter library, the retained particles can be binned or clustered to identify multiple high-probability object pose patterns and several particles corresponding to several object pose patterns.

[0060] If the retained particle set contains only a single optimal particle, it can easily lead to pattern collapse or jumps between multiple approximately equivalent poses, which is detrimental to subsequent operational decisions. To reduce the risk of pattern collapse, this embodiment can also inject new particles into the retained particle set under a preset symmetry transformation based on the known symmetry of the current object. The particle injection can include principal axis flipping injection, principal axis rotation injection, or local perturbation injection in the neighborhood of a high-probability pattern, thereby enhancing the filter's ability to represent multiple solution states and improving the stability of the pose estimation of symmetrical objects within the hand. Finally, by using several object pose patterns and corresponding retained particles, the target pose pattern of the object can be determined, thus obtaining the final target pose pattern of the object within the robot's hand.

[0061] In this embodiment, the output object target pose pattern can be published to Rviz (a robot 3D visualization tool) in real time, and information such as the number of valid samples, the number of contact points corresponding to the target force value, and the calculation time can be output simultaneously.

[0062] This embodiment provides a pose estimation system for objects in the hand of a robot. Please refer to [link to relevant documentation]. Figure 2 It includes an initial pose generation module, a particle observation log-likelihood calculation module, a pose estimation index evaluation module, and a target pose determination module, specifically: The initial pose generation module is used to obtain several effective force value data, effective contact point set and target zero force set based on the robot's full palm and robot hand joint state, and obtain the object's initial pose based on the effective contact point set, so as to obtain the target predicted particle set based on the object's initial pose. The particle observation log likelihood calculation module is used to obtain an enhanced particle observation log likelihood value set based on the target predicted particle set, the target zero force set, the effective contact point set, and several effective force value data. The pose estimation index evaluation module is used to obtain pose estimation indexes based on the target prediction particle set and the enhanced particle observation log likelihood value set. The target pose determination module is used to fine-tune the initial pose of the object if the pose estimation index does not meet the preset pose qualification conditions, obtain the updated pose of the object, and use the updated pose of the object as the initial pose of the object, so as to re-obtain the target prediction particle set based on the initial pose of the object, and re-obtain the enhanced particle observation log likelihood value set based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions, and determine the target pose pattern of the object based on the enhanced particle observation log likelihood value set and the target prediction particle set.

[0063] This embodiment provides a pose estimation system for objects within a robot's palm. In practical applications, only an initial pose generation module is needed. By analyzing the robot's entire palm and the states of its hand joints, tactile contact information that creates effective contact with the object within the robot's entire palm can be filtered out, and non-contact areas are marked, resulting in several standard force values, a set of effective contact points, and a set of target zero forces. Then, the initial pose of the object can be obtained from the set of effective contact points, and a target prediction particle set can be generated, providing basic observation data and initial pose basis for subsequent pose estimation. Next, a particle observation log-likelihood calculation module is used to obtain an enhanced particle observation log-likelihood value set from the target prediction particle set, the target zero force set, the set of effective contact points, and several effective force values. This set can measure the degree of matching between each particle and the actual contact situation of the robot's palm, improving the accuracy of object pose estimation. Then, the pose estimation index evaluation module is used to calculate the pose estimation index by using the target prediction particle set and the enhanced particle observation log likelihood set. The pose estimation index can be used to determine whether the current object pose estimation result is reliable. When the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object can be fine-tuned and the estimation of the object pose pattern can be continuously optimized, avoiding the inability to continuously and stably acquire the object pose due to in-palm operation occlusion.

[0064] Finally, a target pose determination module is used to determine the target pose pattern of the object when the pose estimation index meets the preset pose qualification conditions. This enables fast and stable estimation of the pose of objects within the palm of the robot, relying solely on the robot's full-palm tactile information and hand joint states. This embodiment can also support recording sensor readings for each frame, joint angles corresponding to the robot's hand joint states, and control events (start / pause / freeze, etc.) to a log. In replay mode, the serial port can be skipped for direct playback, which is used to compare different parameters, reproduce failure cases, and generate implementation data.

[0065] Furthermore, the initial pose generation module is used to acquire several effective force value data, effective contact point set, and target zero force set based on the robot's entire hand and robot hand joint states, and to acquire the object's initial pose based on the effective contact point set, so as to acquire the target predicted particle set based on the object's initial pose, including: Based on the robot's entire hand, several standard force values ​​were obtained, and based on the robot's hand joint state, several standard force values, and a preset tactile unit coordinate library, several contact candidate point coordinate data were obtained. Based on a preset rejection force value threshold, several standard force value data are filtered to obtain several effective force value data; Based on several effective power value data, obtain the number of activations in several neighborhoods corresponding to several effective power value data; Based on preset sensor partitioning rules, several effective force value data are divided into several sensor partition subsets; Based on several effective force value data and several neighborhood activation numbers, a comprehensive score for several force values ​​corresponding to several effective force value data is calculated. Based on the comprehensive score of several force values ​​and the preset first retention quantity, several effective force value data in several sensor partition subsets are filtered to obtain several effective sensor partition subsets and an initial zero force set; Based on preset sorting conditions, preset second retention quantity, several effective force value data from several effective sensor partition subsets and initial zero force set, obtain target zero force set and target force value data, and obtain effective contact point set based on the target force value data and several contact candidate point coordinate data; The initial pose of the object is obtained based on the set of effective contact points, and the set of object localization particles is obtained based on the initial pose of the object. The set of object localization particles is randomly perturbed to obtain the set of target prediction particles.

[0066] In this embodiment, several standard force values ​​and several candidate contact point coordinates are obtained using the robot's entire hand, the robot's hand joint states, and a preset tactile unit coordinate library, providing basic data for subsequent screening. Next, the standard force values ​​are screened using a preset rejection threshold to eliminate noise and invalid force values, resulting in several effective force values. Then, several neighborhood activation counts are obtained from these effective force values, which can measure the density of the contact area between the object and the robot's hand, facilitating the priority retention of core contact points with more concentrated and stable contact. Subsequently, the effective force values ​​are divided into several sensor partition subsets using preset sensor partitioning rules, preventing a single sensor partition from dominating the estimation results. Finally, a comprehensive force score is calculated using the effective force values ​​and the number of neighborhood activations, objectively assessing the importance of each effective force value. The comprehensive force score, along with a preset first retention count, is used to screen each sensor partition subset, obtaining effective sensor partition subsets and an initial zero-force set to ensure a balanced distribution of contact points. Next, by using preset sorting conditions, a preset second retention quantity, several effective force value data from several effective sensor partition subsets, and an initial zero force set, the target zero force set and target force value data are obtained. Based on the target force value data and several contact candidate point coordinate data, an effective contact point set is obtained, which can filter out tactile contact information that generates effective contact with the object within the robot's entire palm and mark non-contact areas. Finally, the initial pose of the object is obtained through the effective contact point set, and then the target prediction particle set is obtained through the initial pose of the object, which can provide reliable initial particle samples for subsequent object pose estimation.

[0067] This embodiment also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the functions of the system as described above.

[0068] It is understood that the above system embodiments correspond to the method embodiments of the present invention, and can realize the pose estimation method for an object in the palm of a robot provided by any of the above method embodiments of the present invention.

[0069] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0070] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for estimating the pose of an object in the hand of a robot, characterized in that, include: Based on the robot's entire hand and the state of the robot's hand joints, several effective force values, effective contact point sets, and target zero force sets are obtained. The initial pose of the object is obtained based on the effective contact point set, and the target predicted particle set is obtained based on the initial pose of the object. Based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, an enhanced particle observation log likelihood value set is obtained; Pose estimation metrics are obtained based on the target prediction particle set and the enhanced particle observation log-likelihood set. If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object. The updated pose of the object is then used as the initial pose of the object. The target prediction particle set is then re-obtained based on the initial pose of the object. The enhanced particle observation log-likelihood value set is then re-obtained based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions. Finally, the target pose mode of the object is determined based on the enhanced particle observation log-likelihood value set and the target prediction particle set.

2. The method for estimating the pose of an object in the hand of a robot according to claim 1, characterized in that, The process involves acquiring several effective force values, a set of effective contact points, and a set of target zero forces based on the robot's entire hand and the state of its hand joints. It also involves obtaining the object's initial pose based on the set of effective contact points, and then obtaining a set of predicted target particles based on the object's initial pose. This includes: Based on the robot's entire hand, several standard force values ​​were obtained, and based on the robot's hand joint state, several standard force values, and a preset tactile unit coordinate library, several contact candidate point coordinate data were obtained. Based on a preset rejection force value threshold, several standard force value data are filtered to obtain several effective force value data; Based on several effective power value data, obtain the number of activations in several neighborhoods corresponding to several effective power value data; Based on preset sensor partitioning rules, several effective force value data are divided into several sensor partition subsets; Based on several effective force value data and several neighborhood activation numbers, a comprehensive score for several force values ​​corresponding to several effective force value data is calculated. Based on the comprehensive score of several force values ​​and the preset first retention quantity, several effective force value data in several sensor partition subsets are filtered to obtain several effective sensor partition subsets and an initial zero force set; Based on preset sorting conditions, preset second retention quantity, several effective force value data from several effective sensor partition subsets and initial zero force set, obtain target zero force set and target force value data, and obtain effective contact point set based on the target force value data and several contact candidate point coordinate data; The initial pose of the object is obtained based on the set of effective contact points, and the set of localized particles of the object is obtained based on the initial pose of the object. The set of localized particles of the object is randomly perturbed to obtain the set of predicted particles of the target.

3. The method for estimating the pose of an object in the hand of a robot according to claim 2, characterized in that, The process of obtaining the initial pose of the object based on the effective contact point set, and obtaining the object localization particle set based on the initial pose of the object, and then randomly perturbing the object localization particle set to obtain the target prediction particle set, includes: Calculate sublinear weights for several contact points based on the effective set of contact points; The centroid and average force direction of the contact points are calculated based on the effective set of contact points and the sublinear weights of several contact points. A coarse initial pose is obtained based on the centroid of the contact point and the average force direction; Fine-tune the coarse initial pose based on a preset offset to obtain the object's initial pose. Based on the initial pose of the object, a set of object positioning particles is obtained. The set of object positioning particles is randomly perturbed based on a preset zero-mean three-dimensional Gaussian distribution function and a preset zero-mean three-dimensional angular axis perturbation distribution function to obtain a set of target predicted particle translations. Pose mapping is performed on the target predicted particle translation set to obtain the target predicted particle pose set; The target predicted particle set is obtained based on the target predicted particle translation set and the target predicted particle posture set.

4. The method for estimating the pose of an object in the hand of a robot according to claim 1, characterized in that, The method for obtaining the enhanced particle observation log-likelihood value set based on the target predicted particle set, the target zero-force set, the effective contact point set, and several effective force value data includes: Based on the target predicted particle set and the pre-acquired 3D model of the object, the surface data of the object is determined; Based on the target prediction particle set, target zero force set, effective contact point set, and several effective force value data, calculate the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set; The enhanced particle observation log-likelihood set is obtained based on the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set.

5. The method for estimating the pose of an object in the hand of a robot according to claim 4, characterized in that, The calculation of geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set based on the target predicted particle set, target zero force set, effective contact point set, and several effective force value data includes: Based on the target prediction particle set, the effective contact point set, and the object surface data, several effective contact distance residuals are calculated, and based on several effective contact distance residuals, a preset effective contact point weight set, and a preset distance residual standardization coefficient, a geometric consistency score set is calculated. Based on the target prediction particle set and the effective contact point set, determine the surface normals of several objects corresponding to several effective contact points, and calculate the direction consistency score set based on the effective contact point set, the preset effective contact point weight set and the surface normals of several objects. Based on the effective contact point set, obtain several normal force components and several tangential force components corresponding to the effective contact point set, and calculate the friction cone score set based on the preset effective contact point weight set, several normal force components, several tangential force components and preset friction cone violation standardization coefficient. Based on the target predicted particle set, the target zero force set, and the object surface data, calculate several zero force contact distance residuals, and calculate the zero force constraint score set based on the several zero force contact distance residuals, the preset zero force constraint standardization coefficient, and the preset safety constraint. The half-space constraint score set and static equilibrium score set are calculated based on the target prediction particle set, several effective force value data and effective contact point set.

6. The method for estimating the pose of an object in the hand of a robot according to claim 4, characterized in that, The calculation of the half-space constraint score set and static equilibrium score set based on the target predicted particle set, several effective force value data, and effective contact point set includes: Based on the target prediction particle set, obtain the positions of several object reference points corresponding to several target prediction particles; Several sensitive out-of-plane normals are obtained based on several effective force values, and the half-space constraint score set is calculated based on several sensitive out-of-plane normals, several object reference point positions, and preset half-space constraint standardization coefficients. Several relative moments are calculated based on the effective contact point set and the coordinate positions of the preset reference points. The static equilibrium score set is then calculated based on the effective contact point set, the preset equivalent external load, the preset resultant force residual standardization coefficient, the preset resultant moment residual standardization coefficient, the preset equivalent moment term, and several relative moments.

7. The method for estimating the pose of an object in the hand of a robot according to claim 4, characterized in that, The method for obtaining the enhanced particle observation log-likelihood set based on the geometric consistency score set, directional consistency score set, friction cone score set, zero-force constraint score set, half-space constraint score set, and static equilibrium score set includes: Based on the geometric consistency score set, directional consistency score set, friction cone score set, zero force constraint score set, half-space constraint score set, and static equilibrium score set, calculate the log likelihood values ​​of several real-time particle observations corresponding to several target predicted particles; The log-likelihood values ​​of pre-acquired historical particle observations and several real-time particle observation log-likelihood values ​​are fused to obtain an enhanced set of particle observation log-likelihood values.

8. The method for estimating the pose of an object in the hand of a robot according to claim 1, characterized in that, If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object, and the updated pose of the object is used as the initial pose of the object. The target predicted particle set is then re-obtained based on the initial pose of the object. The enhanced particle observation log-likelihood value set is then re-obtained based on the target predicted particle set, the target zero-force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions. The target pose mode of the object is then determined based on the enhanced particle observation log-likelihood value set and the target predicted particle set, including: If the pose estimation index does not meet the preset pose qualification conditions, the initial pose of the object is fine-tuned to obtain the updated pose of the object. The updated pose of the object is then used as the initial pose of the object to re-obtain the target prediction particle set based on the initial pose of the object. The enhanced particle observation log likelihood value set is then re-obtained based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data until the pose estimation index meets the preset pose qualification conditions. Then, the target prediction particle set is normalized in the number domain based on the enhanced particle observation log likelihood value set to obtain the target prediction particle weight set. The number of valid samples is calculated based on the target predicted particle weight set. If the number of valid samples is less than a preset sample threshold, a set of retained particles is obtained based on the target predicted particle weight set and the target predicted particle set. Based on a preset object pose parameter library, the reserved particle set is divided into modes to obtain several object pose modes and several reserved particles corresponding to several object pose modes. The target pose pattern of an object is determined based on several retained particles corresponding to several object pose patterns and several object pose patterns.

9. A pose estimation system for an object in the palm of a robot, characterized in that, It includes an initial pose generation module, a particle observation log-likelihood calculation module, a pose estimation index evaluation module, and a target pose determination module, specifically: The initial pose generation module is used to obtain several effective force value data, effective contact point set and target zero force set based on the robot's full palm and robot hand joint state, and obtain the object's initial pose based on the effective contact point set, so as to obtain the target predicted particle set based on the object's initial pose. The particle observation log likelihood calculation module is used to obtain an enhanced particle observation log likelihood value set based on the target predicted particle set, the target zero force set, the effective contact point set, and several effective force value data. The pose estimation index evaluation module is used to obtain pose estimation indexes based on the target prediction particle set and the enhanced particle observation log likelihood value set. The target pose determination module is used to fine-tune the initial pose of the object if the pose estimation index does not meet the preset pose qualification conditions, obtain the updated pose of the object, and use the updated pose of the object as the initial pose of the object, so as to re-obtain the target prediction particle set based on the initial pose of the object, and re-obtain the enhanced particle observation log likelihood value set based on the target prediction particle set, the target zero force set, the effective contact point set, and several effective force value data, until the pose estimation index meets the preset pose qualification conditions, and determine the target pose pattern of the object based on the enhanced particle observation log likelihood value set and the target prediction particle set.

10. The pose estimation system for an object in the hand of a robot according to claim 9, characterized in that, The initial pose generation module is used to acquire several effective force values, effective contact point sets, and target zero force sets based on the robot's entire hand and the state of its hand joints, and to acquire the object's initial pose based on the effective contact point set, so as to acquire the target predicted particle set based on the object's initial pose, including: Based on the robot's entire hand, several standard force values ​​were obtained, and based on the robot's hand joint state, several standard force values, and a preset tactile unit coordinate library, several contact candidate point coordinate data were obtained. Based on a preset rejection force value threshold, several standard force value data are filtered to obtain several effective force value data; Based on several effective power value data, obtain the number of activations in several neighborhoods corresponding to several effective power value data; Based on preset sensor partitioning rules, several effective force value data are divided into several sensor partition subsets; Based on several effective force value data and several neighborhood activation numbers, a comprehensive score for several force values ​​corresponding to several effective force value data is calculated. Based on the comprehensive score of several force values ​​and the preset first retention quantity, several effective force value data in several sensor partition subsets are filtered to obtain several effective sensor partition subsets and an initial zero force set; Based on preset sorting conditions, preset second retention quantity, several effective force value data from several effective sensor partition subsets and initial zero force set, obtain target zero force set and target force value data, and obtain effective contact point set based on the target force value data and several contact candidate point coordinate data; The initial pose of the object is obtained based on the set of effective contact points, and the set of localized particles of the object is obtained based on the initial pose of the object. The set of localized particles of the object is randomly perturbed to obtain the set of predicted particles of the target.

Images