A method for calibrating, mapping and generating standard samples of cross-hand shape tactile data based on ontology deformation reference, electronic equipment and medium

Abstract

The application discloses a kind of based on the calibration, mapping and standard sample generation method of cross-hand shape tactile data based on ontology deformation reference, electronic equipment and storage medium, belong to flexible tactile sensing and man-machine teaching data processing technical field, solve the technical defects that existing tactile acquisition glove cannot cross ontology uniform, no physical traceable calibration reference, sample quality is not quantifiable, cannot be directly migrated to the dexterous hand of robot due to hand shape difference, wearing pre-tightening, module slip, flexible deformation;The application establishes physical calibration link driven by entity flexible structure, unlike pure data domain adaptation scheme, with explainability, cross-hand / cross-glove / cross-robot versatility, data traceable, quality quantifiable, greatly reduce the data acquisition cost of dexterous operation teaching, suitable for piezoresistive, capacitance, yarn, visual tactile all types of flexible sensing gloves.

Description

Technical Field

[0001] This invention relates to the fields of flexible tactile sensing, human-computer teaching, and embodied intelligent robot data processing technology, specifically to a method, electronic device, and medium for cross-hand shape tactile data calibration, mapping, and standard sample generation based on body deformation reference. Background Technology

[0002] Embodied intelligence and dexterous robotic manipulation heavily rely on tactile datasets generated through human hand teaching. Existing flexible tactile sensing gloves commonly suffer from interference factors such as flexible substrate deformation, differences in user hand size, varying pre-tightening force, tactile module slippage, and non-contact finger flexion and extension. Therefore, data output solely from the raw sensor channels cannot be directly used for cross-entity model training. Existing publicly available technologies have many inherent limitations, which will be explained one by one based on a literature review:

[0003] Limitations of existing similar patent technologies

[0004] (1) US10831272B1 only uses deformation sensors to estimate hand posture, without constructing a deformation reference for the glove body, and cannot separate task tactile sensation from wearing deformation interference, nor can it output standardized tactile samples with unified coordinates;

[0005] (2) US20190357811A1 discloses a double-layer sensor glove that simultaneously collects pressure and deformation, but does not use topological corridors to construct a global ontological reference, cannot recover the relative six degrees of freedom pose of multiple tactile areas, and cannot complete the unified mapping across hand coordinates;

[0006] (3) CN121018517A and WO2025085232A1 only realize remote operation data synchronization and task primitive slicing, lack the deformation correction link brought by the physical flexible structure, rely on pure data domain adaptation, and have poor physical interpretability.

[0007] (4) CN113124746A only completes single sensor self-calibration. The calibration target is single-channel measurement accuracy. It does not take deformation reference as the core condition for data standardization and cross-hand shape mapping, and there is no standardized sample output link.

[0008] (5) US10481689B1 and US10884496B2 only optimize motion capture and adapt to hand skeleton, and only process joint angle kinematics. They do not perform geometric correction for tactile sampling points, contact normals, contact areas and pressure densities attached to the flexible substrate.

[0009] (6) Existing domain adaptation patents US20200279134A1 and US20240338081A1 are general algorithm-level domain transfers that do not rely on physical observation of physical glove deformation, cannot quantify sample confidence, have no calibration version traceability mechanism, and cannot identify and remove low-quality interference samples.

[0010] Six core defects of existing technology

[0011] ① There is no fixed correspondence between channel number and physical contact position: After different hands open the gloves, the spatial position of the same sensor unit shifts, and the original signal cannot be directly merged into the same dataset;

[0012] ② Wearing and posture interference have no separation mechanism: wristband pretension, non-contact finger flexion, and module slippage will cause tactile zero point and gain drift. The existing data format does not record deformation interference conditions, and interference and real contact cannot be distinguished.

[0013] ③ Only joint skeleton redirection is performed, ignoring tactile geometric parameter mapping: Traditional kinematic mapping only transforms joint angles, without correcting the three-dimensional coordinates of tactile sampling points, contact normals, contact areas, and pressure densities;

[0014] ④ Static one-time calibration failure: During task execution, the glove continues to deform, and a single pre-calibration cannot dynamically compensate for the relative pose changes of the module during operation;

[0015] ⑤ Data lacks physical traceability and quality quantification: The dataset does not store calibration versions, deformation residuals, or sample confidence scores, making it impossible to distinguish whether the model's generalization failure is due to sensor hardware failure or algorithm domain adaptation issues.

[0016] ⑥ High cost of cross-device migration: There is a lack of a unified intermediate standard coordinate system between human hands, different types of gloves, and robotic arms, which requires repeated collection of a large number of teaching samples, resulting in extremely high costs.

[0017] In summary, the industry lacks a complete processing method that relies on the deformation of physical topology corridors to establish physical ontological benchmarks, dynamically recover the six-DOF poses of multiple tactile areas, separate ontological interference, unify cross-hand shape coordinates, and output traceable standardized tactile samples. To address this, we propose a method, electronic equipment, and medium for cross-hand shape tactile data calibration, mapping, and standard sample generation based on ontological deformation benchmarks. Summary of the Invention

[0018] In view of this, the present invention provides a method, electronic device and medium for cross-hand shape tactile data calibration, mapping and standard sample generation based on body deformation reference, so as to solve or alleviate the technical problems existing in the prior art, and at least provide a beneficial option.

[0019] The technical solution of this invention is implemented as follows: This method relies on a special flexible tactile sensing glove, the glove's structural features are as follows:

[0020] 1) Divide the area into multiple independent local tactile reference regions, each with a built-in tactile sensing unit (piezoresistive / capacitive / visual-tactile array), and assign a unique reference_region_id to each reference region;

[0021] 2) A topological strain routing corridor is arranged between two adjacent local tactile reference areas. The corridor contains distributed deformation sensors and outputs a continuous deformation vector. ;

[0022] 3) The glove integrates a clock synchronization module, and the tactile sensing and deformation sensing share a unified hardware timestamp to ensure signal timing alignment. 2.2 Overall method flow (corresponding to claims 1S1-S8, see the Mermaid flowchart below) 1) Synchronous acquisition: All tactile and corridor deformation raw signals are acquired in parallel using the same clock, and hardware identification, hand shape, pretension, and environmental parameters are recorded;

[0023] 2) Constructing the body deformation benchmark: Under non-contact standard posture, a multi-dimensional benchmark vector of the binding calibration version v is generated based on the corridor static deformation, hand shape, and installation parameters. ;

[0024] 3) Dynamically recover the pose of the reference area: Solve the transformation matrix of adjacent reference areas using corridor deformation. The graph optimization closed loop yields the global palm reference transformation. ;

[0025] 4) Separating the tactile components of the task: based on a conditional observation model Subtracting the main body interference Obtain the pure quest touch ;

[0026] 5) Standard hand-shaped coordinate mapping: via

[0027] The local tactile geometry is unified to a standard coordinate system, and the normal, area, and force are synchronously corrected based on the Jacobian matrix, supporting secondary mapping to the robotic arm;

[0028] 6) Confidence quantification: Calculate sample confidence using multi-dimensional weighted methods. ;

[0029] 7) Generate standard tactile samples: Output a data object containing fixed standardized fields, associated with the original data hash, calibration version, and transformation chain;

[0030] 8) Sample splitting: Samples meeting the confidence level are included in the training database; those not meeting the level are downweighted, removed, or recalibrated; if the baseline drift exceeds the standard, a new calibration version is automatically generated to isolate the data.

[0031] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions:

[0032] I. This invention transforms the uncontrollable error sources of glove opening, wearing pre-tightening, and module sliding into observable and calibrable corridor deformation observations, establishes a permanently bound and calibrated ontological reference, and, unlike pure data domain adaptation without hardware support, possesses complete physical traceability. Through topological graph closed-loop optimization, it dynamically recovers the six-degree-of-freedom pose of the entire tactile reference area, and completely calibrates all tactile geometric parameters such as the three-dimensional coordinates of the contact point, the contact normal, the contact area, and the pressure density, breaking through the limitation of traditional methods that only calibrate joint angles.

[0033] Second, this invention sets a standard hand shape as a universal intermediate coordinate system, which can uniformly map data from small / medium / large human hands, different models of collection gloves, and various dexterous robotic hands, greatly reducing the workload of repetitive teaching and collection, and lowering the cost of dataset production. Standard tactile samples are fixed with unified data fields, and each sample is bound to a calibration version, a complete spatial transformation chain, original data hash, and multi-dimensional confidence. When model training generalization fails, sensor hardware failure, calibration drift, and algorithm defects can be quickly distinguished, which facilitates fault location.

[0034] Third, the method of this invention is compatible with all types of flexible sensing gloves, including piezoresistive, capacitive, conductive yarn, and visual-tactile gloves. The deformation-pose mapping operator supports multiple implementation methods such as analytical, finite element, regression, and neural network. There are no hardware binding restrictions. The body reference is automatically constructed without contact with the standard posture. New calibration versions are automatically generated in the scenarios of module replacement and reference drift. There is no need for manual recording of calibration parameters, which reduces the operation and maintenance threshold.

[0035] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a block diagram of the cross-overall hierarchical system of the present invention;

[0038] Figure 2 This is a complete execution timing flowchart of the present invention. Detailed Implementation

[0039] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0040] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0041] like Figure 1-2 As shown,

[0042] Example 1: Generation of standardized tactile samples in batches for three hand shapes (capacitive tactile array gloves)

[0043] Hardware configuration: The flexible tactile glove contains 12 local tactile reference zones, with topological strain routing corridors arranged between adjacent reference zones, and a capacitive pressure sensing array is installed in the reference zone; it is divided into three categories of user hand shapes: small, medium and large.

[0044] Calibration Phase: After wearing gloves, each hand shape was subjected to three contactless standard postures in sequence: relaxed, open palm, and clenched fist; static deformation vectors of each corridor were collected. Hand shape parameters Wristband pretension state Calculate the deformation datum of the body Each type of hand shape generates an independent calibration_version.

[0045] Teaching and data acquisition: Three types of users respectively hold a standard cylinder and a cube to perform pressing, pinching, and sliding operations, and tactile signals and real-time deformation of the corridor are collected simultaneously. .

[0046] Pose recovery: via deformation-pose operator Solving for the transformation matrix of adjacent reference regions, and performing topological graph closed-loop optimization, yields the value of each reference region relative to the palm-sized master reference. .

[0047] Interference separation: Substitute the conditional observation model and subtract the additive offset caused by pre-tightening and stretching. Pure contact pressure data were obtained. .

[0048] Coordinate mapping: registration based on spline of each type of hand-shaped thin plate The local tactile points are mapped to a unified standard hand coordinate system, and the contact area is scaled and the pressure density is corrected simultaneously.

[0049] Confidence calculation: q is calculated by fusing pose residuals, corridor drift, and capacitance channel signal-to-noise ratio, and low-quality deformation drift samples are filtered out.

[0050] Downstream applications: All standard samples are uniformly stored in the database, and "leaving one hand shape" cross-validation is used. Compared with the uncorrected original data, the generalization accuracy of the tactile perception model trained with the standardized samples of this invention is improved by 27.3%.

[0051] Example 2: Dynamic Background Correction for Dual-Vision Haptic Unit Gloves

[0052] Hardware configuration: The glove has one set of visual-tactile units on the index finger and thumb, which are connected by a topological strain corridor. The corridor has a built-in distributed fiber optic deformation sensor.

[0053] Benchmark Construction: Non-contact acquisition of static deformation of the corridor to generate a body deformation benchmark. The external calibration component acquires the initial external parameters of the visual unit.

[0054] Operational correction: During object grasping, corridor deformation is read in real time, and the six-DOF extrinsic parameters of the two visual-tactile units are dynamically updated. Based on the body reference, a non-contact reference background image is dynamically switched to correct pixel distortion caused by elastic layer deformation.

[0055] Coordinate Fusion: Two Sets of Visual-Haptic Point Clouds Mapped to the standard hand coordinate system, the residuals are incorporated into the confidence component. The final output is the fused global standard tactile sample.

[0056] Results: Eliminates visual-tactile image drift caused by finger bending and glove stretching; reduces contact point cloud positioning error from 1.6mm to 0.38mm.

[0057] Example 3: Transferring human hand tactile samples to a seven-DOF dexterous manipulator

[0058] Preprocessing: The hand teaching data is completely converted to the standard hand coordinate system by the method of this invention. Each sample stores the complete transformation chain, source reference area ID, confidence level, and calibration version.

[0059] Cross-device mapping: Divide the hand into four functional areas: thumb-opposite area, fingertip, finger side, and palm surface, and establish a one-to-one correspondence between the functional areas of the standard hand shape and the tactile surface of the robotic hand.

[0060] Parametric conversion: When the same robotic arm area corresponds to multiple human hand sampling points, the total force and average pressure are aggregated by weighting the contact area; when a single human hand area corresponds to multiple robotic arm partitions, the confidence level of the corresponding sample is reduced by attenuating the spatial distance.

[0061] Model training: The mapped standard robotic arm tactile samples are used to train the robot's dexterous pinching and sliding operation behavior cloning strategy, eliminating the need for manual re-teaching of the robot and reducing the cost of teaching data collection by 62%.

[0062] Example 4: Online Adaptive Calibration for Module Replacement

[0063] Scenario: The tactile module of the glove index finger is worn and disassembled, and replaced with a brand new capacitive sensing module of the same model.

[0064] Automatic determination: The system reads the hardware ID of the new module, collects the static deformation of the corridor after installation, and calculates the module's positioning offset.

[0065] Branch processing: If the offset is less than 0.5mm (accuracy threshold), only the local fixed transformation of the module is updated, and the original calibration version is used; if the offset is greater than 0.5mm, a brand new calibration_version is automatically generated. The old and new version samples are stored in separate databases and mixed training is prohibited to avoid calibration drift polluting the dataset.

[0066] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for cross-hand shape tactile data calibration, mapping, and standard sample generation based on ontological deformation datum, characterized in that, A flexible tactile sensing glove is applied to a topological strain routing corridor that incorporates multiple local tactile reference zones and connects adjacent reference zones. The topological strain routing corridor has a built-in deformation sensor, and the local tactile reference zones have built-in tactile sensor arrays / visual-tactile units. The method includes the following steps: S1 Synchronous acquisition step: Synchronously acquire the local tactile measurement signals output from all local tactile reference zones and the corridor deformation vectors output from all topological strain routing corridors based on a unified time reference. Simultaneously input glove ID, haptic module ID, and user hand shape parameters. Wristband pretension status, ambient temperature and humidity; S2 Body Deformation Reference Construction Steps: Perform a standard pose acquisition process without task contact, based on the corridor static deformation vector. Corridor covariance Hand shape parameters Installation pre-tightening state Construct an ontology deformation datum that binds to a uniquely identifiable version v. ; S3 tactile reference area pose real-time recovery steps: For any adjacent tactile reference areas i and j, read the corridor deformation vector at the current moment. Model parameters corresponding to calibration version v Based on the deformation-pose mapping operator Solve for the six-degree-of-freedom transformation matrix of reference region i relative to reference region j. ; S4 Task Tactile Component Separation Steps: Establish a tactile conditional observation model; S5 Standard Hand Coordinate Mapping Steps: Local coordinates of any tactile sampling point within the reference area First, through global transformation Transform to the current local coordinate system of the glove, and then construct a piecewise nonlinear registration mapping based on the hand shape parameters. The tactile geometric parameters are uniformly mapped to the standard hand coordinate system; S6 Sample Confidence Quantification Steps: Integrate corridor channel health, pose closed-loop residual, reference drift, tactile saturation, and module installation deviation to calculate the sample normalized confidence q; S7 Standard Tactile Sample Generation Steps: Construct a standardized data object. The data object has fixed fields that include at least: unified timestamp, reference region number (reference_region_id), pure task tactile data (local_tactile_data), corridor deformation raw vector (corridor_deformation), global spatial transformation state (spatial_relation_state), contact position (contact_position_m) in the standard coordinate system, unit contact normal (contact_normal_unit), force / pressure vector (force_vector_n / pressure_pa), contact area (contact_area_m2), coordinate system identifier (frame_id), hand shape morphology parameter (morphology_parameter), calibration version (calibration_version), confidence level (uncertainty), and raw data hash (raw_data_reference). S8 sample splitting process steps: Determine whether the confidence level q is higher than the preset threshold; If the standard is met, the standard tactile samples will be stored in the dataset for model training and cross-ontology transfer. If the criteria are not met, the sample will be downweighted and removed or a recalibration process will be triggered. In this process, steps S1 to S8 are bound to the same calibration version v. When the baseline drift exceeds the threshold, a new calibration version is automatically generated. The old and new version samples are stored separately and are not mixed for training.

2. The method according to claim 1, characterized in that, The standard postures without task contact in step S2 include at least three basic postures: naturally relaxed hand, fully open hand, and clenched fist. No external object contacts the tactile reference area of ​​the glove throughout the entire acquisition process, and the acquisition window covers the stable response range of the deformation sensor. The body deformation reference Permanently associate the current glove hardware serial number, haptic module model, and calibration algorithm version to form a unique and traceable calibration index. The benchmark construction formula is as follows: In the formula, \(Φ\) is the reference mapping operator, and the output is a multidimensional reference vector containing the original deformation vector, low-dimensional embedding, graph node state, and geometric parameters.

3. The method according to claim 1, characterized in that, The deformation-pose mapping operator in step S3 The implementation method is selected from any one of the following: analytical kinematics solution, finite element reduced-order regression, lookup table interpolation, linear / nonlinear regression, and shallow neural network fitting; the closed-loop consistency optimization adopts a graph optimization algorithm to minimize the pose residual caused by multi-corridor observations and eliminate the cumulative error of topological loops. The solution formula is: Based on the adjacency topology graph formed by all corridors, a closed-loop consistency optimization method is used to solve for the global transformation matrix of each reference area relative to the main reference at the palm. .

4. The method according to claim 1, characterized in that, In step S4, the body interference component It includes three types of interference: zero-point offset caused by wristband pretension, elastic gain drift caused by non-contact finger flexion and extension, and spatial offset caused by the sliding of the tactile module along the flexible base of the glove; additive offset correction is used for capacitive and piezoresistive array sensors, and joint correction of image background dynamic replacement and pixel coordinate remapping is used for visual tactile sensors, and the original tactile signal is corrected. satisfy: In the formula: The tactile components of the task to be extracted. The reference area pose correction state is output from step 3. For ambient temperature, Install calibration parameters for the module. For measuring noise; Based on the body deformation datum Global transformation Construct an inverse correction model to separate the bulk interference components. To obtain the pure task tactile component Separation formula: For the visual-tactile unit, synchronous based Update the non-contact reference background image and correct image distortion caused by elastic layer deformation and cell space offset in layers.

5. The method according to claim 1, characterized in that, The piecewise nonlinear registration mapping in step S5 The implementation methods include: thin-plate spline interpolation, keypoint elastic registration, scale-segmented affine transformation, graph matching, and deep learning deformation registration; for scenarios where multiple human hand sampling points are mapped to the same robotic hand tactile area, force, pressure, and area physical quantities are weighted and aggregated; when a single standard hand shape region is mapped to multiple robotic hand regions, the confidence of the corresponding sample decreases with spatial distance decay, and the mapping formula is: set up For mapping Jacobian matrix, contact normal After transformation and normalization, the contact area is scaled using the absolute value of the Jacobian determinant, and the total force vector is rotated synchronously in the coordinate system. The pressure density is scaled in conjunction with the area. If mapping to the target robotic arm is required, a secondary transformation from the standard hand shape to the robotic arm coordinate system is performed based on the one-to-one correspondence rule of the finger functional areas. Unmatched areas are marked as invalid. The S6 calculation formula is as follows: For preset weighting coefficients, For the pose residual confidence components, As the baseline drift component, For the tactile signal quality component, Install the modules separately.

6. The method according to claim 1, characterized in that, The standard tactile samples generated in step S7 are compatible with three types of downstream applications: 1) training a tactile perception model with body deformation reference and hand shape parameters as input conditions; 2) completing dexterity operation strategy training with standard coordinate tactile data as the main input; 3) dividing different hand shapes and gloves into independent domains and completing cross-domain transfer learning based on sample confidence weighting. Downstream models include convolutional networks, recurrent neural networks, Transformers, graph neural networks, reinforcement learning policy networks, and diffusion models; the method does not restrict the structure of downstream models.

7. The method according to claim 1, characterized in that, It also includes a cross-hand shape batch calibration sub-process: performing S2 to construct corresponding body deformation datums for three typical hand shapes: small, medium, and large. Registration mapping The contact truth values ​​were collected using a unified standard calibration component. The consistency residuals of different hand shape samples mapped to the standard coordinate system were calculated, and the corresponding values ​​for each hand shape were iteratively optimized. Parameters to reduce the bias in the distribution of cross-hand shape data.

8. The method according to claim 1, characterized in that, It also includes a haptic module replacement adapter sub-process: after the sensor module in the local haptic reference area is disassembled and replaced, the unique identifier of the new module and the installation positioning deformation data are read; If the module's positioning repetition accuracy is higher than the preset threshold, only the local fixed transformation of the module to the reference area is updated; if the positioning deviation exceeds the standard, a new calibration_version is automatically generated, and the old and new calibration version samples are stored independently and are prohibited from being mixed in model training.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executable programs; The processor calls the program stored in the memory to execute the cross-hand shape tactile data calibration, mapping and standard sample generation method based on the body deformation reference as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for cross-hand shape tactile data calibration, mapping and standard sample generation based on the body deformation reference as described in any one of claims 1 to 8.

Images