A dexterous hand dynamic adjustment method and device based on tactile feedback and a medium

Abstract

This invention discloses a method, device, and medium for dynamic adjustment of a dexterous hand based on tactile feedback, relating to the field of adaptive control technology. The method includes: acquiring multimodal tactile signals from a bionic tactile array of the dexterous hand and state signals from each joint, performing synchronization alignment and timestamping to form an initial spatiotemporal data block; if reliable historical interaction experience is retrieved, the associated successful adjustment strategy is directly loaded and fine-tuned; otherwise, an identification stimulus sequence is generated and executed, new multimodal tactile signals are acquired, and the probability distribution of object attributes is calibrated online; the calibrated probability distribution of object attributes is input into a hierarchical decision architecture, outputting an adjustment torque command sequence, and the first adjustment torque command is sent to the joint actuator in real time for closed-loop adaptive adjustment. This invention achieves refined semantic parsing of multimodal tactile signals by generating a tactile weight heatmap, transforming high-dimensional tactile data into a probability distribution of object attributes with clear physical meaning.

Description

Technical Field

[0001] This invention relates to the field of adaptive control technology, and in particular to a method, device and medium for dynamic adjustment of a dexterous hand based on tactile feedback. Background Technology

[0002] With the development of dexterous maneuvering technology in robots, adaptive control based on multimodal tactile feedback has become an important research direction. Existing technologies can acquire distributed tactile information through flexible sensor arrays and combine it with joint states to achieve basic grasping force control. At the signal processing level, methods such as temporal data synchronization, feature extraction and fusion are gradually being applied to tactile data processing; at the decision-making level, algorithms such as model predictive control and reinforcement learning are used to generate adjustment strategies, providing fundamental support for the stable operation of dexterous hands in structured environments.

[0003] However, existing methods still have shortcomings when handling tasks involving unknown objects, especially when the physical properties of the object (such as stiffness, coefficient of friction, surface texture, etc.) are uncertain. Control strategies based on fixed-parameter models or shallow feature matching struggle to achieve sophisticated tactile interaction adaptability. This is mainly because the object attribute information contained in the tactile signals is not fully decoupled and utilized, making it impossible to adjust the interaction strategy in real time according to the dynamic changes in the contact state, thus limiting the robustness of operation in unstructured environments. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for dynamic adjustment of dexterous hands based on tactile feedback to solve the problems of insufficient dynamic adjustment accuracy and poor adaptability caused by inadequate utilization of tactile feedback information.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for dynamic adjustment of a dexterous hand based on tactile feedback, comprising: acquiring multimodal tactile signals and joint state signals of a bionic tactile array of the dexterous hand, and performing synchronous alignment and timestamp marking to form an initial spatiotemporal data block; using a spiking neural network to perform bionic encoding on the initial spatiotemporal data block to form a pulse sequence; inputting the pulse sequence into a tactile semantic understanding network to output a tactile weight heatmap; using the tactile weight heatmap to perform soft masking and feature enhancement on the tactile signals; and outputting the probability distribution of object attributes of the currently manipulated object and the confidence score of the current operation. When the confidence score is lower than the preset trigger threshold, the online recognition stimulus mechanism is triggered to retrieve historical interaction experiences similar to the current scene from the historical tactile memory and experience base. If reliable historical interaction experiences are retrieved, the associated successful adjustment strategy is directly loaded and fine-tuned. Otherwise, the recognition stimulus sequence is generated and executed to collect new multimodal tactile signals and calibrate the probability distribution of object attributes online. The calibrated probability distribution of object attributes is input into the hierarchical decision architecture, the adjustment torque command sequence is output, and the first adjustment torque command is sent to the joint actuator in real time for closed-loop adaptive adjustment.

[0008] As a preferred embodiment of the tactile feedback-based dexterity hand dynamic adjustment method of the present invention, the steps for forming the initial spatiotemporal data block are as follows:

[0009] The multimodal signals include normal pressure signals, three-dimensional shear force signals, micro-vibration signals, and temperature signals;

[0010] A precision clock synchronization protocol is used to synchronize and align multimodal signals with the status signals of each joint, and a unified high-precision timestamp is added to each signal to generate standardized data samples with spatiotemporal markers.

[0011] Standardized data samples are encapsulated according to time series and the spatial topology of the dexterous hand bionic tactile array to form initial spatiotemporal data blocks.

[0012] In a preferred embodiment of the tactile feedback-based dexterity hand dynamic adjustment method of the present invention, the step of forming the pulse sequence is as follows:

[0013] Differential pulse coding and collaborative filtering based on pulse temporal dependence plasticity are performed on the initial spatiotemporal data block to generate a primary pulse sequence that is denoised and has enhanced spatiotemporal correlation.

[0014] The primary pulse sequence is input into the adaptive threshold leakage integral firing neuron layer for tactile feature extraction and compression encapsulation, forming a structured pulse data packet;

[0015] The structured pulse data packets are evaluated for encoding quality and confidence flags are generated. The pulse data packets with confidence flags are then used as pulse sequences.

[0016] As a preferred embodiment of the dexterity hand dynamic adjustment method based on tactile feedback described in this invention, the steps for outputting the tactile weight heatmap are as follows:

[0017] Using pulse events at different time steps in the pulse sequence as nodes, edges between nodes are constructed based on the spatial topology of the dexterous hand bionic tactile array and the temporal relationship of the pulse events, forming a spatiotemporal graph structure;

[0018] Graph convolution feature extraction is performed on the spatiotemporal graph structure to generate node-level spatiotemporal features;

[0019] The spatiotemporal features at the node level are mapped back to the spatial layout of the dexterous hand-inspired tactile array, and the spatial domain features are reorganized and normalized to output a tactile weight heatmap.

[0020] As a preferred embodiment of the haptic feedback-based dexterity hand dynamic adjustment method of the present invention, the steps of outputting the probability distribution of the object attributes of the currently manipulated object and the confidence score of the current operation are as follows:

[0021] By fusing tactile weighted heatmaps with pulse sequences, adaptive feature selection and noise suppression are achieved through a learnable gating mechanism, generating spatiotemporal features after soft masking.

[0022] The soft-masked spatiotemporal features are input into the spatiotemporal pyramid encoder for multi-scale feature extraction. Dynamic features and spatial relationships are captured by parallel temporal convolution and spatial graph convolution, respectively. Then, cross-modal attention fusion is performed to form enhanced spatiotemporal features.

[0023] The enhanced spatiotemporal features are input into the probabilistic inference network, and multiple random forward propagations are performed to obtain multiple object attribute probability distributions. The multiple object attribute probability distributions are aggregated and the object attribute probability distribution of the currently operated object is output.

[0024] Calculate the statistical dispersion among the probability distributions of multiple object attributes, and map the statistical dispersion to the confidence score of the current operation.

[0025] As a preferred embodiment of the dexterity hand dynamic adjustment method based on tactile feedback described in this invention, the step of retrieving historical interaction experiences similar to the current scene from a historical tactile memory and experience database is as follows:

[0026] Monitor the confidence score of the current operation, and trigger the online identification incentive mechanism when the confidence score is lower than the preset trigger threshold;

[0027] The tactile weight heatmap, object attribute probability distribution, and confidence score of the current scene are fused into multimodal features and encoded into the current scene feature vector.

[0028] Calculate the similarity score between the current scene feature vector and the feature vectors of each historical scene in the historical tactile memory and experience database;

[0029] Historical interaction experiences are sorted according to similarity scores, and the historical interaction experience with the highest similarity score is retrieved.

[0030] As a preferred embodiment of the tactile feedback-based dexterity hand dynamic adjustment method of the present invention, the online calibration of the object attribute probability distribution includes the following steps.

[0031] If reliable historical interaction experiences are retrieved, the successful adjustment strategy associated with the historical interaction experiences is adaptively fine-tuned based on the meta-reinforcement learning framework to form an adjustment strategy suitable for the current scenario.

[0032] If no reliable historical interaction experience is found, an identification stimulus sequence is generated based on an online reinforcement learning framework, and the identification stimulus sequence is sent to the dexterous hand for execution to collect new multimodal tactile signals;

[0033] By using an online variational Bayesian inference method, new multimodal tactile signals are fused with historical prior knowledge to calibrate the probability distribution of object attributes in real time.

[0034] As a preferred embodiment of the haptic feedback-based dexterity hand dynamic adjustment method of the present invention, the steps of inputting the calibrated object attribute probability distribution into a hierarchical decision architecture, outputting an adjustment torque command sequence, and sending the first adjustment torque command to the joint actuator in real time for closed-loop adaptive adjustment are as follows.

[0035] The calibrated object attribute probability distribution, real-time haptic feedback signal and joint state signal are fused in multiple levels to generate an enhanced state representation.

[0036] The enhanced state representation and object attribute probability distribution are input into the hierarchical decision architecture. Multiple candidate grasping strategies are generated at the policy layer, and the expected utility of each candidate grasping strategy is evaluated through forward simulation. At the trajectory layer, the motion trajectory in the future time domain is optimized based on the expected utility, and an adjustment torque command sequence is generated.

[0037] The first torque adjustment command is extracted from the torque adjustment command sequence and sent to the joint actuator in real time for closed-loop adaptive adjustment.

[0038] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the haptic feedback-based dexterity hand dynamic adjustment method described in the first aspect of the present invention.

[0039] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the haptic feedback-based dexterity hand dynamic adjustment method described in the first aspect of the present invention.

[0040] The beneficial effects of this invention are as follows: by generating tactile weight heatmaps, refined semantic analysis of multimodal tactile signals is achieved, transforming high-dimensional tactile data into probability distributions of object attributes with clear physical meaning; improving the online identification accuracy of key physical attributes such as stiffness and friction coefficient; and laying a solid foundation for achieving robust decision-making and safe strategy switching based on confidence level, thereby enhancing the perceptual interpretability and decision reliability of dexterous hands in tasks involving the manipulation of unknown objects. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of a dexterous hand dynamic adjustment method based on haptic feedback.

[0043] Figure 2 A flowchart for forming a pulse sequence.

[0044] Figure 3 This is a flowchart for outputting the probability distribution and confidence scores of object attributes.

[0045] Figure 4 This is a flowchart for online identification and calibration.

[0046] Figure 5 This is a heatmap of tactile weighting data.

[0047] Figure 6 This is a graph comparing the changes in confidence level over time. Detailed Implementation

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0051] Reference Figures 1-6 This is one embodiment of the present invention, which provides a method for dynamic adjustment of dexterity based on haptic feedback, including the following steps:

[0052] S1. Collect multimodal tactile signals and joint status signals of the dexterous hand bionic tactile array, and perform synchronous alignment and timestamp marking to form an initial spatiotemporal data block; use a spiking neural network to perform bionic encoding on the initial spatiotemporal data block to form a pulse sequence.

[0053] S1.1: Multimodal signals include normal pressure signals, three-dimensional shear force signals, micro-vibration signals, and temperature signals;

[0054] It should be noted that the normal pressure signal refers to the tactile sensor output caused by the force perpendicular to the contact surface, reflecting the magnitude of the normal pressure when grasping or contacting;

[0055] Three-dimensional shear force signal refers to the tactile response of friction or sliding force generated in three mutually perpendicular directions (usually two tangential directions and one torsional direction) within the contact surface;

[0056] Micro-vibration signals refer to the high-frequency dynamic response of tactile sensors caused by high-frequency micro-mechanical disturbances during the surface texture of an object, sliding, or operation.

[0057] Temperature signals refer to temperature change information related to the heat conduction of a contacted object detected by a tactile sensor, used to sense the thermal properties of materials or the thermal state of the environment.

[0058] S1.2: Employ a precision clock synchronization protocol to synchronize and align multimodal signals with the status signals of each joint, and assign a unified high-precision timestamp to each signal to generate standardized data samples with spatiotemporal markers;

[0059] Specifically, sensor interface circuits supporting hardware-level timestamps are deployed in the dexterous hand body and joint actuators, enabling the normal pressure signal, three-dimensional shear force signal, micro-vibration signal, temperature signal, and joint status signals to have an initial time reference provided by a local high-stability crystal oscillator at the start of acquisition. A precision clock synchronization protocol is used to periodically exchange synchronization messages between all sensing nodes and joint encoders. The round-trip time delay of these messages is used to estimate the clock offset and drift of each node, and the local time reference is dynamically corrected accordingly, ensuring that all signal sources share the same time reference system. Based on this, a unified high-precision timestamp, generated according to the corrected local time, is added to each frame of acquired normal pressure signal, three-dimensional shear force signal, micro-vibration signal, temperature signal, and joint status signal. The normal pressure signal, three-dimensional shear force signal, micro-vibration signal, temperature signal, and joint status signals with the unified high-precision timestamps are strictly aligned with the timestamps to form standardized data samples with spatiotemporal markers.

[0060] It should be noted that the joint status signals refer to the physical quantities of angle, angular velocity, and angular acceleration generated by each joint in the dexterous hand during movement.

[0061] S1.3: Encapsulate standardized data samples according to time series and the spatial topology of the dexterous hand bionic tactile array to form initial spatiotemporal data blocks.

[0062] Specifically, standardized data samples with spatiotemporal markers are arranged in chronological order to form a time series. Simultaneously, based on the actual physical arrangement of each sensor in the dexterous hand bionic tactile array on the palm and finger joint surfaces, a corresponding two-dimensional or three-dimensional spatial topology is established. The normal pressure signal, three-dimensional shear force signal, micro-vibration signal, temperature signal, and joint state signal contained in each moment of the time series are mapped to the corresponding sensor nodes in the spatial topology of the dexterous hand bionic tactile array. Finally, the time dimension and spatial topology dimension are integrated and encapsulated into an initial spatiotemporal data block.

[0063] S1.4: Perform differential pulse coding and collaborative filtering based on pulse temporal dependence plasticity on the initial spatiotemporal data block to generate a primary pulse sequence that is denoised and has enhanced spatiotemporal correlation;

[0064] Specifically, the normal pressure signal, three-dimensional shear force signal, micro-vibration signal, and temperature signal of each sensor in the initial spatiotemporal data block are differentiated at consecutive time points. The numerical changes at adjacent time points are converted into pulse events by the numerical changes exceeding the preset pulse threshold, forming differential pulse codes. The pulse events obtained by differential pulse codes are organized into pulse sequences according to their occurrence time and spatial location. Based on the pulse temporal dependence plasticity rule, the pulse transmission intensity between adjacent sensors is adjusted according to the time order of pulse arrival. This makes pulse events that are close in time and adjacent in space mutually reinforce each other, while pulse events that are isolated in time or space are suppressed. Finally, a primary pulse sequence with denoising and enhanced spatiotemporal correlation is obtained.

[0065] It should be noted that the pulse timing-dependent plasticity rule means that when the pulse event of one sensor occurs earlier than the pulse event of the adjacent sensor, the pulse transmission strength between the two is enhanced, and vice versa, thus strengthening the pulse activity pattern with spatiotemporal consistency.

[0066] The pulse threshold is set based on the signal baseline fluctuation of the dexterous hand bionic tactile array under static and typical dynamic contact conditions. The specific steps are as follows: First, the background noise of the multimodal signal is collected under no external force conditions, and its amplitude standard deviation is calculated. Then, dynamic responses are collected during typical operations such as standard grasping, sliding, and light touching to determine the minimum signal change caused by an effective tactile event. Several times the standard deviation of the background noise is compared with this minimum change, and the larger value is taken as the initial threshold. This threshold is then optimized through multiple rounds of repeated grasping experiments to finally determine a stable and reliable pulse threshold. An exemplary value range is: normal pressure 0.5%–3% of full scale, three-dimensional shear force 1%–5% of full scale, micro-vibration 2–5 times the RMS of the background noise, and temperature change 0.1℃–0.5℃. This value aims to reliably respond to real contact or property changes while suppressing spurious signals such as thermal drift, electromagnetic interference, and mechanical vibration.

[0067] S1.5: Input the primary pulse sequence into the adaptive threshold leakage integral firing neuron layer for tactile feature extraction and compression encapsulation to form a structured pulse data packet;

[0068] Specifically, in the adaptive threshold leakage integral firing neuron layer, each neuron corresponds to a spatial location in the dexterous hand bionic tactile array, receiving primary pulse events from that location and neighboring locations. The neuron accumulates membrane potential for the arriving pulse events, and generates a new firing pulse when the membrane potential exceeds the current adaptive threshold. After firing, the membrane potential is reset, and the adaptive threshold is dynamically adjusted according to the recent firing frequency. The membrane potential leakage mechanism suppresses the potential residue when there is no pulse for a long time, so that the neuron only responds to the continuously occurring spatiotemporal pulse pattern. Finally, the generated firing pulses are organized according to time and spatial location to form a structured pulse data packet.

[0069] It should be noted that the adaptive threshold leakage integral firing neuron layer is a pulse coding structure composed of multiple neurons with membrane potential accumulation, leakage, firing and threshold dynamic adjustment capabilities, used for spatiotemporal feature extraction and sparsification encapsulation of primary pulse sequences.

[0070] The adaptive threshold is set based on the historical firing activity of each neuron in the adaptive threshold leakage integral firing neuron layer. The specific steps are as follows: record the number of firings of each neuron within a unit time window to form a recent firing frequency statistic; dynamically adjust the threshold according to the frequency change—increase the threshold to suppress over-response when the firing frequency increases, and decrease the threshold to maintain sensitivity to weak signals when the frequency decreases, so that the threshold can stably reflect the activity level of the current tactile input and match the spatiotemporal density of the primary pulse sequence; the typical value range of the adaptive threshold is 20% to 80% of the full scale of the membrane potential; below 20% is prone to causing neurons to over-respond to noise, reducing the signal-to-noise ratio of the pulse data; above 80% may miss continuous but moderately strong effective tactile features, weakening the subsequent ability to distinguish object attributes and operation states.

[0071] S1.6: Evaluate the coding quality of structured pulse data packets and generate confidence flags, then use the pulse data packets with confidence flags as pulse sequences.

[0072] Specifically, the pulse firing density and spatiotemporal continuity at each spatial location within each time window are statistically analyzed. Combined with the adaptive threshold change trend of the adaptive threshold leakage integral firing neuron layer within the window, it is determined whether the pulse event exhibits a stable, repetitive, or coordinated pattern. Based on the sparsity of pulse firing, cross-unit synchronization, and the degree of matching with historical typical tactile response patterns, the reliability level of the tactile information represented by the current structured pulse data packet is determined. According to the reliability level, a corresponding confidence flag is generated, which is attached to the structured pulse data packet in the form of discrete levels. Finally, the structured pulse data packet with confidence flags is used as a pulse sequence.

[0073] S2. Input the pulse sequence into the tactile semantic understanding network, output the tactile weight heatmap, use the tactile weight heatmap to perform soft masking and feature enhancement on the tactile signal, and output the probability distribution of the object attributes of the currently operated object and the confidence score of the current operation.

[0074] S2.1: Using pulse events at different time steps in the pulse sequence as nodes, construct the edges between nodes based on the spatial topology of the dexterous hand bionic tactile array and the temporal relationship of the pulse events to form a spatiotemporal graph structure;

[0075] Specifically, each pulse event occurring at each time step in the pulse sequence is considered an independent node. Each node contains the spatial coordinates of the corresponding sensor in the dexterous hand bionic tactile array, the pulse occurrence time, and a confidence level. Based on the spatial topology of the dexterous hand bionic tactile array, spatial edges are established between nodes corresponding to physically adjacent sensors within the same time step. Simultaneously, based on the temporal relationship of pulse events, temporal edges are established between nodes generated by the same sensor or adjacent sensors between consecutive time steps. Spatial edges reflect the spatial adjacency of tactile signals on the surface of the dexterous hand, while temporal edges reflect the temporal continuity of the dynamic evolution of tactile sensation. Through the joint connection of spatial edges and temporal edges, a spatiotemporal graph structure is formed.

[0076] S2.2: Perform graph convolution feature extraction on the spatiotemporal graph structure to generate node-level spatiotemporal features;

[0077] Specifically, a feature vector is initialized for each node. This feature vector contains the confidence flag of the corresponding pulse event and the spatial position encoding of the sensor to which it belongs in the dexterous hand bionic tactile array. According to the spatial and temporal edges defined in the spatiotemporal graph structure, each node and its directly connected neighboring nodes are aggregated using a weighted summation method, with the weights determined by the confidence flags of the neighboring nodes. After one round of aggregation, the aggregation result is combined with the original node features to form the updated node features. The aggregation and combination operations are repeated several times to allow each node to gradually integrate the spatiotemporal context information within its multi-hop neighborhood. The final node features are the node-level spatiotemporal features.

[0078] It should be noted that graph convolution, as a core component of the tactile semantic understanding network, is optimized end-to-end based on a large-scale labeled tactile interaction dataset during the pre-training stage. This dataset contains multimodal tactile pulse sequences and their corresponding semantic labels collected under known object properties (such as material, stiffness, coefficient of friction, etc.) and standard operating actions (such as grasping, sliding, pressing). During training, the supervision signal of the tactile weight heatmap is used as the target, and the consistency between the output heatmap and the target heatmap is constrained by the L1 or MSE loss function. The weight parameters of the graph convolution layer are jointly optimized through backpropagation, thereby learning the ability to extract spatiotemporal features sensitive to semantically related regions and obtaining a pre-trained tactile semantic understanding network.

[0079] S2.3: Map the spatiotemporal features at the node level back to the spatial layout of the dexterous hand bionic tactile array, perform spatial domain feature recombination and normalization processing, and output a tactile weight heatmap.

[0080] Specifically, the spatiotemporal features at the node level are mapped point by point back to the original spatial layout grid according to the spatial coordinates of the corresponding sensors in the dexterous hand bionic tactile array, forming a feature distribution map consistent with the geometry of the dexterous hand bionic tactile array. On this feature distribution map, the feature values ​​at all positions are aligned to the maximum value in the global range, so that the features at each position are in a uniform dimension. Then, the feature values ​​are normalized to a fixed range by linear scaling to eliminate the amplitude offset caused by differences in pulse density or confidence. The final two-dimensional or three-dimensional visualization map is the tactile weight heatmap.

[0081] It should be noted that, Figure 5 Using the tactile array index as the horizontal and vertical axes, the color bars represent the normalized "weights". The brighter the color, the greater the contribution of that spatial location to the discrimination / decision in the current interaction, and the darker the color, the smaller the contribution. Therefore, it "compresses" the high-dimensional tactile spatiotemporal features into an interpretable distribution with clear spatial physical meaning, and allows for a direct view of the key areas that the dexterous hand "focuses" on the contact surface.

[0082] S2.4: The tactile weight heatmap is fused with the pulse sequence, and the adaptive selection of features and noise suppression are achieved through a learnable gating mechanism to generate spatiotemporal features after soft masking;

[0083] Specifically, the haptic weight heatmap is aligned with the confidence flags and timestamps of each node in the pulse sequence, so that the weight of each spatial location in the haptic weight heatmap corresponds to the pulse event at the same location and time in the pulse sequence. The learnable gating mechanism consists of a parameter array isomorphic to the haptic weight heatmap, whose initial values ​​are provided by the haptic weight heatmap and adjusted according to the task objective during training. Through this gating mechanism, the spatiotemporal features at the node level of each node in the pulse sequence are modulated element-wise. Features corresponding to high-weight regions are preserved or enhanced, while features corresponding to low-weight regions are attenuated. The final modulation result is the spatiotemporal feature after soft masking.

[0084] It should be noted that the learnable gating mechanism is initialized with the tactile weight heatmap as the initial parameter during the pre-training stage, and its structure is isomorphic to the heatmap. On the labeled tactile-manipulation task dataset, through end-to-end training, with task objectives such as object attribute recognition or grasping stability as supervision signals, the gating parameters are automatically adjusted using backpropagation, so that the impulse features of highly semantically relevant regions are enhanced and low-relevance or noisy regions are suppressed, thereby optimizing the discriminative power of the spatiotemporal features after soft masking for downstream tasks, and obtaining a well-trained learnable gating mechanism.

[0085] S2.5: Input the soft-masked spatiotemporal features into the spatiotemporal pyramid encoder for multi-scale feature extraction. Dynamic features and spatial relationships are captured by parallel temporal convolution and spatial graph convolution, respectively. Then, cross-modal attention fusion is performed to form enhanced spatiotemporal features.

[0086] Specifically, the soft-masked spatiotemporal features are fed into a spatiotemporal pyramid encoder. At each scale of this encoder, local patterns are extracted along the time dimension for features at the same spatial location at consecutive time steps through temporal convolution, capturing dynamically changing features. Simultaneously, spatial graph convolution aggregates neighborhood information for features at adjacent spatial locations in the dexterous hand bionic tactile array at the same time, capturing spatial structural relationships. Temporal convolution and spatial graph convolution are applied to the soft-masked spatiotemporal features in parallel, each generating temporal and spatial feature branches at the corresponding scale. The temporal and spatial feature branches at the same scale are fed into a cross-modal attention mechanism, and fusion weights are assigned based on the semantic relevance between them, integrating them into an enhanced representation. The enhanced representations at multiple scales are upsampled or concatenated to form the final enhanced spatiotemporal features.

[0087] It should be noted that the spatiotemporal pyramid encoder, together with graph convolution, is incorporated into the overall pre-training framework of the tactile semantic understanding network. On the labeled tactile dataset, the encoder is jointly trained with graph convolution, gating mechanism and probabilistic inference network. By minimizing the cross-entropy or KL divergence loss between the final object attribute prediction distribution and the real label, gradients are backpropagated to simultaneously optimize the temporal convolution kernel, spatial graph convolution parameters and cross-modal attention weights in the spatiotemporal pyramid, ensuring the effectiveness and task orientation of multi-scale spatiotemporal features.

[0088] S2.6: Input the enhanced spatiotemporal features into the probabilistic inference network, perform multiple random forward propagations to obtain multiple object attribute probability distributions, aggregate the multiple object attribute probability distributions, and output the object attribute probability distribution of the currently operated object.

[0089] Specifically, the enhanced spatiotemporal features are fed into a probabilistic inference network. During each forward propagation, this network introduces randomness into the internal connections or node activation states, causing the same enhanced spatiotemporal feature to generate different internal response paths in multiple independent forward propagations. Each forward propagation generates a corresponding object attribute probability distribution, representing the probability of the manipulated object in attribute categories such as material, shape, stiffness, and surface texture. Multiple randomized forward propagations are repeated to obtain multiple object attribute probability distributions. These multiple object attribute probability distributions are then aggregated by averaging the probability values ​​for each attribute category, forming a stable and robust comprehensive estimate, ultimately yielding the object attribute probability distribution of the current manipulated object.

[0090] It should be noted that the pre-training process of the probabilistic inference network is based on a large number of labeled tactile interaction datasets, which contain standardized data samples collected under known object attributes. Enhanced spatiotemporal features are used as input to the probabilistic inference network, and real object attribute labels are used as supervision targets. Random forward propagation and loss functions guide the network parameter updates. During training, the Monte Carlo Dropout mechanism is used to preserve the randomness of inference, and cross-entropy or KL divergence is used to measure the difference between the predicted object attribute probability distribution and the real labels. After multiple rounds of iterative optimization, the probabilistic inference network can output a probability distribution consistent with the real attributes when faced with new samples, thus obtaining a well-trained probabilistic inference network.

[0091] S2.7: Calculate the statistical dispersion among the probability distributions of multiple object attributes and map the statistical dispersion to the confidence score of the current operation.

[0092] Specifically, the statistical dispersion is converted into the confidence score of the current operation through a monotonically decreasing normalization mapping relationship, so that the lower the statistical dispersion, the higher the confidence score, and vice versa.

[0093] The statistical dispersion among the probability distributions of multiple object attributes is calculated using the following expression:

[0094] ;

[0095] In the formula, Represents the statistical dispersion among the probability distributions of multiple object attributes; This represents the total number of random forward propagations performed on the same enhanced spatiotemporal feature; This represents the number of all ordered and distinct pairs of object attribute probability distributions. It represents the reciprocal of the number of all ordered and distinct object attribute probability distributions; Indicates the first Index of the probability distribution of object attributes generated by the random forward propagation; Indicates the first Index of the probability distribution of object attributes generated by the random forward propagation; This means that during the summation process, comparisons between itself and itself are excluded, and only the differences between the results of different forward propagations are calculated. Indicates the first The probability distribution of object attributes output by the next random forward propagation; Indicates the first The probability distribution of object attributes output by the next random forward propagation; Indicates calculation and The Jensen-Shannon divergence values ​​between them.

[0096] It should be noted that, Figure 6 With time as the horizontal axis and confidence score as the vertical axis, the confidence curves, confidence observation points, and slip event markers of the two schemes (Group A / Group B) are overlaid. The main plot at the top shows the overall fluctuation of confidence and the location of slip points in the complete time series, while the enlarged partial plot at the bottom focuses on the time window "near slip" to more clearly compare the differences and fluctuation details of confidence between the two schemes in the critical risk period.

[0097] S3. When the confidence score is lower than the preset trigger threshold, the online recognition incentive mechanism is triggered to retrieve historical interaction experiences similar to the current scene from the historical tactile memory and experience base.

[0098] S3.1: Monitor the confidence score of the current operation. When the confidence score is lower than the preset trigger threshold, trigger the online identification incentive mechanism.

[0099] Specifically, the confidence score of the current operation is continuously acquired and compared with a preset trigger threshold. When the confidence score of the current operation is lower than the preset trigger threshold, the online recognition incentive mechanism is activated. When the confidence score is higher than the preset trigger threshold, the current perception result is determined to be sufficiently reliable.

[0100] It should be noted that the trigger threshold is set based on the statistical characteristics of the confidence score distribution of dexterous hands in typical operational tasks. The specific setting steps are as follows: In a large number of standard grasping, sliding, and adjusting operations involving known objects, the lower limit of the confidence score corresponding to high-reliability operations (such as successful and stable grasping, and adjustment without slippage) is statistically analyzed, and combined with the upper limit of the confidence score corresponding to low-reliability operations (such as object slippage, and misjudging material), a boundary region that can effectively distinguish between reliable and unreliable perception states is determined. Within this boundary region, the value that optimally combines the triggering frequency of the online identification stimulus mechanism with the task success rate is selected through cross-validation as the trigger threshold. An exemplary value range is 0.4 to 0.7. The basis for this value is to ensure timely activation of the supplementary cognitive mechanism when the uncertainty of actual perception is high, while avoiding frequent triggering of unnecessary exploratory actions when perception is sufficient, thus affecting operational efficiency. A value below 0.4 will cause the online identification stimulus mechanism to be overly sensitive, activating frequently even when tactile information is sufficient, increasing latency and interfering with the normal operational process; a value above 0.7 will cause the mechanism to respond sluggishly, failing to intervene in time when supplementary information is truly needed, leading to incorrect decisions or grasping failures.

[0101] S3.2: Perform multimodal feature fusion on the tactile weight heatmap, object attribute probability distribution, and confidence score of the current scene, and encode them into a feature vector of the current scene;

[0102] Specifically, the tactile weight heatmap, object attribute probability distribution, and confidence score of the current scene are stitched and aligned. The tactile weight heatmap is flattened into a one-dimensional vector according to the spatial layout of the dexterous hand bionic tactile array, preserving the spatial correspondence of each sensor. The object attribute probability distribution is arranged in category order as another independent vector, and the confidence score is expanded into a constant padding vector matching the length of the above vectors. The three vectors are combined bit by bit along the feature dimension to form a unified multimodal joint representation. This joint representation is linearly projected and dimensionally regularized through a fixed-structure fully connected coding layer to generate the current scene feature vector.

[0103] S3.3: Calculate the similarity score between the current scene feature vector and the feature vectors of each historical scene in the historical tactile memory and experience base. The expression is:

[0104] ;

[0105] In the formula, Indicates the current scene feature vector and the first... Similarity scores between feature vectors of historical scenes; Represents the feature vector index of historical scenes in the historical tactile memory and experience database; Represents the feature vector of the current scene; Indicates the first Each historical scene feature vector; The Euclidean norm of the feature vector of the current scene; Indicates the first The Euclidean norm of the feature vectors of a historical scene.

[0106] S3.4: Sort historical interaction experiences according to similarity scores and retrieve the historical interaction experience with the highest similarity score.

[0107] Specifically, based on the similarity score between the current scene feature vector and the feature vectors of each historical scene in the historical tactile memory and experience base, all historical interaction experiences are sorted in descending order of similarity score from high to low. During the sorting process, each historical interaction experience is bound to its corresponding similarity score. After sorting, the historical interaction experience at the top of the sequence is selected as the historical interaction experience with the highest similarity score.

[0108] It should be noted that when the similarity score corresponding to the historical interaction experience with the highest similarity score is greater than or equal to the preset reliability threshold, it is determined that a reliable historical interaction experience has been retrieved; when the similarity score corresponding to the historical interaction experience with the highest similarity score is less than the preset reliability threshold, it is determined that no reliable historical interaction experience has been retrieved.

[0109] The reliability threshold is set based on the statistical distribution of similarity scores of known successful operation cases in historical tactile memory and experience base. The specific setting steps are as follows: Calculate the highest similarity score that each successful case can match, forming a set of similarity scores for successful operations; based on this, analyze the lower quartile of this set or set a conservative boundary value covering more than 90% of successful cases as the initial reliability threshold; evaluate the impact of this threshold on the success rate and misuse rate of strategy reuse in actual tasks through cross-validation to determine the reliability threshold; the exemplary value range is 0.55 to 0.75; the basis for the value is to ensure that the retrieved historical interaction experience has sufficient consistency with the current scene in semantics and physical response, effectively supporting strategy transfer while avoiding erroneous execution due to overgeneralization; a value below 0.55 will make the judgment condition too lenient, leading to the loading of irrelevant or inapplicable successful adjustment strategies, causing operational errors or object damage; a value above 0.75 will make the judgment condition too strict, rejecting even reusable experience, frequently triggering identification stimulus sequences, reducing operational efficiency and increasing unnecessary exploration actions.

[0110] S4. If reliable historical interaction experience is retrieved, the associated successful adjustment strategy is directly loaded and fine-tuned; otherwise, the identification stimulus sequence is generated and executed, new multimodal tactile signals are collected, and the probability distribution of object attributes is calibrated online.

[0111] S4.1: If reliable historical interaction experience is retrieved, the successful adjustment strategy associated with the historical interaction experience is adaptively fine-tuned based on the meta-reinforcement learning framework to form an adjustment strategy suitable for the current scenario.

[0112] Specifically, if reliable historical interaction experience is retrieved, the successful adjustment strategy associated with that historical interaction experience is extracted; the successful adjustment strategy is used as the initial policy parameter in the meta-reinforcement learning framework; several trial operations are performed in the current scene, and tactile weight heatmaps, object attribute probability distributions, and confidence scores are collected from the feedback of the dexterous hand bionic tactile array to form the policy evaluation basis for the current scene; based on the degree of matching between the policy evaluation basis and the target operation effect, a small number of gradient updates are performed on the parameters of the successful adjustment strategy under the meta-reinforcement learning framework to adapt the strategy to the contact characteristics and dynamic response of the currently operated object, and the resulting strategy is the adjustment strategy suitable for the current scene.

[0113] It should be noted that reliable historical interaction experience refers to historical operation records that meet the following conditions and result in successful operation: the interaction fully achieves the expected task objectives (such as stable grasping, slip-free control, or precise assembly), and no abnormal situations such as object falling, loss of control, or structural damage occur during the process; the perception judgment is consistent and reliable: during the interaction, high-confidence object attribute inference results are continuously output, indicating that tactile semantic understanding has sufficient basis for judging the contact state and object characteristics, rather than based on fuzzy or noise-dominated signals; and it is highly similar to the current scene: the key contextual features such as the object attribute distribution, spatial activation pattern of the tactile weight heatmap, and initial contact configuration corresponding to the historical experience show significant consistency with the current task state in semantics and structure, and are confirmed to be closely related by similarity measures (such as KL divergence between distributions, spatial cosine similarity of heatmaps, etc.).

[0114] Only historical interactions that simultaneously meet all three conditions are considered "reliable" and can be used for strategy reuse and fine-tuning; otherwise, they will be considered as lacking experience to draw upon, and an active exploration mechanism will be initiated instead.

[0115] It should be noted that the pre-training process of the meta-reinforcement learning framework is based on a large-scale historical interaction experience database, in which each experience contains a complete operation trajectory, tactile feedback sequence, joint state, and final operation success or failure label under specific object attributes; the training adopts meta-learning algorithms such as MAML (Model-Agnostic Meta-Learning) or Reptile, with the goal of "quickly adapting to new objects", and optimizes the initial parameters of the policy network so that it can be effectively fine-tuned with only a small number of current scene samples.

[0116] S4.2: If no reliable historical interaction experience is found, generate an identification stimulus sequence based on the online reinforcement learning framework, and send the identification stimulus sequence to the dexterous hand for execution to collect new multimodal tactile signals;

[0117] Specifically, if no reliable historical interaction experience is found, an online reinforcement learning framework is activated. This framework uses the uncertainty of the current object attribute probability distribution as an initial condition to generate a set of recognition stimulus sequences designed to stimulate differences in tactile responses. These sequences include exploratory action instructions such as fingertip micro-sliding, local pressing, and slight twisting. The recognition stimulus sequences are then sent to the dexterous hand for execution. The dexterous hand drives each joint to complete the corresponding action according to the instructions. During the action execution, the dexterous hand's bionic tactile array continuously collects primary impulse events during the contact process and simultaneously generates corresponding tactile weight heatmaps, updated object attribute probability distributions, and confidence scores, forming new multimodal tactile signals.

[0118] S4.3: Using online variational Bayesian inference, new multimodal tactile signals are fused with historical prior knowledge to calibrate the probability distribution of object attributes in real time.

[0119] Specifically, using the online variational Bayesian inference method, the tactile weight heatmap, object attribute probability distribution, and confidence score in the new multimodal tactile signal are jointly modeled with the prior distribution of object attributes carried by similar historical interaction experiences in the historical tactile memory and experience base. The object attribute probability distribution corresponding to successful operations in historical interaction experiences is used as the prior distribution, and the current observation evidence reflected by the new multimodal tactile signal is used as the likelihood information. By iteratively optimizing the variational distribution parameters, the posterior distribution is made to approximate the real object attribute distribution. The final posterior distribution is the calibrated object attribute probability distribution.

[0120] S5. Input the calibrated object attribute probability distribution into the hierarchical decision architecture, output the adjustment torque command sequence, and send the first adjustment torque command to the joint actuator in real time for closed-loop adaptive adjustment.

[0121] S5.1: The calibrated object attribute probability distribution, real-time haptic feedback signal and joint state signal are fused in multiple levels to generate an enhanced state representation;

[0122] Specifically, the calibrated object attribute probability distribution is organized into a high-level semantic vector according to the attribute category order; the real-time tactile feedback signal is flattened into a mid-level spatial perception vector according to the spatial layout of the dexterous hand bionic tactile array; and the joint state signals are arranged into a low-level motion state vector according to the joint number order. The three vectors are connected sequentially in the feature dimension to form a joint representation. The joint representation is nonlinearly mapped and dimensionally compressed through a fixed-structure multilayer perception encoder to generate an enhanced state representation.

[0123] S5.2: Input the enhanced state representation and object attribute probability distribution into the hierarchical decision architecture, generate multiple candidate grasping strategies at the policy layer, evaluate the expected utility of each candidate grasping strategy through forward simulation, optimize the motion trajectory in the future time domain based on the expected utility at the trajectory layer, and generate an adjustment torque command sequence.

[0124] Specifically, the enhanced state representation and object attribute probability distribution are fed into a hierarchical decision architecture. In the strategy layer, based on the semantic information such as material, shape, and stiffness described by the current object attribute probability distribution, and combined with the tactile and joint state context contained in the enhanced state representation, multiple candidate grasping strategies are generated. Each candidate grasping strategy corresponds to a grasping posture, contact point configuration, and initial force control scheme. For each candidate grasping strategy, a forward simulation is performed in a physical simulation environment. The simulation process, based on the dynamics of the dexterous hand bionic tactile array and the physical characteristics in the object attribute probability distribution, deduces the contact stability, slippage risk, and operation success rate during the grasping process, forming the expected utility of the candidate grasping strategy. In the trajectory layer, the candidate grasping strategy with the highest expected utility is selected as the target, and its corresponding grasping configuration is used as the terminal constraint. Combined with the current joint state signals, a joint motion trajectory that meets the dynamic feasibility is planned in the future time domain, and an adjustment torque command sequence matching the trajectory is generated.

[0125] It should be noted that the pre-training data for the hierarchical decision architecture (including the candidate policy generator in the policy layer and the trajectory optimizer in the trajectory layer) comes from tens of thousands of successful / failed grasping tasks completed in a simulation environment. The input is the calibrated object attribute distribution and multimodal state signal, and the supervision signal is the manually demonstrated trajectory or the optimal policy derived through inverse reinforcement learning. During training, the policy layer uses classification or generative loss (such as cross-entropy or Wasserstein distance), and the trajectory layer uses trajectory tracking error and dynamic constraint loss. The two are jointly optimized through a shared encoder and joint backpropagation to ensure that the generated candidate policy is semantically reasonable and the trajectory dynamics are feasible.

[0126] S5.3: Extract the first adjustment torque command from the adjustment torque command sequence and send it to the joint actuator in real time for closed-loop adaptive adjustment.

[0127] Specifically, the first adjustment torque command located at the beginning of the time series is extracted from the adjustment torque command sequence. This command corresponds to the target torque value of each joint in the current control cycle. The first adjustment torque command is sent to the actuators of each joint of the dexterous hand through the real-time communication interface. Each joint actuator drives the motor to output the corresponding torque according to the first adjustment torque command, and combines the current position and speed feedback in the status signals of each joint to form a closed-loop adaptive adjustment, so that the dexterous hand can dynamically respond to changes in object properties and tactile feedback during contact.

[0128] This embodiment also provides a computer device applicable to the dynamic adjustment method of dexterous hand based on tactile feedback, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the dynamic adjustment method of dexterous hand based on tactile feedback as proposed in the above embodiment.

[0129] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0130] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the method for dynamic adjustment of a dexterous hand based on haptic feedback as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0131] In summary, this invention achieves refined semantic analysis of multimodal tactile signals by generating tactile weighted heatmaps, transforming high-dimensional tactile data into probability distributions of object attributes with clear physical meaning; it improves the online identification accuracy of key physical attributes such as stiffness and friction coefficient; and it lays a solid foundation for achieving robust decision-making and safe strategy switching based on confidence levels, thereby enhancing the perceptual interpretability and decision-making reliability of dexterous hands in tasks involving the manipulation of unknown objects.

[0132] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamic adjustment of a dexterous hand based on haptic feedback, characterized in that: include, Multimodal tactile signals and joint status signals of the dexterous hand bionic tactile array are collected, and synchronized and timestamped to form an initial spatiotemporal data block. A spiking neural network is used to biomimeticly encode the initial spatiotemporal data block to form a spiking sequence; The pulse sequence is input into the tactile semantic understanding network, which outputs a tactile weight heatmap. The tactile weight heatmap is used to perform soft masking and feature enhancement on the tactile signal, and outputs the probability distribution of the object attributes of the currently operated object and the confidence score of the current operation. When the confidence score is lower than the preset trigger threshold, the online recognition incentive mechanism is triggered to retrieve historical interaction experiences similar to the current scene from the historical tactile memory and experience base. If reliable historical interaction experience is retrieved, the associated successful adjustment strategy is directly loaded and fine-tuned; otherwise, an identification stimulus sequence is generated and executed, new multimodal tactile signals are collected, and the probability distribution of object attributes is calibrated online. The calibrated object attribute probability distribution is input into the hierarchical decision architecture, which outputs a sequence of adjustment torque commands and sends the first adjustment torque command to the joint actuator in real time for closed-loop adaptive adjustment.

2. The method for dynamic adjustment of dexterous hand based on haptic feedback as described in claim 1, characterized in that: The steps for forming the initial spatiotemporal data block are as follows: The multimodal signals include normal pressure signals, three-dimensional shear force signals, micro-vibration signals, and temperature signals; A precision clock synchronization protocol is used to synchronize and align multimodal signals with the status signals of each joint, and a unified high-precision timestamp is added to each signal to generate standardized data samples with spatiotemporal markers. Standardized data samples are encapsulated according to time series and the spatial topology of the dexterous hand bionic tactile array to form initial spatiotemporal data blocks.

3. The method for dynamic adjustment of dexterous hand based on haptic feedback as described in claim 2, characterized in that: The steps for forming the pulse sequence are as follows: Differential pulse coding and collaborative filtering based on pulse temporal dependence plasticity are performed on the initial spatiotemporal data block to generate a primary pulse sequence that is denoised and has enhanced spatiotemporal correlation. The primary pulse sequence is input into the adaptive threshold leakage integral firing neuron layer for tactile feature extraction and compression encapsulation, forming a structured pulse data packet; The structured pulse data packets are evaluated for encoding quality and confidence flags are generated. The pulse data packets with confidence flags are then used as pulse sequences.

4. The method for dynamic adjustment of dexterous hand based on haptic feedback as described in claim 3, characterized in that: The steps for generating the output tactile weight heatmap are as follows: Using pulse events at different time steps in the pulse sequence as nodes, edges between nodes are constructed based on the spatial topology of the dexterous hand bionic tactile array and the temporal relationship of the pulse events, forming a spatiotemporal graph structure; Graph convolution feature extraction is performed on the spatiotemporal graph structure to generate node-level spatiotemporal features; The spatiotemporal features at the node level are mapped back to the spatial layout of the dexterous hand-inspired tactile array, and the spatial domain features are reorganized and normalized to output a tactile weight heatmap.

5. The method for dynamic adjustment of dexterous hand based on haptic feedback as described in claim 4, characterized in that: The steps for outputting the probability distribution of the object attributes of the currently operated object and the confidence score of the current operation are as follows: By fusing tactile weighted heatmaps with pulse sequences, adaptive feature selection and noise suppression are achieved through a learnable gating mechanism, generating spatiotemporal features after soft masking. The soft-masked spatiotemporal features are input into the spatiotemporal pyramid encoder for multi-scale feature extraction. Dynamic features and spatial relationships are captured by parallel temporal convolution and spatial graph convolution, respectively. Then, cross-modal attention fusion is performed to form enhanced spatiotemporal features. The enhanced spatiotemporal features are input into the probabilistic inference network, and multiple random forward propagations are performed to obtain multiple object attribute probability distributions. The multiple object attribute probability distributions are aggregated and the object attribute probability distribution of the currently operated object is output. Calculate the statistical dispersion among the probability distributions of multiple object attributes, and map the statistical dispersion to the confidence score of the current operation.

6. The method for dynamic adjustment of dexterous hand based on haptic feedback as described in claim 5, characterized in that: The steps for retrieving similar historical interaction experiences from the historical tactile memory and experience database are as follows: Monitor the confidence score of the current operation, and trigger the online identification incentive mechanism when the confidence score is lower than the preset trigger threshold; The tactile weight heatmap, object attribute probability distribution, and confidence score of the current scene are fused into multimodal features and encoded into the current scene feature vector. Calculate the similarity score between the current scene feature vector and the feature vectors of each historical scene in the historical tactile memory and experience database; Historical interaction experiences are sorted according to similarity scores, and the historical interaction experience with the highest similarity score is retrieved.

7. The method for dynamic adjustment of dexterous hand based on haptic feedback as described in claim 6, characterized in that: The steps for online calibration of the probability distribution of object attributes are as follows: If reliable historical interaction experiences are retrieved, the successful adjustment strategy associated with the historical interaction experiences is adaptively fine-tuned based on the meta-reinforcement learning framework to form an adjustment strategy suitable for the current scenario. If no reliable historical interaction experience is found, an identification stimulus sequence is generated based on an online reinforcement learning framework, and the identification stimulus sequence is sent to the dexterous hand for execution to collect new multimodal tactile signals; By using an online variational Bayesian inference method, new multimodal tactile signals are fused with historical prior knowledge to calibrate the probability distribution of object attributes in real time.

8. The method for dynamic adjustment of dexterous hand based on haptic feedback as described in claim 7, characterized in that: The steps are as follows: inputting the calibrated object attribute probability distribution into the hierarchical decision architecture, outputting an adjustment torque command sequence, and sending the first adjustment torque command to the joint actuator in real time for closed-loop adaptive adjustment. The calibrated object attribute probability distribution, real-time haptic feedback signal and joint state signal are fused in multiple levels to generate an enhanced state representation. The enhanced state representation and object attribute probability distribution are input into the hierarchical decision architecture. Multiple candidate grasping strategies are generated at the policy layer, and the expected utility of each candidate grasping strategy is evaluated through forward simulation. At the trajectory layer, the motion trajectory in the future time domain is optimized based on the expected utility, and an adjustment torque command sequence is generated. The first torque adjustment command is extracted from the torque adjustment command sequence and sent to the joint actuator in real time for closed-loop adaptive adjustment.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the dexterity hand dynamic adjustment method based on haptic feedback as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the dexterous hand dynamic adjustment method based on tactile feedback as described in any one of claims 1 to 8.

Images