Robot dexterous hand bionic sensing method based on pulse generation

Through the combination of dual-channel encoder and flexible materials, the pulse tactile data of the robot's dexterous hand is extracted, which solves the problem of insufficient tactile perception ability in complex environments, realizes efficient object recognition and environmental adaptation, and enhances the tactile perception of the robot's dexterous hand ability.

CN119188748APending Publication Date: 2024-12-27TONGJI UNIV

Patent Information

Application Number
CN202411381410.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-30
Publication Date
2024-12-27

AI Technical Summary

Technical Problem

In complex and unknown environments, the robot's nimble hands lacks tactile perception capabilities, making it difficult to effectively obtain object information, affecting its execution efficiency and accuracy in specific tasks.

Method used

Using a bionic perception method based on pulse generation, the timing and spatial characteristics are extracted through a dual-channel encoder, combined with a multimodal interaction module, the sensory knowledge of objects is realized, and flexible materials and RC circuits are used to simulate skin tactile information to generate pulsed tactile data. .

Benefits of technology

It improves the tactile perception ability of the robot's dexterous hands, enhances the recognition ability of complex objects, is suitable for agile operation and environmental adaptation, simulates the human tactile perception mechanism, reduces the amount of data and calculation, and improves the quality of feature.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119188748A_ABST
    Figure CN119188748A_ABST
Patent Text Reader

Abstract

The invention relates to a robot dexterous hand bionic sensing method based on pulse generation. The method comprises the following steps: acquiring pulse tactile data acquired by a sensing unit of a robot dexterous hand; the pulse tactile data are input into an encoder with two channels, spatial and temporal features are extracted and interactively fused, perception recognition of an object is completed, and the encoder with the two channels comprises a time sequence sub-encoder, a space sub-encoder and a multi-mode interaction module; the step of completing the perception recognition of the object comprises the following steps of: inputting pulse tactile data into a time sequence sub-encoder to extract time sequence characteristics; inputting the pulse tactile data into a spatial sub-encoder to extract spatial features; and inputting the time sequence features and the spatial features into a multi-modal interaction module for feature interaction to obtain spatial and temporal features, processing the spatial and temporal features by adopting a control operation, and finally outputting a perception recognition result of the object. Compared with the prior art, the method has the advantages of efficiently processing the tactile cognition task and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a bionic perception method for a robot's dexterous hand based on pulse generation. Background Art

[0002] In unknown and unstructured environments, robots use a range of perception methods to build and improve their understanding of the world. Especially in complex unknown environments, robots do not understand the external environment or objects, but can physically interact with the surrounding environment to obtain the desired information and thus build their relevant representations. Accurately estimating an unknown object or environmental information is one of the challenges in robot embodied intelligence applications, which can be used for subsequent tasks with different purposes, such as pushing, grasping, and dexterous rotation.

[0003] Robots' embodied perception has great potential in replacing human operation tasks. In some special environments, such as space exploration, deep-sea exploration or disaster relief, robots need to perform tasks instead of humans. In these cases, the robot's dexterous hands and tactile perception are particularly important. Tactile perception can not only provide information about the shape, texture, temperature and other information of objects, but also help robots better adapt to and understand their working environment. For example, through tactile feedback, robots can judge the hardness, temperature and texture of objects.

[0004] In order to better understand the robot's embodied information from a human perspective, the robot's tactile perception system is also evolving to simulate human tactile perception. The development of bionic tactile perception is very critical. However, since tactile data is usually high-dimensional and sparse, this poses a challenge to the efficiency and accuracy of environmental cognition. Therefore, a perception method is urgently needed to overcome the defects of the above-mentioned existing technologies. Summary of the invention

[0005] The purpose of the present invention is to provide a bionic perception method for a robot dexterous hand based on pulse generation, which can effectively improve the tactile perception ability.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A bionic perception method for a robot dexterous hand based on pulse generation comprises the following steps:

[0008] Acquiring pulse tactile data collected by a sensor unit of the robot's dexterous hand;

[0009] The pulse tactile data is input into an encoder with two channels, and the spatiotemporal features are extracted and interactively fused to complete the perception and recognition of the object, wherein the encoder with two channels includes a temporal sub-encoder, a spatial sub-encoder and a multimodal interaction module;

[0010] The steps of completing the perception and recognition of the object include:

[0011] Inputting the pulse tactile data into the timing sub-encoder to extract timing features;

[0012] Inputting the impulse tactile data into the spatial sub-encoder to extract spatial features;

[0013] The temporal features and spatial features are input into the multimodal interaction module for feature interaction to obtain spatiotemporal features, which are processed using a concatenation operation to ultimately output a perception recognition result of the object.

[0014] Furthermore, the generation process of the pulse tactile data includes:

[0015] Flexible materials are used to meet the bionic flexibility characteristics, and the pressing conduction characteristics are used to meet the bionic time trigger characteristics, so as to construct a flexible tactile sensor as a sensing unit;

[0016] By simulating the mechanism of generating skin tactile information and utilizing the capacitor charging characteristics of the RC circuit, a voltage comparator is designed. When the capacitor is fully charged, a pulse is generated in the comparator. Based on the mechanical structure characteristics of the robot's dexterous hand, a distributed design of flexible tactile sensors is performed to achieve the final generation of pulsed tactile data.

[0017] Furthermore, the capacitor charging principle of the RC circuit is:

[0018] V(t)=V0*(1-e -t / RC )

[0019] Where V(t) is the output voltage, V0 is the input voltage, and RC is the time constant. The time constant changes according to the degree of pressure.

[0020] Furthermore, the temporal sub-encoder includes a linear layer, a temporal position encoding module and a temporal attention feature extraction module, the linear layer is used to perform linear transformation on the pulse tactile data, the temporal position encoding fusion module is used to fuse the linearly transformed pulse tactile data with its position encoding to obtain the pulse tactile data embedded with time information; the temporal attention feature extraction module is used to perform temporal feature extraction on the pulse tactile data embedded with time information.

[0021] Furthermore, in the temporal position coding fusion module, the expression of the position coding is:

[0022]

[0023] In the formula, PE(·) is the position code, pos represents the position index, i.e., the step length, and d modelis the dimension of the word vector, and i is the dimension index.

[0024] Furthermore, the temporal attention feature extraction module uses an attention mechanism to extract temporal features, wherein the expression of the attention mechanism is:

[0025] e i =w (2) (w (1) H+b (1) +b (2) )

[0026]

[0027] In the formula, ei is the feature after nonlinear transformation of the input feature, w (1) 、w (2) are the transformation matrices, b (1) 、b (2) are the bias parameters, H is the input feature, αi is the attention parameter, ek is the feature sequence, n is the total number of input features, C t is the output feature, h i is the input feature, and H is the feature at the current moment.

[0028] Furthermore, the spatial sub-encoder includes a graph structure building module, a spatial position encoding module, a spatial position matching module and a spatial attention feature extraction module.

[0029] The graph structure building module is used to represent the spatial position of the sensor unit using a graph structure, where the nodes in the graph structure represent the sensor units and the edges represent the relationship between the sensor units;

[0030] The spatial position encoding module is used to encode the spatial position of the sensing unit in the graph structure;

[0031] The spatial position matching module is used to match the spatial position on the position-encoded graph structure with the impulse tactile data to embed the spatial position into the impulse tactile data;

[0032] The spatial attention feature extraction module is used to extract spatial features based on the matching results.

[0033] Furthermore, in the spatial position encoding module, the expression of position encoding is:

[0034]

[0035] In the formula, H (l) is the node feature matrix of the lth layer, where each row represents the feature vector of a node, H (l+1)is the node feature matrix of the l+1th layer, that is, the node features updated after the graph convolution operation, σ(·) is the nonlinear activation function, is the adjacency matrix A of the graph plus the identity matrix In, n is the number of nodes, for The degree matrix of is a diagonal matrix whose diagonal elements are The sum of each row, W (l) is the weight matrix of the lth layer, which is used to perform linear transformation on node features.

[0036] Furthermore, the spatial attention feature extraction module uses a multi-head self-attention mechanism to extract spatial features, wherein the expression of the multi-head self-attention mechanism is:

[0037] Q,K,V=LN(Linear(LN(Linear(X,W))))

[0038]

[0039] Where Q, K, V are query, key, and value, respectively; LN(·) is LayerNorm; Linear(·) is the linear layer; X and W are input features and a random initialization matrix, respectively; MSA(·) is the self-attention feature calculation; and d is the scale parameter.

[0040] Furthermore, the encoder with two channels is trained using a mean square error loss function, and the expression of the mean square error loss function is:

[0041]

[0042] In the formula, MSE is the mean square error loss, n is the number of samples, and y i is the actual value, is the predicted value.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) The present invention uses a dual-channel encoder to extract temporal features and spatial features from tactile data from the time characteristics and spatial characteristics respectively, to achieve multi-feature perception, and through feature interaction fusion, to realize the tactile perception of the robot's dexterous hand, solving the problem of the difficulty in obtaining bionic embodied information during the operation of complex objects in the robot's dexterous hand, and effectively improving the tactile perception ability of the robot's dexterous hand.

[0045] (2) The temporal sub-encoder and spatial sub-encoder of the present invention respectively perform time step embedding operations and spatial position embedding operations on tactile data to ensure the sequential nature of time and improve the ability to capture temporal features. The more important contact moments and contact positions are highlighted through the temporal attention feature extraction module and the spatial attention feature extraction module respectively to improve the quality of the acquired features.

[0046] (3) The present invention constructs an event-triggered data collection process through flexible materials. The tactile data is first generated into pulses through an RC oscillation circuit to simulate the skin event triggering information transmission method and the neuron information transmission mechanism, which can not only reduce the amount of data, but also reduce the calculation speed. Then, the distributed design is used to adapt to the operating environment in the robot's dexterous palm and design the action space margin for dexterous operation. The distributed design can coordinate the flexible operating characteristics in the dexterous palm and highlight the tactile perception ability of important areas.

[0047] (4) The present invention can be adapted to the joint shape of the robot's dexterous hand to construct a distributed tactile perception capability, and the tactile perception mechanism that simulates human skin can better provide perceptual feedback for the humanoid dexterous hand, and provide embodied perceptual information such as objects and environment for the robot's dexterous hand to understand the environment and subsequent operation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0049] Figure 2 It is a schematic diagram of the flow chart of the object shape reconstruction method of the present invention;

[0050] Figure 3 It is the principle diagram of the sensor of the present invention;

[0051] Figure 4 This is a structural diagram of a dual-channel encoder of the present invention, wherein (a) shows the dual-channel encoder and (b) shows the multimodal interaction module. DETAILED DESCRIPTION

[0052] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0053] This embodiment provides a bionic perception method for a robot dexterous hand based on pulse generation, such as Figure 2As shown in the figure, firstly, the perception receiver is constructed by using flexible materials to realize the flexibility and event triggering functions. Secondly, the sensor is distributed according to the shape of the robot's dexterous hand, and the RC oscillation circuit is used to realize the neuron information transmission method. Then, the tactile data is embedded in the form of time series and space, and the perception information is extracted through the dual-channel time series feature extraction module. Finally, the bionic perception method of the dexterous hand is constructed. This method collects tactile information by constructing a bionic tactile sensor through flexible materials, distributed design, and pulse triggering principles. The data is passed through an encoder and a spatiotemporal feature extraction network to finally realize the cognition of tactile perception information.

[0054] Specifically, Figure 1 As shown, the method comprises the following steps:

[0055] Step S1: construct a flexible tactile signal sensing receiver.

[0056] Flexible materials are used to meet the bionic flexibility characteristics, and the bionic time trigger characteristics are met through the press-on characteristics. The sensing unit is a component of the sensor, and multiple sensing units are jointly constructed into a flexible tactile signal perception receiver (referred to as a tactile sensor). The sensor unit consists of three parts: power supply, perception, and flexible material. There are 6 power supply points and 12 perception points on each sensing unit, which are used to sense the pressure signal and generate pulse data according to the pressure size. The flexible tactile sensor is composed of multiple sensing units. In this embodiment, the tactile data of each sensing unit is collected by polling.

[0057] Specifically, Figure 3 As shown, E receives power from the GPIO pin of the microcontroller as the power supply point, S is the trigger point and transmits the voltage value through the ADC, and multiple patches are jointly constructed as a bionic sensor receiver. A flexible covering material with variable resistivity is used to form a path to collect contact force information and contact position information in space, and collect the voltage value after passing through the flexible covering material as tactile information. The information form of each receiver is; x i =s1,s2,...,s n .

[0058] Step S2: construct a method for transmitting tactile information in the form of pulses.

[0059] The mechanism of skin tactile information generation is simulated, and pulses are generated by RC circuit to complete the generation of embodied tactile information. Specifically, the mechanism of tactile information generation adopts the capacitor charging characteristics of RC circuit. The resistance change only affects the capacitor charging speed. The voltage comparator is designed. The capacitor is filled with the comparator to generate pulses. The periodic difference of pulse generation contains contact force information. The charging principle is; V(t) = V0*(1-e -t / RC ).

[0060] Step S3: Build distributed perception capabilities within the dexterous palm.

[0061] According to the mechanical structure characteristics of the robot's dexterous hand, the flexible tactile sensors are designed in a distributed manner to ensure the accuracy of data collection.

[0062] The distributed design is designed to coordinate the flexible operating features within the dexterous palm and highlight the tactile perception capabilities of important areas. The distributed function is implemented by polling operations, which assign different serial numbers to tactile receivers in different areas and complete the tactile data collection of multiple receivers with low latency.

[0063] Step S4: Build tactile information recognition capability.

[0064] This step mainly inputs the acquired pulse tactile data into the encoder to complete the temporal and spatial feature extraction of the data, and uses the network to identify objects and environmental information. The encoder completes the data embedding, such as Figure 4 As shown in Figure 1, the encoder consists of two channels, which encode the temporal and spatial characteristics of the tactile data respectively. First, the pulse tactile data is used as the input of the spatiotemporal feature extraction module for dual-channel embedding processing. The two features are processed through the feature interaction module ( Figure 4 The cross-attention mechanism in (b) in the figure interacts with each other. The graphic modality and the temporal modality regard themselves as query variables and obtain interactive features through the cross-attention layer respectively. The temporal modality pathway further extracts features through the self-attention layer. The channel outputs of the two modalities are further concatenated to output the probability of the position of the category to which the tactile object belongs, and finally the back-propagation loss calculation is realized to update the network matrix.

[0065] In step 4, the dual-channel tactile data in the encoder is embedded, and a time sub-encoder is constructed using a linear layer and position encoding to achieve time step embedding, and a spatial sub-encoder is constructed using a graph neural network to achieve spatial position embedding. The function of the time sub-encoder in the encoder is to divide the tactile data according to the time, fuse the linearly transformed tactile data with the position encoding, ensure the order of time, and improve the ability to capture time series features; its position encoding formula is:

[0066]

[0067] Then, the time attention feature extraction module in the spatiotemporal feature extraction module is used to extract tactile data features. The function of the time attention feature extraction module in the spatiotemporal feature extraction module is to complete the feature extraction of time series embedding, highlight more important contact moments, and improve the quality of the acquired features. Its attention formula is:

[0068] e i =w (2)(w (1) H+b (1) +b (2) ) (3)

[0069]

[0070] The function of the spatial sub-encoder in the encoder is to match the tactile data according to the spatial position of the sensor unit. The characteristics of the graph neural network can be used to correspond the sensor unit to the space, and the importance of different sensor units can be reflected by manually building a map. The spatial sub-encoder includes a graph structure construction module, a spatial position encoding module, a spatial position matching module and a spatial attention feature extraction module. The graph structure construction module is used to represent the spatial position of the sensor unit using a graph structure, wherein the nodes in the graph structure represent the sensor unit and the edges represent the relationship between the sensor units; the spatial position encoding module is used to encode the spatial position of the sensor unit on the graph structure; the spatial position matching module is used to match the spatial position on the position-encoded graph structure with the pulse tactile data to embed the spatial position into the pulse tactile data. Its position encoding formula is:

[0071]

[0072] Among them, H l represents the node feature matrix of the lth layer, where each row represents the feature vector of a node. H(l+1) represents the node feature matrix of the l+1th layer, that is, the node features updated after the graph convolution operation. =A+In is the adjacency matrix A of the graph plus the identity matrix In (to account for self-connections), where N is the number of nodes. yes The degree matrix of is a diagonal matrix whose diagonal elements are The sum of each row. W l is the weight matrix of the lth layer, which is used to linearly transform the node features. σ is the nonlinear activation function.

[0073] Then, the spatial attention feature extraction module in the spatiotemporal feature extraction module is used to complete the feature extraction of spatial position embedding. The multi-head self-attention mechanism is used to extract features, highlight more important contact positions, and improve the quality of the acquired features. The formula is:

[0074] Q,K,V=LN(Linear(LN(Linear(X,W)))) (7)

[0075]

[0076] Optionally, the dual-channel feature concatenation is performed in the form of pulses to output a pulse sequence, and the classification type corresponds to the sequence position, which can be expressed as: Y pre =[0,0,1,0,..,0] [batch_size,1] .

[0077] Optionally, the loss calculation process is characterized in that the difference calculation is performed through two series of pulses, the tactile data is passed through the feature learning network to generate pulses of the predicted position, and the label data is encoded by one-hot to generate pulses corresponding to the label position. The two pulses are calculated by mse_loss, and a higher penalty is given to larger errors, which helps the model pay more attention to samples with larger prediction deviations during training. The formula is:

[0078]

[0079] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0080] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0081] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0084] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0085] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A bionic perception method for a robot dexterous hand based on pulse generation, characterized in that: The following steps are involved: Acquiring pulse tactile data collected by a sensor unit of the robot's dexterous hand; The pulse tactile data is input into an encoder with two channels, and the spatiotemporal features are extracted and interactively fused to complete the perception and recognition of the object, wherein the encoder with two channels includes a temporal sub-encoder, a spatial sub-encoder and a multimodal interaction module; The steps of completing the perception and recognition of the object include: Inputting the pulse tactile data into the timing sub-encoder to extract timing features; Inputting the impulse tactile data into the spatial sub-encoder to extract spatial features; The temporal features and spatial features are input into the multimodal interaction module for feature interaction to obtain spatiotemporal features, which are processed using a concatenation operation to ultimately output a perception recognition result of the object.

2. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 1 is characterized in that: The generation process of the pulse tactile data includes: Flexible materials are used to meet bionic flexible characteristics, and press-on characteristics are used to meet bionic time trigger characteristics to construct a flexible tactile sensor, wherein the flexible tactile sensor includes multiple sensing units; According to the mechanical structure characteristics of the robot's dexterous hand, the flexible tactile sensor is designed in a distributed manner; The mechanism of generating skin tactile information is simulated, and the capacitor charging characteristics of the RC circuit are used to design a voltage comparator. When the capacitor is fully charged, the comparator generates a pulse, thereby realizing the generation of pulse tactile data.

3. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 2 is characterized in that: The capacitor charging principle of the RC circuit is: V(t)=V0*(1-e -t / RC ) Where V(t) is the output voltage, V0 is the input voltage, and RC is the time constant. The time constant changes according to the degree of pressure.

4. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 1 is characterized in that: The temporal sub-encoder includes a linear layer, a temporal position encoding module and a temporal attention feature extraction module. The linear layer is used to perform linear transformation on the pulse tactile data. The temporal position encoding fusion module is used to fuse the linearly transformed pulse tactile data with its position encoding to obtain the pulse tactile data embedded with time information; the temporal attention feature extraction module is used to extract temporal features from the pulse tactile data embedded with time information.

5. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 4 is characterized in that: In the temporal position coding fusion module, the expression of the position coding is: In the formula, PE(·) is the position code, pos represents the position index, i.e., the step length, and d model is the dimension of the word vector, and i is the dimension index.

6. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 4 is characterized in that: The temporal attention feature extraction module uses the attention mechanism to extract temporal features, where the expression of the attention mechanism is: e i =w (2) (w (1) H+b (1) +b (2) ) In the formula, ei is the feature after nonlinear transformation of the input feature, w (1) 、w (2) are the transformation matrices, b (1) 、b (2) are the bias parameters, H is the input feature, αi is the attention parameter, ek is the feature sequence, n is the total number of input features, C t is the output feature, h i is the input feature, and H is the feature at the current moment.

7. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 1 is characterized in that: The spatial sub-encoder includes a graph structure building module, a spatial position encoding module, a spatial position matching module and a spatial attention feature extraction module. The graph structure building module is used to represent the spatial position of the sensor unit using a graph structure, where the nodes in the graph structure represent the sensor units and the edges represent the relationship between the sensor units; The spatial position encoding module is used to encode the spatial position of the sensing unit in the graph structure; The spatial position matching module is used to match the spatial position on the position-encoded graph structure with the impulse tactile data to embed the spatial position into the impulse tactile data; The spatial attention feature extraction module is used to extract spatial features based on the matching results.

8. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 7 is characterized in that: In the spatial position encoding module, the expression of position encoding is: In the formula, H (l) is the node feature matrix of the lth layer, where each row represents the feature vector of a node, H (l+1) is the node feature matrix of the l+1th layer, that is, the node features updated after the graph convolution operation, σ(·) is the nonlinear activation function, is the adjacency matrix A of the graph plus the identity matrix In, n is the number of nodes, for The degree matrix of is a diagonal matrix whose diagonal elements are The sum of each row, W (l) is the weight matrix of the lth layer, which is used to perform linear transformation on node features.

9. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 7 is characterized in that: The spatial attention feature extraction module uses a multi-head self-attention mechanism to extract spatial features, where the expression of the multi-head self-attention mechanism is: Q,K,V=LN(Linear(LN(Linear(X,W)))) Where Q, K, V are query, key, and value, respectively; LN(·) is LayerNorm; Linear(·) is the linear layer; X and W are input features and a random initialization matrix, respectively; MSA(·) is the self-attention feature calculation; and d is the scale parameter.

10. The bionic perception method of a robot dexterous hand based on pulse generation according to claim 1, characterized in that: The encoder with two channels is trained using a mean square error loss function, and the expression of the mean square error loss function is: In the formula, MSE is the mean square error loss, n is the number of samples, and y i is the actual value, is the predicted value.

Citation Information

Patent Citations

  • Tactile time sequence signal identification method based on deep learning

    CN116502069A

  • Tactile texture recognition method, system and terminal based on biological interpretability

    CN118606800A

  • Communication robot

    JP2009066716A

Cited By

  • Deformable object interactive operation control method based on visual touch-language-action multi-mode model

    CN119526422A