A Method and System for Electrical Impedance Tactile Imaging Based on Graph Networks and Attention Mechanisms

Abstract

This invention proposes a resistive tactile imaging method and system based on graph networks and attention mechanisms. The method includes: acquiring measured spatial boundary voltage sequences and measured boundary voltage sequences under pressure using a resistive tactile sensor to be imaged, and calculating the difference to obtain the conductivity distribution, which is then mapped to the graph structure features of the corresponding graph nodes in the finite element simulation model of the sensor; inputting these graph structure features into a preset hybrid graph neural network model to obtain the predicted conductivity values ​​of the corresponding graph nodes, ultimately achieving imaging; wherein each layer of the hybrid graph neural network model employs a hybrid graph convolutional module composed of sequentially connected graph convolutional networks and graph attention networks. This invention combines graph neural networks to extract spatial topological features and utilizes attention mechanisms to enhance local responses, suppress imaging noise and artifacts, achieve robust transfer from simulation to actual measurement, and improve the reconstruction accuracy and spatial resolution of the tactile contact area.

Description

Technical Field

[0001] This invention relates to the field of robot tactile perception and signal processing technology, specifically to a resistive tactile imaging method and system based on graph networks and attention mechanisms. Background Technology

[0002] With the rapid development of intelligent robots, flexible electronics, and human-computer interaction technologies, the precision requirements for robots' perception of the external environment are increasing. As a core component for robots to acquire the shape, pressure distribution, and sliding state of objects, the imaging performance of tactile sensing systems directly determines the robot's performance in tasks such as dexterous manipulation and safe interaction.

[0003] Electrical Impedance Tomography (EIT), a typical tactile sensing technique, reconstructs the conductivity distribution within a flexible conductive medium by injecting an excitation current at the boundary and measuring the response voltage. Compared to traditional piezoresistive or capacitive array sensors, EIT-based resistive tactile sensors offer significant advantages such as simple structure, sparse wiring (requiring only boundary electrodes), ease of large-area flexible coverage, and high durability, making them highly promising for applications in flexible electronic skin.

[0004] However, in the actual engineering process, electrical impedance tactile imaging still faces the following key technical bottlenecks:

[0005] First, the ill-conditioned nature of the inverse problem and the blurring of the image. The inverse problem of electrical impedance imaging has serious nonlinearity and ill-conditioned nature. Although traditional reconstruction algorithms (such as the Gauss-Newton method and the total variational regularization algorithm) have clear physical meaning, they often exhibit severe edge smoothing effects and artifacts when dealing with complex contact shapes or multi-point contacts, making it difficult to achieve the requirements of high spatial resolution.

[0006] Second, there is a mismatch between geometric topological information and the learning algorithm. To adapt to the diverse geometries of flexible sensors, the finite element method (FEM) is typically used for physical field modeling, generating meshes that are usually irregular triangular units with complex topological relationships. Existing deep learning reconstruction methods mostly employ convolutional neural networks (CNNs), which require the input data to have a regular array of image pixels (Euclidean structure). The process of forcibly resampling irregular physical meshes into regular pixels leads to the loss of physical spatial connectivity information of the sensor, thus limiting the model's ability to perceive local pressure details.

[0007] Third, there is the "domain gap" between simulation models and real hardware. In practical applications, sensor hardware suffers from non-ideal factors such as manufacturing errors, uneven interlayer contact impedance, electrode contact resistance, and environmental electromagnetic interference. Because these factors are difficult to accurately model in a simulation environment, deep learning algorithms trained only on simulation datasets often experience a significant drop in imaging accuracy or even failure when transferred to real acquisition systems.

[0008] In summary, existing electrical impedance tomography (EIT) techniques lack a comprehensive solution that can fully utilize prior information about the mesh topology, effectively suppress measured noise, and improve edge reconstruction accuracy. Summary of the Invention

[0009] This invention addresses several technical challenges in existing electrical impedance tomography (EIT) techniques for tactile perception applications. These challenges include the ill-conditioned nature of the reconstruction inverse problem, blurred edge recognition, the inability of conventional deep learning models to adapt to irregular finite element meshes leading to topological information loss, and a significant decrease in reconstruction accuracy due to noise and nonlinear interference during the migration of simulation models to actual hardware. To address these challenges, this invention proposes an electrical impedance tomography tactile imaging method and system based on graph networks and attention mechanisms. This invention combines graph neural networks to extract spatial topological features and utilizes attention mechanisms to enhance local responses, suppress imaging noise and artifacts, achieving robust migration from simulation to real-world measurements and improving the reconstruction accuracy and spatial resolution of the tactile contact area.

[0010] A first aspect of this invention proposes a resistive tactile imaging method based on graph networks and attention mechanisms, comprising:

[0011] The measured boundary voltage sequence of the airfield and the measured boundary voltage sequence under pressure are acquired using the resistive impedance tactile sensor to be imaged;

[0012] Based on the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence, the conductivity distribution is calculated and mapped to the graph structure features of the corresponding graph nodes in the sensor finite element simulation model;

[0013] The graph structure features are input into a preset hybrid graph neural network model to obtain the predicted conductivity values ​​of the corresponding graph nodes, and finally to achieve imaging. The hybrid graph neural network model adopts a graph-based U-Net architecture, which consists of a skip-connected encoder and a decoder. Each layer of the encoder and the decoder adopts a hybrid graph convolution module, which is composed of a graph convolutional network and a graph attention network connected in sequence.

[0014] In one specific embodiment of the present invention, it further includes:

[0015] A finite element simulation model is constructed for the resistive tactile sensor; the finite element simulation model is divided into triangular mesh elements; each triangular mesh element is abstracted as a graph node, and connecting edges are established according to the geometric adjacency relationship of the mesh elements; wherein, if mesh elements i and j share at least two vertices in physical space, then connecting edges are established between the two mesh elements in the graph.

[0016] In one specific embodiment of the present invention, it further includes:

[0017] Differential voltage data is obtained by using the difference between the measured boundary voltage sequence and the measured open-field boundary voltage sequence. Based on the differential voltage data, a preliminary spatial distribution of conductivity is calculated using a one-step Gauss-Newton algorithm, thereby generating the feature vector of each graph node in the finite element simulation model. ,in, Let be the geometric coordinates of the centroid of the i-th grid cell. The initial conductivity value is given for the i-th grid cell; the eigenvectors of all graph nodes constitute the graph structure features.

[0018] In one specific embodiment of the present invention, before inputting the graph structure features into a preset hybrid graph neural network model, the method further includes:

[0019] Train the hybrid graph neural network model;

[0020] Training the hybrid graph neural network model includes:

[0021] 1) Construct a training set using the simulation data obtained from the finite element simulation model. The input of each training sample in the training set is the graph structure feature composed of the feature vectors of each graph node under the simulated contact scenario, and the label is the actual conductivity distribution under the corresponding scenario.

[0022] 2) Construct the hybrid graph neural network model;

[0023] 3) The hybrid graph neural network model is trained using the training set, wherein the graph convolutional network in the hybrid graph convolutional module performs Laplacian smoothing aggregation on the input full graph to filter out measurement noise; the graph attention network dynamically learns the connection weights between nodes through a multi-head self-attention mechanism, so that the model focuses on the contact edge region where the conductivity changes abruptly; after training, the final hybrid graph neural network model is obtained.

[0024] In a specific embodiment of the present invention, the steps for constructing the training set are as follows:

[0025] 1) Initialize the background conductivity of all mesh elements in the finite element model to a uniform value, perform forward electric field calculation, and obtain the boundary voltage sequence of the empty field simulation;

[0026] 2) Randomly generate contact areas of different numbers, shapes, and random positions and sizes in the simulation to obtain multiple simulated contact scenarios;

[0027] In each simulated contact scenario, based on the geometric parameters of the contact area in that scenario, the mesh cells whose centroid falls within the contact area are obtained; then, the conductivity values ​​of the mesh cells whose centroid falls within the contact area are modified to a preset perturbation value, while the background conductivity of the remaining mesh cells remains unchanged, thus obtaining the true conductivity distribution in that contact scenario.

[0028] 3) For each contact scenario with a realistic conductivity distribution, perform forward electric field calculations to obtain the simulated boundary voltage sequence under pressure.

[0029] 4) Calculate the difference between the simulated boundary voltage sequence and the simulated boundary voltage sequence in the empty field for each contact scenario to obtain the simulated differential voltage data;

[0030] Based on the simulated differential voltage data, the preliminary spatial distribution of conductivity under this contact scenario is calculated using a one-step Gauss-Newton algorithm, and then the feature vector of each graph node in the finite element simulation model under this contact scenario is generated.

[0031] The graph structure features composed of feature vectors of each graph node in each contact scenario are used as input, and the actual conductivity distribution in the corresponding scenario is used as the label to form a training sample; all training samples form a training set.

[0032] A second aspect of the present invention provides a resistive tactile imaging device based on graph networks and attention mechanisms, comprising:

[0033] The voltage sequence acquisition module is used to acquire the measured boundary voltage sequence of the airfield and the measured boundary voltage sequence under pressure using the resistive impedance tactile sensor to be imaged;

[0034] The graph structure feature generation module is used to calculate the conductivity distribution and map it to the graph structure features of the corresponding graph nodes in the sensor finite element simulation model based on the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence.

[0035] An imaging module is used to input the graph structure features into a preset hybrid graph neural network model to obtain the predicted conductivity values ​​of the corresponding graph nodes, and finally achieve imaging. The hybrid graph neural network model adopts a graph-based U-Net architecture, which consists of a skip-connected encoder and a decoder. Each layer of the encoder and the decoder adopts a hybrid graph convolution module, which is composed of a graph convolutional network and a graph attention network connected in sequence.

[0036] In one specific embodiment of the present invention, it further includes:

[0037] A finite element simulation model is constructed for the resistive tactile sensor; the finite element simulation model is divided into triangular mesh elements; each triangular mesh element is abstracted as a graph node, and connecting edges are established according to the geometric adjacency relationship of the mesh elements; wherein, if mesh elements i and j share at least two vertices in physical space, then connecting edges are established between the two mesh elements in the graph.

[0038] In one specific embodiment of the present invention, it further includes:

[0039] Differential voltage data is obtained by using the difference between the measured boundary voltage sequence and the measured open-field boundary voltage sequence. Based on the differential voltage data, a preliminary spatial distribution of conductivity is calculated using a one-step Gauss-Newton algorithm, thereby generating the feature vector of each graph node in the finite element simulation model. ,in, Let be the geometric coordinates of the centroid of the i-th grid cell. The initial conductivity value is given for the i-th grid cell; the eigenvectors of all graph nodes constitute the graph structure features.

[0040] In one specific embodiment of the present invention, before inputting the graph structure features into a preset hybrid graph neural network model, the method further includes:

[0041] Train the hybrid graph neural network model;

[0042] Training the hybrid graph neural network model includes:

[0043] 1) Construct a training set using the simulation data obtained from the finite element simulation model. The input of each training sample in the training set is the graph structure feature composed of the feature vectors of each graph node under the simulated contact scenario, and the label is the actual conductivity distribution under the corresponding scenario.

[0044] 2) Construct the hybrid graph neural network model;

[0045] 3) The hybrid graph neural network model is trained using the training set, wherein the graph convolutional network in the hybrid graph convolutional module performs Laplacian smoothing aggregation on the input full graph to filter out measurement noise; the graph attention network dynamically learns the connection weights between nodes through a multi-head self-attention mechanism, so that the model focuses on the contact edge region where the conductivity changes abruptly; after training, the final hybrid graph neural network model is obtained.

[0046] In a specific embodiment of the present invention, the steps for constructing the training set are as follows:

[0047] 1) Initialize the background conductivity of all mesh elements in the finite element model to a uniform value, perform forward electric field calculation, and obtain the boundary voltage sequence of the empty field simulation;

[0048] 2) Randomly generate contact areas of different numbers, shapes, and random positions and sizes in the simulation to obtain multiple simulated contact scenarios;

[0049] In each simulated contact scenario, based on the geometric parameters of the contact area in that scenario, the mesh cells whose centroid falls within the contact area are obtained; then, the conductivity values ​​of the mesh cells whose centroid falls within the contact area are modified to a preset perturbation value, while the background conductivity of the remaining mesh cells remains unchanged, thus obtaining the true conductivity distribution in that contact scenario.

[0050] 3) For each contact scenario with a realistic conductivity distribution, perform forward electric field calculations to obtain the simulated boundary voltage sequence under pressure.

[0051] 4) Calculate the difference between the simulated boundary voltage sequence and the simulated boundary voltage sequence in the empty field for each contact scenario to obtain the simulated differential voltage data;

[0052] Based on the simulated differential voltage data, the preliminary spatial distribution of conductivity under this contact scenario is calculated using a one-step Gauss-Newton algorithm, and then the feature vector of each graph node in the finite element simulation model under this contact scenario is generated.

[0053] The graph structure features composed of feature vectors of each graph node in each contact scenario are used as input, and the actual conductivity distribution in the corresponding scenario is used as the label to form a training sample; all training samples form a training set.

[0054] A third aspect of the present invention provides an electronic device comprising:

[0055] At least one processor; and a memory communicatively connected to said at least one processor;

[0056] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to execute the above-described resistive tactile imaging method based on graph networks and attention mechanisms.

[0057] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described resistive tactile imaging method based on graph networks and attention mechanisms.

[0058] The features and beneficial effects of this invention are as follows:

[0059] Topological consistency guarantee: This invention performs calculations directly on non-Euclidean grids through graph feature engineering, which can avoid the geometric information loss and distortion problems that may occur during the resampling process of traditional CNN methods, and is conducive to maintaining the spatial fidelity of tactile perception.

[0060] Improved edge perception accuracy: The hybrid graph neural model of this invention combines the stability of GCN and the local sensitivity of GAT, aiming to improve the edge blurring bottleneck commonly found in EIT reconstruction algorithms, and helps to improve the spatial resolution and detail representation of imaging.

[0061] Excellent hardware adaptability and robustness: The virtual-real cross-domain inference mechanism proposed in this invention uses large-scale simulation data to establish reconstruction rules, enabling the model to migrate and apply in real hardware noise environments, effectively reducing the dependence on expensive measured and labeled data.

[0062] System perception closed-loop optimization: This invention provides an integrated solution from sensor topology design to deep learning reconstruction, offering an efficient and reliable technical path for obtaining high-fidelity robotic flexible tactile imaging. Attached Figure Description

[0063] Figure 1 This is a flowchart of an electrical impedance tactile imaging method based on graph networks and attention mechanisms according to an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram of the structure of a hybrid graph neural network model according to a specific embodiment of the present invention. Detailed Implementation

[0065] This invention proposes a resistive tactile imaging method and system based on graph networks and attention mechanisms. To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the application. Rather, this application covers any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this application as defined by the claims. Furthermore, to provide the public with a better understanding of this application, certain specific details are described in detail in the following description. Those skilled in the art can fully understand this application even without these detailed descriptions.

[0066] A first aspect of this invention proposes a resistive tactile imaging method based on graph networks and attention mechanisms, comprising:

[0067] The measured boundary voltage sequence of the airfield and the measured boundary voltage sequence under pressure are acquired using the resistive impedance tactile sensor to be imaged;

[0068] Based on the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence, the conductivity distribution is calculated and mapped to the graph structure features of the corresponding graph nodes in the sensor finite element simulation model;

[0069] The graph structure features are input into a preset hybrid graph neural network model to obtain the predicted conductivity values ​​of the corresponding graph nodes, and finally to achieve imaging. The hybrid graph neural network model adopts a graph-based U-Net architecture, which consists of a skip-connected encoder and a decoder. Each layer of the encoder and the decoder adopts a hybrid graph convolution module, which is composed of a graph convolutional network and a graph attention network connected in sequence.

[0070] In a specific embodiment of the present invention, the overall process of the resistive tactile imaging method based on graph networks and attention mechanisms is as follows: Figure 1 As shown, it includes the following steps:

[0071] 1) Training phase.

[0072] 1-1) Constructing a training set through simulation, the specific steps are as follows:

[0073] 1-1-1) Construct the corresponding finite element simulation model for the resistive tactile sensor to be imaged.

[0074] This embodiment does not impose any exclusive structural requirements on the sensor hardware and is, in principle, applicable to all types of flexible tactile sensors based on the principle of electrical impedance tomography (EIT). That is, as long as the sensor uses an edge electrode mode, the medium has piezoresistive characteristics (changes in internal conductivity due to pressure), and can output boundary voltage data through a data acquisition system, the method described in this embodiment can be used.

[0075] In one specific embodiment of the present invention, a square resistive tactile sensor array is constructed. The sensor adopts a three-layer composite flexible structure: the bottom layer is a conductive paint layer, the middle layer is a highly conductive fabric layer, and the outer layer is a neoprene foam layer with good resilience.

[0076] The sensor has 16 evenly distributed measuring electrodes around its perimeter. When an external object presses on the sensor surface, the outer rubber layer undergoes elastic deformation, causing the middle conductive fabric to make microscopic contact with the underlying conductive paint. According to the principles of contact mechanics, changes in the contact area lead to nonlinear fluctuations in the interlayer contact impedance. The data acquisition system injects a high-frequency AC excitation signal between adjacent electrode pairs using a controlled current source and cyclically measures the boundary voltage drop of the remaining electrode pairs to obtain a one-dimensional voltage sequence reflecting the internal impedance distribution.

[0077] In this embodiment, in order to achieve a precise match between the algorithm and the hardware, a finite element (FEM) simulation model consistent with the structure of the physical sensor is established using MATLAB and the EIDORS toolkit based on the geometric dimensions and electrode distribution parameters of the physical sensor, and the model is divided into a fine mesh composed of triangular elements.

[0078] 1-1-2) Based on the simulation model obtained in step 1-1-1), generate the simulated boundary voltage sequence for the empty field and the simulated boundary voltage sequence for each contact scenario, along with the corresponding actual conductivity distribution; the specific steps are as follows:

[0079] 1-1-2-1) Obtain the boundary voltage sequence of the empty field simulation.

[0080] The background conductivity of all mesh elements in the finite element model is initialized to a uniform value (e.g., 1 S / m) (simulating a state without pressure). The forward electric field calculation (i.e., the forward physical field calculation) is performed, and a set of reference voltage data is obtained as the boundary voltage sequence for the empty field simulation.

[0081] It should be noted that in this embodiment, "boundary" refers to the outer contour area of ​​the sensor, that is, the location of the electrode.

[0082] 1-1-2-2) Randomly generate contact areas of different numbers (e.g., single points, double points), different shapes (e.g., circles, squares, triangles), and random positions and sizes in the simulation scene to obtain multiple contact scenes.

[0083] In one specific embodiment of the present invention, more than 10,000 contact scenarios were simulated.

[0084] In each contact scenario, based on the geometric parameters of the contact area in that scenario, the triangular mesh cells whose centroids fall within the contact area are obtained. Then, the conductivity values ​​of the triangular mesh cells whose centroids fall within the contact area are modified to a preset perturbation value (for example, based on the piezoresistive characteristics, the conductivity is set to 3 times the background value), while the conductivity of the remaining mesh cells remains unchanged from the background. This yields the true conductivity distribution in that contact scenario (which serves as the training label for later use).

[0085] 1-1-2-3) For each contact scenario with a real conductivity distribution, perform forward electric field calculations to obtain the simulated boundary voltage sequence under the pressure state.

[0086] In this embodiment, in order to simulate a real environment, Gaussian white noise with a preset signal-to-noise ratio can be superimposed on the voltage data.

[0087] 1-1-3) Based on the results of step 1-1-2), the preliminary conductivity distribution (physical prior) under each contact scenario is obtained using the traditional reconstruction algorithm.

[0088] Based on the results of steps 1-1-2), the difference between the simulated boundary voltage sequence and the simulated boundary voltage sequence in the empty field is calculated for each contact scenario to obtain the simulated differential voltage data. Here, the contact scenario refers to the simulation of the sensor under pressure, which is represented in the simulation model as a local region with abnormal conductivity; in each contact scenario, the simulated contact area is the region with abnormal conductivity, and its conductivity value deviates from the uniform background value.

[0089] Then, the preliminary spatial distribution of conductivity in the corresponding scenario is calculated using the differential voltage data. In this embodiment, the one-step Gauss-Newton (GN) algorithm is selected as the solver to quickly invert and calculate using the differential voltage data. Although the preliminary distribution contains artifacts and blurring due to linear approximation algorithms and measurement noise, it effectively recovers the key two-dimensional target location information from the one-dimensional voltage, providing necessary physical guidance for subsequent deep learning models.

[0090] 1-1-4) Based on the physical topological relationship of the finite element mesh in the finite element model of step 1-1-1), the result of step 1-1-3) is converted into a data format that can be processed by the graph neural network, thereby generating training samples.

[0091] Specifically, the N triangular mesh elements in the finite element model are abstracted as graph nodes, and connecting edges are established based on the geometric adjacency relationships of the meshes: if mesh element i and element j share at least two vertices in physical space, then connecting edges are established between these two mesh elements in the graph. Furthermore, a multidimensional feature vector is constructed for each graph node. ,in The geometric coordinates of the centroid of the i-th grid cell represent its spatial location information. The initial conductivity value corresponding to the i-th grid cell is obtained from step 1-1-3), representing physical property information. This process achieves a deep integration of geometric topology and electrical priors.

[0092] The graph structure features, composed of feature vectors from each graph node in each contact scenario, are used as input, and the ground truth conductivity distribution set in step 1-1-2) is used as the label to form a training sample. This training sample is used to drive the deep learning model to learn a nonlinear mapping from low-quality preliminary images to high-precision ground truth.

[0093] All training samples constitute the training set.

[0094] 1-2) Construct a hybrid graph neural network model. Use the training set obtained in step 1-1) to train the model and obtain the trained hybrid graph neural network model.

[0095] This embodiment constructs a hybrid graph neural network model integrating GCN graph convolutional network, GAT graph attention network, and U-Net, and adopts a strategy of "learning mapping rules in the simulation space and performing image transfer in the real measurement space"; the specific steps are as follows:

[0096] 1-2-1) Construct a hybrid graph neural network model suitable for processing non-Euclidean grid data.

[0097] Figure 2 This is a schematic diagram of the structure of a hybrid graph neural network model according to a specific embodiment of the present invention. For example... Figure 2 As shown, the model employs a graph-based U-Net architecture, including an encoder (shrinking path) and a decoder (expanding path), with skip connections between the encoder and decoder. The features received at the model's input are as follows: Figure 2 The "preliminary conductivity distribution (including noise)" shown is the initial conductivity distribution obtained by inversion using traditional physics algorithms; the model's output is the denoised conductivity distribution. Figure 2 The bottom text label, "The convolutional layer is a GCN-GAT hybrid graph convolution," indicates that the encoder and decoder embed a hybrid graph convolution module. This module consists of a combination of Graph Convolutional Network (GCN) layers and Graph Attention Network (GAT) layers, used to extract global topological features and local attention features, respectively. Meanwhile, Figure 2 The bottom is labeled "Spatial Size × Number of Channels". In the graph data structure of this invention embodiment, "Spatial Size" specifically refers to the total number of graph nodes after finite element mesh mapping, and "Number of Channels" specifically refers to the feature dimension contained in each node.

[0098] Specifically, in the "encoding" stage of the network structure (such as...) Figure 2 As shown on the left), the network contains four downsampling layers. Through graph pooling, the network progressively compresses the input graph size from the initial 576×16 to the bottleneck layer. During this process, with each downsampling layer, the number of graph nodes (spatial size) is halved, while the number of feature channels is doubled to extract deep, abstract semantic features. The specific evolution of its feature dimension is as follows: the initial feature map dimension is 576×16; after the first downsampling compression, it is compressed to 288×32; after the second downsampling compression, it is compressed to 144×64; after the third downsampling compression, it is compressed to 72×128; after the fourth downsampling, it finally reaches the bottleneck layer, where the dimension is compressed to 36×256. In the "decoding" stage of the network structure (such as...), ... Figure 2As shown on the right), it contains four corresponding upsampling layers. Through graph unpooling, the decoder gradually restores the feature map of the bottleneck layer to its original resolution. During this process, with each upsampling layer, the number of graph nodes (spatial size) doubles layer by layer, while the number of feature channels halves layer by layer. The specific restoration process of the feature dimensions is as follows: the feature map is gradually upsampled and convolved from 36×256 in the bottleneck layer, becoming 72×128, 144×64, and 288×32 respectively, finally restoring it to the same resolution as the input, 576×16. Furthermore, as... Figure 2 As indicated by the arrow at the top spanning the encoding and decoding stages, the model utilizes a skip connection mechanism. This mechanism combines the shallow spatial location features extracted from each level of the encoder (i.e., Figure 2 The feature blocks (576×16, 288×32, 144×64, and 72×128) corresponding to the labeled dimensions are directly transmitted and concatenated to the corresponding layer of the decoder. This design effectively preserves the geometric details of the physical mesh, alleviates the gradient vanishing phenomenon caused by deep networks, and significantly enhances the positioning accuracy of contact edges.

[0099] Furthermore, in the design of convolutional units, each layer in the network adopts a hybrid GCN-GAT module. Specifically, this module first uses a graph convolutional network (GCN) to perform Laplacian smoothing aggregation on the entire image to filter out measurement noise; then, a cascaded graph attention network (GAT) dynamically learns the connection weights between nodes through a multi-head self-attention mechanism, enabling the model to automatically focus on contact edge regions with abrupt changes in conductivity, thereby capturing fine tactile contours.

[0100] 1-2-2) Use the training set obtained in step 1-1) to train the model established in step 1-2-1).

[0101] In this embodiment, the hybrid graph neural network model is trained under full supervision using the training set obtained from simulation. The graph structure features corresponding to the preliminary conductivity distribution from the simulation are used as input, and the actual conductivity distribution is used as the label. The parameters are optimized using a joint loss function of mean squared error (MSE) and structural similarity (SSIM), enabling the model to learn a nonlinear mapping from a low-quality preliminary image to a high-precision ground truth.

[0102] In one specific embodiment of the present invention, the training parameters are set as follows: the Adam optimizer is used, the initial learning rate is 0.001 with exponential decay, the batch size is set to 64, and the maximum number of iterations is 200.

[0103] The training termination condition is as follows: training stops when the validation set loss no longer decreases within 20 consecutive epochs (Early Stopping) or when the maximum number of iterations is reached, resulting in a trained hybrid graph neural network.

[0104] 2) Application stage.

[0105] This stage utilizes a trained hybrid graph neural network model to perform real-time imaging of the real scene. The specific steps are as follows:

[0106] 2-1) Acquire measured boundary voltage data using the resistive impedance tactile sensor to be imaged.

[0107] In one specific embodiment of the present invention, a set of data is first collected under no-load conditions as a measured empty-field boundary voltage sequence. Then, using a constructed 16-channel high-precision impedance data acquisition system, a real pressure is applied to the sensor. The scenario is not specifically limited; it can be a single-point pressure, multi-point pressure, or pressure from objects of different shapes. Under pressure, the measured boundary voltage sequence output by the sensor is recorded in real time.

[0108] 2-2) Data preprocessing and feature construction.

[0109] To adapt to the model input, the processing steps 1-1-3) and 1-1-4) need to be repeated for the data collected in step 2-1). Specifically, the difference between the measured boundary voltage sequence and the measured open-field boundary voltage sequence is calculated, and the measured preliminary conductivity distribution is obtained by inversion using the traditional Gauss-Newton (GN) algorithm. Subsequently, based on the finite element mesh topology, this measured preliminary conductivity distribution is mapped into graph structure feature data, that is, a feature vector containing the node geometric coordinates and the measured preliminary conductivity values ​​is generated.

[0110] 2-3) Input the measured graph structure feature data obtained in step 2-2) into the hybrid graph neural network model trained in step 1). After forward propagation, the model calculates and outputs the predicted conductivity values ​​corresponding one-to-one with the input graph nodes based on the feature mapping rules learned during the training phase.

[0111] 2-4) Use the predicted conductivity values ​​obtained in step 2-3) for imaging.

[0112] In this embodiment, the predicted conductivity values ​​of each graph node output in steps 2-3) are used in combination with the geometric position information of the finite element mesh to perform color mapping and interpolation rendering, and finally the tactile conductivity distribution image reflecting the current contact state is displayed on the host computer interface.

[0113] This embodiment demonstrates how to deeply integrate physical models with deep learning techniques to solve the inverse problem in impedance tactile imaging. The large-scale dataset constructed using finite element simulations allows the neural network to fully learn the nonlinear mapping relationship between conductivity distribution and boundary voltage, effectively avoiding the difficulty of labeling measured data. Simultaneously, the combination of physical topological prior graph structure design and attention mechanisms significantly enhances the model's ability to process irregular grid data and its accuracy in capturing contact edges. Finally, through a virtual-real transfer strategy, this embodiment can significantly suppress imaging noise and artifacts while preserving the geometric position information obtained through traditional physical inversion, providing an effective technical solution for obtaining high spatial resolution and robust flexible tactile perception.

[0114] To achieve the above embodiments, a second aspect of the present invention provides a resistive tactile imaging device based on graph networks and attention mechanisms, comprising:

[0115] The voltage sequence acquisition module is used to acquire the measured boundary voltage sequence of the airfield and the measured boundary voltage sequence under pressure using the resistive impedance tactile sensor to be imaged;

[0116] The graph structure feature generation module is used to calculate the conductivity distribution and map it to the graph structure features of the corresponding graph nodes in the sensor finite element simulation model based on the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence.

[0117] An imaging module is used to input the graph structure features into a preset hybrid graph neural network model to obtain the predicted conductivity values ​​of the corresponding graph nodes, and finally achieve imaging. The hybrid graph neural network model adopts a graph-based U-Net architecture, which consists of a skip-connected encoder and a decoder. Each layer of the encoder and the decoder adopts a hybrid graph convolution module, which is composed of a graph convolutional network and a graph attention network connected in sequence.

[0118] In one specific embodiment of the present invention, it further includes:

[0119] A finite element simulation model is constructed for the resistive tactile sensor; the finite element simulation model is divided into triangular mesh elements; each triangular mesh element is abstracted as a graph node, and connecting edges are established according to the geometric adjacency relationship of the mesh elements; wherein, if mesh elements i and j share at least two vertices in physical space, then connecting edges are established between the two mesh elements in the graph.

[0120] In one specific embodiment of the present invention, it further includes:

[0121] Differential voltage data is obtained by using the difference between the measured boundary voltage sequence and the measured open-field boundary voltage sequence. Based on the differential voltage data, a preliminary spatial distribution of conductivity is calculated using a one-step Gauss-Newton algorithm, thereby generating the feature vector of each graph node in the finite element simulation model. ,in, Let be the geometric coordinates of the centroid of the i-th grid cell. The initial conductivity value is given for the i-th grid cell; the eigenvectors of all graph nodes constitute the graph structure features.

[0122] In one specific embodiment of the present invention, before inputting the graph structure features into a preset hybrid graph neural network model, the method further includes:

[0123] Train the hybrid graph neural network model;

[0124] Training the hybrid graph neural network model includes:

[0125] 1) Construct a training set using the simulation data obtained from the finite element simulation model. The input of each training sample in the training set is the graph structure feature composed of the feature vectors of each graph node under the simulated contact scenario, and the label is the actual conductivity distribution under the corresponding scenario.

[0126] 2) Construct the hybrid graph neural network model;

[0127] 3) The hybrid graph neural network model is trained using the training set, wherein the graph convolutional network in the hybrid graph convolutional module performs Laplacian smoothing aggregation on the input full graph to filter out measurement noise; the graph attention network dynamically learns the connection weights between nodes through a multi-head self-attention mechanism, so that the model focuses on the contact edge region where the conductivity changes abruptly; after training, the final hybrid graph neural network model is obtained.

[0128] In a specific embodiment of the present invention, the steps for constructing the training set are as follows:

[0129] 1) Initialize the background conductivity of all mesh elements in the finite element model to a uniform value, perform forward electric field calculation, and obtain the boundary voltage sequence of the empty field simulation;

[0130] 2) Randomly generate contact areas of different numbers, shapes, and random positions and sizes in the simulation to obtain multiple simulated contact scenarios;

[0131] In each simulated contact scenario, based on the geometric parameters of the contact area in that scenario, the mesh cells whose centroid falls within the contact area are obtained; then, the conductivity values ​​of the mesh cells whose centroid falls within the contact area are modified to a preset perturbation value, while the background conductivity of the remaining mesh cells remains unchanged, thus obtaining the true conductivity distribution in that contact scenario.

[0132] 3) For each contact scenario with a realistic conductivity distribution, perform forward electric field calculations to obtain the simulated boundary voltage sequence under pressure.

[0133] 4) Calculate the difference between the simulated boundary voltage sequence and the simulated boundary voltage sequence in the empty field for each contact scenario to obtain the simulated differential voltage data;

[0134] Based on the simulated differential voltage data, the preliminary spatial distribution of conductivity under this contact scenario is calculated using a one-step Gauss-Newton algorithm, and then the feature vector of each graph node in the finite element simulation model under this contact scenario is generated.

[0135] The graph structure features composed of feature vectors of each graph node in each contact scenario are used as input, and the actual conductivity distribution in the corresponding scenario is used as the label to form a training sample; all training samples form a training set.

[0136] This enables the extraction of spatial topological features by combining graph neural networks and enhancing local response using attention mechanisms, suppressing imaging noise and artifacts, achieving robust transfer from simulation to actual measurement, and improving the reconstruction accuracy and spatial resolution of the tactile contact area.

[0137] To implement the above embodiments, a third aspect of the present invention provides an electronic device, comprising:

[0138] At least one processor; and a memory communicatively connected to said at least one processor;

[0139] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to execute the above-described resistive tactile imaging method based on graph networks and attention mechanisms.

[0140] To implement the above embodiments, a fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described resistive tactile imaging method based on graph networks and attention mechanisms.

[0141] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0142] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform a resistive tactile imaging method based on graph networks and attention mechanisms according to the above embodiments.

[0143] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0145] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0146] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0147] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0148] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0149] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0150] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0151] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A resistance impedance tactile imaging method based on graph network and attention mechanism, characterized in that, include: The measured boundary voltage sequence of the airfield and the measured boundary voltage sequence under pressure are acquired using the resistive impedance tactile sensor to be imaged; Based on the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence, the conductivity distribution is calculated and mapped to the graph structure features of the corresponding graph nodes in the sensor finite element simulation model; The graph structure features are input into a preset hybrid graph neural network model to obtain the predicted conductivity values ​​of the corresponding graph nodes, and finally to achieve imaging. The hybrid graph neural network model adopts a graph-based U-Net architecture, which consists of a skip-connected encoder and a decoder. Each layer of the encoder and the decoder adopts a hybrid graph convolution module, which is composed of a graph convolutional network and a graph attention network connected in sequence.

2. The method of claim 1, wherein, Also includes: A finite element simulation model is constructed for the resistive tactile sensor; the finite element simulation model is divided into triangular mesh elements; Each triangular mesh cell is abstracted as a graph node, and connecting edges are established based on the geometric adjacency of the mesh cells; wherein, if mesh cells i and j share at least two vertices in physical space, then connecting edges are established between the two mesh cells in the graph.

3. The method of claim 2, wherein, Also includes: Differential voltage data are obtained by using the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence; Based on the differential voltage data, a preliminary conductivity spatial distribution is calculated using a one-step Gauss-Newton algorithm, and a feature vector of each graph node in the finite element simulation model is generated wherein, is a geometric coordinate of the i-th grid cell centroid, is a preliminary conductivity value corresponding to the i-th grid cell; the feature vectors of all graph nodes constitute the graph structure features.

4. The method according to claim 3, characterized in that, Before inputting the graph structure features into the preset hybrid graph neural network model, the method further includes: Train the hybrid graph neural network model; Training the hybrid graph neural network model includes: 1) Construct a training set using the simulation data obtained from the finite element simulation model. The input of each training sample in the training set is the graph structure feature composed of the feature vectors of each graph node under the simulated contact scenario, and the label is the actual conductivity distribution under the corresponding scenario. 2) Construct the hybrid graph neural network model; 3) The hybrid graph neural network model is trained using the training set, wherein the graph convolutional network in the hybrid graph convolutional module performs Laplacian smoothing aggregation on the input full graph to filter out measurement noise; the graph attention network dynamically learns the connection weights between nodes through a multi-head self-attention mechanism, so that the model focuses on the contact edge region where the conductivity changes abruptly; after training, the final hybrid graph neural network model is obtained.

5. The method according to claim 4, characterized in that, The specific steps for constructing the training set are as follows: 1) Initialize the background conductivity of all mesh elements in the finite element model to a uniform value, perform forward electric field calculation, and obtain the boundary voltage sequence of the empty field simulation; 2) Randomly generate contact areas of different numbers, shapes, and random positions and sizes in the simulation to obtain multiple simulated contact scenarios; In each simulated contact scenario, based on the geometric parameters of the contact area in that scenario, the mesh cells whose centroid falls within the contact area are obtained; then, the conductivity values ​​of the mesh cells whose centroid falls within the contact area are modified to a preset perturbation value, while the background conductivity of the remaining mesh cells remains unchanged, thus obtaining the true conductivity distribution in that contact scenario. 3) For each contact scenario with a realistic conductivity distribution, perform forward electric field calculations to obtain the simulated boundary voltage sequence under pressure. 4) Calculate the difference between the simulated boundary voltage sequence and the simulated boundary voltage sequence in the empty field for each contact scenario to obtain the simulated differential voltage data; Based on the simulated differential voltage data, the preliminary spatial distribution of conductivity under the contact scenario is calculated using a one-step Gauss-Newton algorithm, and then the feature vector of each graph node in the finite element simulation model under the contact scenario is generated. The graph structure features composed of feature vectors of each graph node in each contact scenario are used as input, and the actual conductivity distribution in the corresponding scenario is used as the label to form a training sample; all training samples form a training set.

6. A resistive tactile imaging device based on graph networks and attention mechanisms, characterized in that, include: The voltage sequence acquisition module is used to acquire the measured boundary voltage sequence of the airfield and the measured boundary voltage sequence under pressure using the resistive impedance tactile sensor to be imaged; The graph structure feature generation module is used to calculate the conductivity distribution and map it to the graph structure features of the corresponding graph nodes in the sensor finite element simulation model based on the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence. An imaging module is used to input the graph structure features into a preset hybrid graph neural network model to obtain the predicted conductivity values ​​of the corresponding graph nodes, and finally achieve imaging. The hybrid graph neural network model adopts a graph-based U-Net architecture, which consists of a skip-connected encoder and a decoder. Each layer of the encoder and the decoder adopts a hybrid graph convolution module, which is composed of a graph convolutional network and a graph attention network connected in sequence.

7. The apparatus according to claim 6, characterized in that, Also includes: A finite element simulation model is constructed for the resistive tactile sensor; the finite element simulation model is divided into triangular mesh elements; Each triangular mesh cell is abstracted as a graph node, and connecting edges are established based on the geometric adjacency of the mesh cells; wherein, if mesh cells i and j share at least two vertices in physical space, then connecting edges are established between the two mesh cells in the graph.

8. The apparatus according to claim 7, characterized in that, Also includes: Differential voltage data are obtained by using the difference between the measured boundary voltage sequence and the measured open field boundary voltage sequence; Based on the differential voltage data, a preliminary spatial distribution of conductivity is calculated using a one-step Gauss-Newton algorithm, thereby generating the feature vector of each graph node in the finite element simulation model. ,in, Let be the geometric coordinates of the centroid of the i-th grid cell. The initial conductivity value is given for the i-th grid cell; the eigenvectors of all graph nodes constitute the graph structure features.

9. The apparatus according to claim 8, characterized in that, Before inputting the graph structure features into the preset hybrid graph neural network model, the method further includes: Train the hybrid graph neural network model; Training the hybrid graph neural network model includes: 1) Construct a training set using the simulation data obtained from the finite element simulation model. The input of each training sample in the training set is the graph structure feature composed of the feature vectors of each graph node under the simulated contact scenario, and the label is the actual conductivity distribution under the corresponding scenario. 2) Construct the hybrid graph neural network model; 3) The hybrid graph neural network model is trained using the training set, wherein the graph convolutional network in the hybrid graph convolutional module performs Laplacian smoothing aggregation on the input full graph to filter out measurement noise; the graph attention network dynamically learns the connection weights between nodes through a multi-head self-attention mechanism, so that the model focuses on the contact edge region where the conductivity changes abruptly; after training, the final hybrid graph neural network model is obtained.

10. The apparatus according to claim 9, characterized in that, The specific steps for constructing the training set are as follows: 1) Initialize the background conductivity of all mesh elements in the finite element model to a uniform value, perform forward electric field calculation, and obtain the boundary voltage sequence of the empty field simulation; 2) Randomly generate contact areas of different numbers, shapes, and random positions and sizes in the simulation to obtain multiple simulated contact scenarios; In each simulated contact scenario, based on the geometric parameters of the contact area in that scenario, the mesh cells whose centroid falls within the contact area are obtained; then, the conductivity values ​​of the mesh cells whose centroid falls within the contact area are modified to a preset perturbation value, while the background conductivity of the remaining mesh cells remains unchanged, thus obtaining the true conductivity distribution in that contact scenario. 3) For each contact scenario with a realistic conductivity distribution, perform forward electric field calculations to obtain the simulated boundary voltage sequence under pressure. 4) Calculate the difference between the simulated boundary voltage sequence and the simulated boundary voltage sequence in the empty field for each contact scenario to obtain the simulated differential voltage data; Based on the simulated differential voltage data, the preliminary spatial distribution of conductivity under this contact scenario is calculated using a one-step Gauss-Newton algorithm, and then the feature vector of each graph node in the finite element simulation model under this contact scenario is generated. The graph structure features composed of feature vectors of each graph node in each contact scenario are used as input, and the actual conductivity distribution in the corresponding scenario is used as the label to form a training sample; all training samples form a training set.

Images