Tactile perception positioning system and method

Abstract

The application discloses a kind of tactile perception positioning system and method.The system includes tactile sensor and data acquisition processing module;Tactile sensor includes shell, flexible perception layer and multiple electrodes, electrode is fixed on shell, the end of electrode away from shell is connected with flexible perception layer, flexible perception layer is covered on shell, and flexible perception layer is used to change the impedance data between at least part of electrode when being pressed by external object;Data acquisition processing module is electrically connected with multiple electrodes, and data acquisition processing module is used to apply excitation signal to flexible perception layer by electrode, and the response signal of flexible perception layer is measured by electrode, to obtain the impedance data between electrode and determine the contact position information of tactile sensor and external object based on impedance data.In the present application, tactile perception positioning is carried out by electrode impedance measurement mode, which can effectively simplify the sensor structure to realize reliable adhesion on curved surface finger tip.

Description

Technical Field

[0001] This invention relates to the field of sensing technology, and in particular to a tactile sensing and positioning system and method. Background Technology

[0002] In fields such as dexterous robotic manipulation and tactile feedback in intelligent prosthetics, fingertip tactile sensing technology is crucial. Accurate contact position sensing is fundamental to achieving tasks such as precise grasping and flexible manipulation. Traditional contact position sensing systems often employ array-type sensor solutions for curved surface tactile sensing, which achieves position sensing by integrating a matrix of numerous independent sensing units (such as piezoresistive, capacitive, or piezoelectric units) onto a flexible substrate. However, this design requires a complex internal sensing unit matrix and multiple layers of signal leads, resulting in high hardware integration difficulty, high manufacturing costs, and challenges in achieving stable adhesion and structural reliability on highly curved fingertip surfaces. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a tactile sensing positioning system and method to solve the problems of complex structure and poor fit on high curvature surfaces of array sensor schemes.

[0004] In a first aspect, the present invention provides a tactile sensing and positioning system. The tactile sensing and positioning system includes a tactile sensor and a data acquisition and processing module; The tactile sensor includes a housing, a flexible sensing layer, and multiple electrodes. The electrodes are fixed to the housing, and one end of the electrode away from the housing is connected to the flexible sensing layer. The flexible sensing layer covers the housing and is used to cause a change in the impedance data between at least some of the electrodes when pressed by an external object. The data acquisition and processing module is electrically connected to the plurality of electrodes. The data acquisition and processing module is used to apply an excitation signal to the flexible sensing layer through the electrodes and measure the response signal of the flexible sensing layer through the electrodes to obtain impedance data between the electrodes and determine the contact position information between the tactile sensor and the external object based on the impedance data.

[0005] In one implementation of the first aspect, the data acquisition and processing module includes a data acquisition module and a data processing module, wherein the data acquisition module and the data processing module are communicatively connected. The data acquisition module is electrically connected to the plurality of electrodes. The data acquisition module is used to apply an excitation signal to the flexible sensing layer through the electrodes and measure the response signal of the flexible sensing layer through the electrodes to obtain impedance data between the electrodes, and then send the impedance data to the data processing module. The data processing module is used to determine the contact position information between the tactile sensor and the external object based on the impedance data.

[0006] In one implementation of the first aspect, the data acquisition module includes a main control unit, a first multiplexer, and a second multiplexer; The main control unit is connected to the first multiplexer and the second multiplexer respectively, and the main control unit is used to control the channel selection of the first multiplexer and the second multiplexer; The first multiplexer is connected to multiple electrodes, and the first multiplexer is used to select the excitation electrode from the multiple electrodes under the control of the main control unit; The second multiplexer is connected to multiple electrodes and is used to select a measurement electrode from the multiple electrodes under the control of the main control unit.

[0007] In one implementation of the first aspect, a flexible protective film is also included, which is sleeved on the outer surface of the flexible sensing layer.

[0008] Secondly, the present invention provides a tactile sensing positioning method. The tactile sensing positioning method is applied to the tactile sensing positioning system described above, and the tactile sensing positioning method includes: When the flexible sensing layer in the tactile sensor is pressed by an external object, the data acquisition and processing module applies a first excitation signal to the flexible sensing layer through electrodes, and measures the first response signal of the flexible sensing layer through electrodes to obtain contact impedance data. The data acquisition and processing module obtains the background impedance data of the tactile sensor in a non-contact reference state; The data acquisition and processing module determines the contact position information between the tactile sensor and the external object based on the background impedance data and the contact impedance data.

[0009] In one implementation of the second aspect, a data acquisition and processing module applies a first excitation signal to the flexible sensing layer via electrodes, and measures a first response signal of the flexible sensing layer via electrodes to obtain contact impedance data, including: Two electrodes are randomly selected from the plurality of electrodes of the tactile sensor to form an electrode pair, thereby determining all possible electrode pairs; For each electrode pair, the first excitation signal is applied to one of the electrodes to inject into the flexible sensing layer, and the first response signal generated by the flexible sensing layer is measured through the other electrode; Based on the first excitation signal and the first response signal corresponding to each electrode pair, the inter-electrode impedance data of each electrode pair is calculated, thereby obtaining the contact impedance data of all electrode pair combinations.

[0010] In one implementation of the second aspect, the data acquisition and processing module determines the contact position information between the tactile sensor and the external object based on the background impedance data and the contact impedance data, including: The differential impedance signal is determined based on the difference between the contact impedance data and the background impedance data; The contact position information is determined based on the differential impedance signal.

[0011] In one implementation of the second aspect, determining the contact position information based on the differential impedance signal includes: inputting the differential impedance signal into a preset tactile positioning model to obtain the contact position information.

[0012] In one implementation of the second aspect, the tactile localization model includes a physical prior reconstruction layer based on the Jacobian matrix, a surface graph neural network module, a heat map branch, and an offset branch. Correspondingly, the differential impedance signal is input into a preset tactile positioning model to obtain the contact position information, including: The differential impedance signal is input into the physical prior reconstruction layer, and the differential impedance signal is reconstructed into the initial features of the surface vertices in the physical prior reconstruction layer; The initial features of the surface vertices are input into the surface graph neural network module for neighborhood propagation and fusion to obtain high-dimensional features at the vertex level; The vertex-level high-dimensional features are input into the heatmap branch and the offset branch respectively to determine the contact probability distribution and three-dimensional local offset vector of each surface vertex. The contact position information is obtained by weighted summation of the contact probability distribution, the three-dimensional local offset vector, and the three-dimensional coordinates of the surface vertices.

[0013] In one implementation of the second aspect, the reconstructing of the differential impedance signal into initial features of the surface vertices based on the Jacobian matrix includes: The initial response distribution of the volume element space is obtained by solving the pseudo-inverse with regularization; The initial response distribution of the volume element space is mapped to the surface vertices by area-weighted aggregation, forming the initial features of the surface vertices.

[0014] The beneficial effects of this invention are as follows: By arranging multiple electrodes on the housing of the tactile sensor and covering it with a flexible sensing layer, wherein the ends of the electrodes away from the housing are connected to the flexible sensing layer, the electrodes can serve as the electrical excitation and measurement interface for the flexible sensing layer. Based on this, utilizing the piezoresistive response characteristic that causes at least some of the electrodes to change impedance when the flexible sensing layer is pressed (i.e., when pressed by an external object, the impedance data between at least some of the electrodes changes), the data acquisition and processing module applies an excitation signal to the flexible sensing layer through the electrodes and obtains impedance data by measuring the response signal generated by the flexible sensing layer. Then, based on the impedance data, the contact position information between the tactile sensor and the external object is determined, thereby avoiding the defects of traditional array sensor solutions that require the internal arrangement of complex independent sensing unit matrices and multi-layer leads, and achieving a high degree of integration and fit of the sensor structure on the curved fingertip. Attached Figure Description

[0015] Figure 1 This is a system architecture diagram of a tactile sensing and positioning system disclosed in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of the structure of a tactile sensor disclosed in an embodiment of the present invention from one viewpoint.

[0017] Figure 3 This is a schematic diagram of the structure of a tactile sensor disclosed in an embodiment of the present invention from another perspective.

[0018] Figure 4 This is an exploded view of the structure of a tactile sensor disclosed in an embodiment of the present invention from one perspective.

[0019] Figure 5 This is a bottom view of a tactile sensor disclosed in an embodiment of the present invention.

[0020] Figure 6 yes Figure 5 The diagram shows a cross-sectional view of the tactile sensor.

[0021] Figure 7 This is a schematic diagram showing the impedance change of a flexible sensing layer before and after pressing, according to an embodiment of the present invention.

[0022] Figure 8 This is a flowchart of the steps of a tactile sensing and positioning method disclosed in an embodiment of the present invention.

[0023] Figure 9 This is a schematic diagram of the structure of a surface graph neural network module disclosed in an embodiment of the present invention.

[0024] Label Explanation: 1. Tactile sensor; 11. Housing; 12. Electrode; 13. Flexible sensing layer; 14. Receiving cavity. Detailed Implementation

[0025] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0026] To facilitate understanding of the technical solutions provided in the embodiments of the present invention by those skilled in the art, the relevant technologies are described below: In fields such as dexterous robotic manipulation and tactile feedback in intelligent prosthetics, fingertip tactile sensing technology is crucial. Accurate contact position sensing is fundamental for tasks such as precise grasping and flexible manipulation. To achieve accurate contact position sensing, array-type sensor solutions are often used for curved surface tactile sensing. Array-type sensor solutions involve densely arranging a large number of independent sensing units (such as piezoresistive, capacitive, or piezoelectric units) on a flexible substrate in a matrix arrangement. Each unit independently senses local pressure and outputs the local pressure information through independent signal leads. Finally, the signals from all units are combined to reconstruct a tactile image for contact localization.

[0027] However, according to the inventors, the aforementioned array-type sensor solution requires a complex internal sensing unit matrix and multiple layers of signal leads, resulting in high hardware integration difficulty, high manufacturing costs, and difficulty in achieving stable adhesion and structural reliability on highly curved fingertip surfaces. Specifically: on the one hand, to improve spatial resolution, extremely dense sensing units must be integrated, leading to exceptionally complex internal wiring; on the other hand, multiple layers of signal leads and discrete sensing unit matrices greatly limit the sensor's flexibility, making it difficult to perfectly fit complex three-dimensional curved surfaces, and prone to structural failure under frequent bending. Therefore, high hardware integration difficulty, high manufacturing costs, and poor surface adaptability have become key bottlenecks restricting the application of array-type sensor solutions.

[0028] In summary, the above-mentioned array-type sensor scheme requires the internal integration of a complex sensor unit matrix and multiple layers of signal leads, which makes the fabrication difficult and results in poor fit and structural stability on high-curvature surfaces.

[0029] To at least address the aforementioned problems, this invention provides a tactile sensing and positioning scheme. In this scheme, a tactile sensor structure is adopted, in which electrodes 12 are arranged on the surface of the housing 11 and covered with a flexible sensing layer 13. Electrode impedance measurement is used instead of the array structure scheme described above. This avoids the problems of structural complexity, high manufacturing cost, and reduced flexibility caused by integrating a large number of independent sensing units and multiple signal leads inside the sensor. This allows the sensor to conform to the high curvature surface of the fingertip while maintaining structural reliability.

[0030] The technical solutions according to the present invention will be described below with reference to specific embodiments and in conjunction with the accompanying drawings.

[0031] Figure 1 This is a system architecture diagram of a tactile sensing and positioning system disclosed in an embodiment of the present invention. (Refer to...) Figure 1 The tactile sensing and positioning system includes a tactile sensor 1 and a data acquisition and processing module. The tactile sensor 1 is electrically connected to the data acquisition and processing module and is used to contact external objects.

[0032] The tactile sensor 1 includes a housing 11, a flexible sensing layer 13, and a plurality of electrodes 12. The electrodes 12 are fixed on the housing 11, and one end of the electrode 12 away from the housing 11 is connected to the flexible sensing layer 13. The flexible sensing layer 13 covers the housing 11 and is used to cause at least some of the impedance data between the electrodes to change when pressed by an external object. The data acquisition and processing module is electrically connected to the plurality of electrodes 12. The data acquisition and processing module is used to apply an excitation signal to the flexible sensing layer 13 through the electrodes 12, and to measure the response signal of the flexible sensing layer 13 through the electrodes 12, so as to obtain impedance data between the electrodes and determine the contact position information between the tactile sensor 1 and the external object based on the impedance data.

[0033] As can be seen from the above description, the beneficial effects of the present invention are as follows: by arranging multiple electrodes 12 on the housing 11 of the tactile sensor 1 and covering it with a flexible sensing layer 13, wherein the end of the electrode 12 away from the housing 11 is connected to the flexible sensing layer 13, the electrode 12 can serve as the electrical excitation and measurement interface of the flexible sensing layer 13. Based on this, utilizing the piezoresistive response characteristic that causes at least some of the electrodes to change impedance when the flexible sensing layer 13 is pressed (i.e., when pressed by an external object, at least some of the impedance data between the electrodes changes), the data acquisition and processing module applies an excitation signal to the flexible sensing layer 13 through the electrodes 12 and obtains impedance data by measuring the response signal generated by the flexible sensing layer 13. Then, based on the impedance data, the contact position information between the tactile sensor 1 and the external object is determined, thereby avoiding the defects of traditional array sensor schemes that require the internal arrangement of complex independent sensing unit matrices and multi-layer leads, and realizing the high integration and fit of the sensor structure on the curved fingertip.

[0034] The following combination Figure 1 The tactile sensor 1, the data acquisition and processing unit, and other optional structures described above are described in detail.

[0035] like Figure 2 as well as Figure 3As shown, the tactile sensor 1 can be a curved fingertip contact type tactile sensor, which can adopt a finger sleeve type three-dimensional structure adapted to the shape of the finger. The curved fingertip contact type tactile sensor 1 includes, from the inside to the outside, a 3D printed shell 11, an electrode 12 (i.e., boundary electrode), a flexible sensing layer 13, and a flexible protective film (i.e., encapsulation protective layer).

[0036] The housing 11 serves as the base for the tactile sensor 1 and has a receiving cavity 14. This cavity 14 is adapted to the shape of the target object, which can be the fingertip of a robot or prosthetic limb, etc., and is not limited thereto. Figure 4 As shown, the outer surface of the housing 11 is provided with a preset number of uniformly arranged slots, which are used to accommodate the electrodes 12 and the number of slots corresponds one-to-one with the electrodes 12.

[0037] The plurality of electrodes 12 are secured in the slots of the housing 11, with one end extending out of the outer surface of the housing 11 and connecting to the flexible sensing layer 13 to form direct electrical contact. The number of electrodes 12 can be set according to actual needs, and they are evenly distributed on the outer surface of the housing 11 and embedded in the slots. In practical applications, 16 electrodes 12 (such as gold-plated electrodes) can be evenly arranged on the housing 11 as an interface for electrical excitation and measurement. Specifically, for example... Figure 4 , Figure 5 as well as Figure 6 As shown, three electrodes 12 can be respectively arranged on the front, back, left and right sides of the housing 11. The three electrodes 12 on each side are evenly spaced along the axial direction of the housing 11. Four electrodes 12 are also provided on the top of the housing 11. Three of the electrodes 12 are evenly spaced along the horizontal circumferential direction, and the other is located at the center of the horizontal circumferential direction.

[0038] The flexible sensing layer 13 covers the outer surface of the housing 11 and can be made of carbon nanotube hydrogel with a thickness of approximately 2.5 mm. The carbon nanotube hydrogel is a flexible conductive composite material formed by dispersing carbon nanotubes as conductive fillers in a hydrogel matrix (such as polyacrylamide). It combines the high flexibility of hydrogels with the excellent conductivity of carbon nanotubes, and can convert pressure into a measurable impedance change through the reconstruction of the internal conductive network under pressure. It should be noted that due to the excellent flexibility and piezoresistive response characteristics of this material, when external pressure is applied to the curved surface of the fingertip of the tactile sensor 1, the carbon nanotube conductive network inside the carbon nanotube hydrogel is reshaped. This reshaping of the local conductivity distribution, caused by reduced contact resistance and quantum tunneling effect, converts the tactile pressure signal into a change in internal impedance.

[0039] The flexible protective film is applied to the outer surface of the flexible sensing layer 13. This film, with a thickness of 1 mm, seals the sensing layer to prevent moisture evaporation and enhances the sensor's mechanical durability in complex environments. In other words, by adding a flexible protective film to the outer surface of the flexible sensing layer 13, a seal and mechanical protection are formed for the internal sensing layer, reducing damage from moisture evaporation and external friction, and improving the sensor's durability and environmental adaptability under repeated contact conditions.

[0040] In the specific work process, such as Figure 7 As shown, when the tactile sensor 1 (i.e. the flexible sensing layer 13) is not pressed by the outside, the carbon nanotubes are relatively sparsely distributed in the hydrogel matrix, the number of contact points of the internal conductive network is small, and the spacing between the carbon nanotubes is large. It is difficult for electrons to form an effective conductive path through direct contact or tunneling effect, so the overall impedance of the material is high.

[0041] When the tactile sensor 1 is pressed, the hydrogel matrix undergoes compression deformation, causing the internal carbon nanotubes to move closer together. On the one hand, the number of contact points between the carbon nanotubes increases significantly, and the conductive network gradually changes from a sparse state to a dense state. On the other hand, the spacing between non-directly contacting carbon nanotubes decreases to the nanoscale, allowing electrons to transport across the tubes via quantum tunneling, further enhancing conductivity. Under the combined effect of these two factors, the local impedance of the material decreases significantly with increasing pressure, forming an impedance change signal that can be detected by the acquisition system. Based on this, the impedance response signal before and after pressing is acquired through electrode 12, so as to reconstruct the internal conductivity distribution based on the impedance distribution change and achieve accurate sensing of the contact position. It should be noted that in Figure 7 In the diagram, the left side shows the structure when not subjected to external pressure, and the right side shows the structure when the tactile sensor 1 is subjected to external pressure. The black strips are carbon nanotubes, and the red arrows represent one of the tunneling distances.

[0042] In summary, the aforementioned carbon nanotube hydrogel-based tactile sensor 1 utilizes a 3D-printed shell 11 adapted to the shape of the fingertip as a support. Electrodes 12 are uniformly arranged on the outer surface of the shell 11, and carbon nanotube hydrogel is used as a flexible sensing layer 13. By leveraging the piezoresistive response characteristics caused by the reconstruction of the internal conductive network and quantum tunneling effect of the carbon nanotube hydrogel under pressure, local tactile stimulation is converted into a measurable impedance change signal. This effectively solves the problems of complex structure, numerous leads, poor adhesion, and weak stability on curved surfaces with high curvature, such as the fingertip, in array-type tactile sensor solutions.

[0043] The above mainly introduced tactile sensor 1; the following will combine... Figure 1 A detailed introduction to the data acquisition and processing module.

[0044] In some embodiments, such as Figure 1 As shown, the data acquisition and processing module includes a data acquisition module and a data processing module. The data acquisition module is communicatively connected to the data processing module and electrically connected to the plurality of electrodes 12. The data acquisition module applies an excitation signal to the flexible sensing layer 13 through the electrodes 12 and measures the response signal generated by the flexible sensing layer 13 through the electrodes 12 to obtain impedance data between the electrodes, and then sends the impedance data to the data processing module. The data processing module determines the contact position information between the tactile sensor 1 and the external object based on the impedance data.

[0045] Specifically, such as Figure 1 As shown, the data acquisition module includes components such as a main control unit (MCU), a multi-channel selection switch array, an impedance measurement module, and a serial communication module. The following description, in conjunction with... Figure 1 Detailed introduction.

[0046] The main control unit is connected to a multi-channel selection switch array, an impedance measurement module, and a serial communication module, respectively. The main control unit and the impedance measurement module can communicate via the SPI protocol. Figure 1 As shown, the main control unit controls the impedance measurement module to generate a high-frequency AC excitation signal through the SPI bus, and performs multiplexing configuration based on the multiplexing switch array to switch the connection state of multiple electrodes 12 through the multiplexing switch array. An optional embodiment is provided below.

[0047] In some embodiments, such as Figure 1 As shown, the multiplexer array includes a first multiplexer and a second multiplexer, both of which are connected to the main control unit. The main control unit controls the channel selection of the first and second multiplexers. The first multiplexer is connected to multiple electrodes 12 and, under the control of the main control unit, selects an excitation electrode from the multiple electrodes 12. The second multiplexer is connected to multiple electrodes 12 and, under the control of the main control unit, selects a measurement electrode from the multiple electrodes 12.

[0048] Specifically, the multiplexer switch array is an electronic system composed of multiple multiplexers (i.e., multiplexers), which can select a specific channel from numerous input / output channels for signal conduction according to control commands. For example... Figure 1As shown, the multiplexer array includes a first multiplexer and a second multiplexer. The first multiplexer is used to select excitation electrodes (such as current injection electrodes) from the plurality of electrodes 12; the second multiplexer is used to select measurement electrodes (such as voltage measurement electrodes) from the plurality of electrodes 12. For example, selecting an excitation electrode may refer to selecting a specific electrode from the plurality of electrodes 12 through switch control to connect to a constant current source to inject a current excitation signal; selecting a measurement electrode may refer to selecting a specific electrode from the plurality of electrodes 12 through switch control to connect to a measurement circuit to acquire a voltage response signal.

[0049] For example, the main control unit independently controls two analog multiplexers via 4-bit binary address signals (A0~A3) to select 16 electrodes 12. The first multiplexer selects the excitation electrode, and the second multiplexer selects the measurement electrode. During each measurement, the two multiplexers select one path from each of the 16 electrodes 12, serving as the injection and measurement ends respectively, thus enabling impedance measurement between any two electrodes 12. The system sequentially traverses all pairwise combinations, completing 120 independent measurements and obtaining 120 sets of inter-electrode impedance data.

[0050] As described above, by using the main control unit in conjunction with two multiplexers to select the excitation electrode and the measurement electrode respectively, the pairwise combination measurement control of multiple electrodes 12 is realized with fewer hardware channels, reducing the number of independent measurement channels and wiring, and lowering the circuit complexity and hardware cost.

[0051] The impedance measurement module is used to apply an AC excitation signal of a set frequency to the flexible sensing layer 13 in the tactile sensor 1 based on the principle of AC impedance analysis, and to measure the response signal of the flexible sensing layer 13. In practical applications, the impedance measurement module can be equipped with a hardware DFT engine to directly output the real part (i.e., Z) of the impedance at the corresponding frequency. re ) and the imaginary part (i.e., Z) imSpecifically, the hardware DFT engine integrated within the impedance measurement module can perform Discrete Fourier Transform on the acquired excitation signal (such as a current signal) and response signal (such as a voltage signal), extracting high signal-to-noise ratio impedance characteristics, filtering out environmental noise and harmonic interference, accurately extracting the fundamental component with the same frequency as the excitation, and decomposing it into the real part of the resistance component in phase with the excitation and the imaginary part of the reactance component orthogonal to the excitation, thus directly outputting it for the main control unit to read. After reading the real and imaginary parts, the main control unit can calculate the inter-electrode impedance data using corresponding calculation formulas. The inter-electrode impedance data is the quantized numerical result (such as amplitude, phase, real part, and imaginary part) obtained after measuring and calculating the impedance between the electrode pairs. In this invention, the inter-electrode impedance data mainly includes the impedance amplitude and phase value. The specific calculation formula is as follows:

[0052]

[0053] Where Z represents impedance; |Z| is the magnitude of the impedance; It represents the real part of the impedance and the resistive component; The imaginary part of impedance represents the reactance component; This indicates the phase value of the impedance.

[0054] It should be noted that this invention employs a two-end full combination measurement mode when acquiring background impedance data / contact impedance data. That is, two electrodes 12 are randomly selected from multiple electrodes 12 as the excitation end and the measurement end, respectively. After traversing all combinations, multiple sets of acquired data are obtained. Each set of acquired data includes an excitation signal (such as a first excitation signal / second excitation signal) and a response signal (such as a first response signal / second response signal). Assuming the number of electrodes is N, then when acquiring background impedance data / contact impedance data, the following can be obtained: Group data collection, and then obtain Inter-electrode impedance data, this The collection of impedance data between the electrodes can be used as background impedance data / contact impedance data.

[0055] In its specific operation, this invention employs impedance differential imaging technology to achieve tactile sensing and positioning. Specifically, firstly, a full combination impedance measurement is performed on multiple electrodes 12 (e.g., 16) on the tactile sensor 1: In a non-contact reference state without pressure, any two electrodes 12 are selected from the 16 electrodes 12 as the excitation end and the measurement end, according to a 16-to-2 selection (i.e., ... A total of 120 independent data sets were collected using a combination of methods. These 120 sets of data, processed by a hardware DFT engine, yielded 120 sets of inter-electrode impedance data, which served as background impedance data (Zh). Subsequently, under external pressure on the sensor, another 120 sets of data were collected using the same electrode combination. This process also yielded 120 sets of inter-electrode impedance data, which served as contact impedance data (Zi). Theoretically, differential impedance imaging can invert the contact position using differential voltage signals. However, since the measured voltage is proportional to the impedance under small disturbance conditions, and the core objective of this method is to determine the contact position, differential impedance signals were used instead of differential voltage signals for subsequent calculations. Finally, the contact impedance data in the contact state was subtracted from the background impedance data in the non-contact reference state to obtain the differential impedance signal: ΔZ = Zi. Zh, where ΔZ is the differential impedance signal, which is the input to the subsequent data processing module. This differential impedance signal is directly used for subsequent physical prior response reconstruction and three-dimensional contact position calculation, providing the original observation basis for the entire perception and positioning process.

[0056] In summary, by controlling the multi-channel switch matrix through the main control unit and traversing all pairwise combinations of the boundary electrodes for impedance acquisition, and by acquiring data through the full combination measurement mode at both ends, this invention can achieve higher information utilization and better imaging stability. It can provide high-quality raw data support for subsequent algorithms, thereby solving the problems of high data redundancy, severe ill-conditioned inverse problem, and complex hardware gating control in existing electrical impedance imaging systems under complex curved surfaces.

[0057] As described above, after obtaining the corresponding impedance data, the main control unit will upload the impedance data to the data processing module (such as the host computer) through the serial communication module. The host computer will subtract the contact impedance data in the contact state from the background impedance data in the non-contact background state to obtain the differential impedance signal ΔZ, which will be used as the input for the subsequent physical prior reconstruction and contact position calculation model, so as to provide the original electrical data support for the subsequent high-precision contact positioning.

[0058] Specifically, the data processing module may include an input differential impedance module and a tactile localization model, among other modules. The tactile localization model may include a physical prior reconstruction layer based on the Jacobian matrix, a surface graph neural network module, and a heatmap and offset module (i.e., heatmap branch and offset branch). Specifically, the input differential impedance module determines the differential impedance signal based on the difference between the contact impedance data and the background impedance data; the physical prior reconstruction layer reconstructs the differential impedance signal into initial features of the surface vertices; the surface graph neural network module performs neighborhood propagation and fusion on the initial vertex features to obtain high-dimensional features at the vertex level; and the heatmap branch and offset branch process the high-dimensional features at the vertex level to obtain the contact probability distribution and three-dimensional local offset vector for each surface vertex. Based on this, the data processing module can perform a weighted summation based on the obtained contact probability distribution, three-dimensional local offset vector, and three-dimensional coordinates of the surface vertices to obtain the contact position information.

[0059] Figure 8 This is a flowchart illustrating the steps of a tactile sensing and localization method according to an embodiment of the present invention. This tactile sensing and localization method is applied to the tactile sensing and localization system described above, with reference to... Figure 8 The tactile sensing localization method includes the following steps 100 to 104.

[0060] Step 100: When the flexible sensing layer 13 in the tactile sensor 1 is pressed by an external object, the data acquisition and processing module applies a first excitation signal to the flexible sensing layer 13 through the electrode 12, and measures the first response signal of the flexible sensing layer 13 through the electrode 12 to obtain contact impedance data. Step 102: The data acquisition and processing module acquires the background impedance data of the tactile sensor 1 in a non-contact reference state; Step 104: The data acquisition and processing module determines the contact position information between the tactile sensor 1 and the external object based on the background impedance data and the contact impedance data.

[0061] It should be noted that the tactile perception and positioning method of the present invention can be applied to different applications and can be set according to actual conditions, without limitation here. For example, it can be applied to the field of robotic tactile perception, specifically to bionic robotic grippers or collaborative robot end effectors with high precision manipulation capabilities; used to acquire precise contact position information of robot fingertips on complex three-dimensional curved surfaces in real time; it can also be applied to the field of medical rehabilitation, specifically to intelligent tactile feedback prostheses for patients with partial finger loss; used to reconstruct the tactile perception ability of residual limbs, enabling patients to perceive external pressure stimuli and finely control prosthetic limb movements.

[0062] As described above, the beneficial effects of this invention are as follows: By acquiring contact impedance data under contact conditions and background impedance data under non-contact reference conditions, a data foundation for differential measurement is established. This allows subsequent calculations to eliminate the influence of sensor static background impedance, inherent system bias, and environmental drift based on the comparison of the two sets of data. This highlights the effective impedance change characteristics caused by external object pressure, improving the signal-to-noise ratio and reliability of the input signal. Furthermore, by determining the contact position information between the tactile sensor 1 and the external object using background impedance data and contact impedance data, precise tactile perception positioning can be achieved. This effectively eliminates system bias caused by individual sensor differences, environmental temperature changes, and hardware drift, thereby improving the accuracy and precision of the contact position information.

[0063] The following combination Figure 8 The steps in steps 100 to 104, as well as other optional steps, are described in detail.

[0064] Step 100: When the flexible sensing layer 13 in the tactile sensor 1 is pressed by an external object, the data acquisition and processing module applies a first excitation signal to the flexible sensing layer 13 through the electrode 12, and measures the first response signal of the flexible sensing layer 13 through the electrode 12 to obtain contact impedance data.

[0065] The contact impedance data refers to the set of impedance data between each electrode obtained by traversing all electrode pairs when the flexible sensing layer 13 is pressed by an external object.

[0066] The first excitation signal refers to the AC excitation signal of a set frequency applied by the data acquisition and processing module to the flexible sensing layer 13 through the electrode 12 in the contact state.

[0067] The first response signal refers to the electrical response signal generated by the flexible sensing layer 13 and collected by the electrode 12 under the action of the first excitation signal.

[0068] In some embodiments, in step 100, the data acquisition and processing module applies a first excitation signal to the flexible sensing layer 13 through the electrode 12, and measures the first response signal of the flexible sensing layer 13 through the electrode 12 to obtain contact impedance data, including: Step 200: Select any two electrodes 12 from the plurality of electrodes 12 of the tactile sensor 1 to form an electrode pair, in order to determine all possible electrode pairs; Step 202: For each electrode pair, apply the first excitation signal to one of the electrodes 12 to inject it into the flexible sensing layer 13, and measure the first response signal generated by the flexible sensing layer 13 through the other electrode 12. Step 204: Based on the first excitation signal and the first response signal corresponding to each electrode pair, calculate the inter-electrode impedance data of each electrode pair, thereby obtaining the contact impedance data of all electrode pair combinations.

[0069] In practical applications, the data acquisition module continuously applies excitation signals to the electrodes and measures the response signals to determine whether the tactile sensor is in contact with an external object by monitoring the inter-electrode impedance data of the electrode pairs in real time. Specifically, the system can use the background impedance data in a non-contact reference state as a reference. When it detects that the inter-electrode impedance data of at least some electrode pairs has changed significantly relative to the background reference value (i.e., the inter-electrode impedance data in the background impedance data) (e.g., greater than the corresponding preset value), it is determined that the tactile sensor has made contact with an external object. At this time, the data acquisition module will collect the inter-electrode impedance data of all electrode pairs according to the preset acquisition sequence, thereby obtaining the contact impedance data of all electrode pair combinations. If no inter-electrode impedance data shows a significant change, the monitoring state continues.

[0070] As described above, by traversing all electrode pair combinations, applying excitation to each electrode pair and measuring the response, and calculating the impedance value, the background impedance data under the entire combination is obtained. This full combination traversal method makes the most of the information of the boundary electrodes, providing rich and redundant observation data for subsequent differential positioning, which helps to improve positioning accuracy.

[0071] Step 102: The data acquisition and processing module acquires the background impedance data of the tactile sensor 1 in a non-contact reference state. It should be noted that this background impedance data can be pre-collected before the tactile sensing and positioning system begins formal operation. After a set of background impedance data has been pre-collected, this set of background impedance data can be reused each time "contact position information" is calculated.

[0072] The non-contact reference state refers to the initial state of the flexible sensing layer 13 of the tactile sensor 1 when it is not pressed by any external object, and the background impedance data collected at this time is used as a background reference.

[0073] The background impedance data refers to the set of impedance data between all electrodes measured and calculated by traversing all electrode pairs in a non-contact reference state, used to eliminate system static deviations. In practical applications, the corresponding background impedance data can be collected before proceeding to step 100, so that the collected background impedance data can be directly obtained for calculation when the flexible sensing layer 13 is pressed by an external object. The background impedance data collection operation can be as follows.

[0074] In some other alternative embodiments, prior to step 100, a background impedance data acquisition operation may be included, which may specifically include the following steps: Step 300: Select any two electrodes 12 from the plurality of electrodes 12 of the tactile sensor 1 to form an electrode pair, in order to determine all possible electrode pairs; Step 302: For each electrode pair, apply the second excitation signal to one of the electrodes 12 to inject it into the flexible sensing layer 13, and measure the second response signal generated by the flexible sensing layer 13 through the other electrode 12; Step 304: Based on the second excitation signal and the second response signal corresponding to each electrode pair, calculate the contact impedance data of each electrode pair, thereby obtaining the background impedance data of all electrode pair combinations.

[0075] It should be noted that the same electrode combination method is used for both the background impedance data acquisition process and the contact impedance data acquisition process. This same electrode combination method means that the electrode pairs traversed and their measurement order must be completely consistent when acquiring background impedance data and contact impedance data. For example, if electrode pairs (1,2), (1,3), (1,4)... are measured sequentially according to a preset acquisition order when acquiring background impedance data, then electrode pairs (1,2), (1,3), (1,4)... should also be measured sequentially according to the exact same acquisition order when acquiring contact impedance data, until all multiple sets (e.g., 120 sets) of electrode pairs have been traversed. In other words, the measurement order of each set of electrode pairs corresponds one-to-one in both measurements, thus ensuring strict spatial alignment of the data during subsequent differential calculations.

[0076] The second excitation signal is an AC signal applied to the flexible sensing layer 13 by the data acquisition and processing module through electrode 12, used to stimulate the electrical response of the flexible sensing layer 13. In practical applications, the second excitation signal and the first excitation signal can be signals with the same frequency and amplitude.

[0077] The second response signal is a voltage or current signal generated by the flexible sensing layer 13 under the action of the excitation signal, which contains the impedance information of the flexible sensing layer 13.

[0078] As described above, by using the same electrode combination as the non-contact reference state under the limited contact state for full-combination measurement, the consistency of the two sets of impedance data in electrode pair selection is ensured. This guarantees the accuracy and comparability of differential impedance calculation, providing a well-aligned observation data foundation for subsequent contact position calculation. Simultaneously, full utilization of all electrode pair information provides rich and reliable data for subsequent positioning.

[0079] Step 104: The data acquisition and processing module determines the contact position information between the tactile sensor 1 and the external object based on the background impedance data and the contact impedance data.

[0080] The contact position information refers to the three-dimensional contact position coordinates of the external object on the curved surface of the tactile sensor 1.

[0081] In some embodiments, in step 104, the data acquisition and processing module determines the contact position information between the tactile sensor 1 and the external object based on the background impedance data and the contact impedance data, including: Step 400: Determine the differential impedance signal based on the difference between the contact impedance data and the background impedance data; Step 402: Determine the contact position information based on the differential impedance signal.

[0082] The differential impedance signal refers to the difference obtained by subtracting the contact impedance data from the background impedance data. It is used to highlight the impedance change caused by pressure and eliminate static background and system deviations. In practical applications, the data acquisition and processing module includes an input differential impedance module, which can calculate the differential impedance signal. Specifically, after receiving the background impedance data (Zh) and contact impedance data (Zi) sent by the data acquisition module, the input differential impedance module can perform the subtraction (ΔZ = Zi). Zh) calculates the differential impedance signal to eliminate static background and system deviation, highlighting the impedance change characteristics caused by contact, thus obtaining a 120-dimensional differential impedance signal ΔZ, which is the output of the input differential impedance module.

[0083] As described above, by subtracting the contact impedance data from the background impedance data to obtain the differential impedance signal, the static background and inherent noise of the system can be eliminated through differential processing, highlighting the effective impedance change characteristics caused by contact, thereby improving the signal-to-noise ratio of the input signal used for position calculation and the accuracy of subsequent positioning.

[0084] In some embodiments, determining the contact position information based on the differential impedance signal in step 402 may include: inputting the differential impedance signal into a preset tactile positioning model to obtain the contact position information.

[0085] It should be noted that the tactile localization model is an algorithmic model used to map differential impedance signals into contact position information. It may include a physical prior reconstruction layer based on the Jacobian matrix, a surface graph neural network module, a heatmap and offset module (i.e., heatmap branches and offset branches), and a decoding output module. Therefore, in step 402, inputting the differential impedance signal into the preset tactile localization model to obtain the contact position information may include the following steps: Step 500: Input the differential impedance signal into the physical prior reconstruction layer, and reconstruct the differential impedance signal into the initial features of the surface vertices in the physical prior reconstruction layer; Step 502: Input the initial features of the vertices of the surface into the surface graph neural network module for neighborhood propagation and fusion to obtain high-dimensional features at the vertex level; Step 504: Input the vertex-level high-dimensional features into the heatmap branch and the offset branch respectively to obtain the contact probability distribution and three-dimensional local offset vector of each surface vertex. Step 506: The contact position information is obtained by weighted summation of the contact probability distribution, the three-dimensional local offset vector, and the three-dimensional coordinates of the surface vertex.

[0086] For example, the following details the steps for calculating contact position information in conjunction with the specific structure of the tactile localization model: (1) Physical Prior Reconstruction Layer The input to the physical prior reconstruction layer is the differential impedance signal ΔZ, and its output is the initial features of the surface vertices, corresponding one-to-one with the surface vertices, i.e., the initial feature matrix H(0). Each initial feature of the surface vertex includes the physical response, the geometric coordinates of the surface vertex, and boundary marker information. Here, the surface vertex refers to the point element of the finite element mesh on the surface of the tactile sensor. In practical applications, after dividing the outer surface of the tactile sensor into a finite element mesh, the point elements of the finite element mesh are the surface vertices.

[0087] It should be noted that in the contact sensing and positioning task, the core is to infer the change in conductivity caused by contact inside the tactile sensor 1 through the change in boundary impedance. This is a typical inverse problem, which has inherent difficulties in solving: First, the boundary impedance is low-dimensional observation data, while the internal conductivity is a continuous high-dimensional distribution, resulting in insufficient observation information; second, the impedance change and conductivity change have a strong nonlinear relationship, making direct modeling difficult; third, simultaneous measurement noise easily amplifies errors, leading to poor stability of direct inversion. It should be noted that the aforementioned boundary impedance change refers to the difference between the inter-electrode impedance data (i.e., impedance values) measured between the multiple electrodes 12 in the tactile sensor 1 before and after being pressed by an external object.

[0088] To address this, this invention introduces a physical prior reconstruction layer based on the Jacobian matrix. This layer provides a linear approximate initial solution to the inverse problem based on physical laws, transforming low-dimensional impedance changes into an internal response distribution with clear physical meaning. This provides reliable physical prior information for subsequent models, reducing the learning difficulty of cross-domain mapping. Specifically, this invention pre-calculates the Jacobian matrix (J) based on the sensor's finite element model and sets it in the physical prior reconstruction layer. The Jacobian matrix is ​​a sensitivity matrix, where each element characterizes the influence of the conductivity change of a single volume element within the tactile sensor 1 on the impedance changes between electrodes, establishing a linear mapping relationship between the internal conductivity distribution and boundary impedance measurement. It should be noted that the volume element refers to each triangular facet obtained by dividing the surface of the tactile sensor into a finite element mesh (such as a triangular finite element mesh). When dividing the surface into triangular finite element meshes, the meshing object is limited to the outer surface of the tactile sensor (not the entire sensor volume). Each triangular finite element mesh represents a small region on the sensor surface, which is a volume element.

[0089] In practical applications, actual tactile contact is typically limited to a small, localized area of ​​pressure, resulting in a limited change in internal conductivity, thus satisfying the small perturbation condition. Under this condition, the effect of internal conductivity change on boundary impedance can be approximated as a linear relationship as follows:

[0090] Where ΔZ is the differential impedance signal; J is the Jacobian matrix; This represents the conductivity change of the internal volume unit of the tactile sensor 1. Theoretically, the conductivity change Δσ can be directly inferred from the differential impedance signal ΔZ based on the above linear relationship. However, since the stability of directly inferring the conductivity change from the differential impedance signal ΔZ is poor, this invention introduces a pseudo-inverse solution strategy with regularization. The pseudo-inverse solution formula with regularization is as follows:

[0091] in, The initial response distribution in the volume element space is directly related to the change in conductivity. The subscript e represents a triangular finite element mesh (i.e., a volume element), which means that the volume element is the smallest solution unit. Let J be the transpose of the Jacobian matrix, used to constrain linear systems to improve solution stability. It is the transpose of the Jacobian matrix; This indicates that the two are multiplied together, and is used to solve for pseudo-inverses; Let represent the pseudo-inverse matrix with regularization constraints, where For diagonal regularization, noise amplification is suppressed by weighting according to unit sensitivity; λ represents a learnable parameter that can adaptively adjust the regularization strength during training. Indicates taking The diagonal matrix is ​​formed by the diagonal elements; ΔZ is the differential impedance signal.

[0092] As shown in the above "Pseudo-inverse solution formula with regularization", this invention adds a diagonal regularization constraint term to the solution process of the above pseudo-inverse solution strategy. This term is weighted by unit sensitivity to suppress the amplification of measurement noise and improve the solution stability of the linear system. Simultaneously, the regularization strength is controlled by a learnable parameter λ, which can be adaptively adjusted during training to balance noise reduction and fidelity preservation. The initial response distribution of the volume unit space is obtained after solving. This initial response distribution is directly related to the change in internal conductivity, reflecting the degree of contact disturbance to each volume unit.

[0093] It should be noted that the above regularized pseudo-inverse solution yields the response distribution in the volume element space, while subsequent feature learning needs to be carried out based on the surface vertices. Therefore, the volume element responses need to be further mapped to the surface vertices. This mapping process consists of two core steps: volume element response reconstruction and surface vertex feature aggregation, achieving an effective transformation from boundary observations to surface features.

[0094] The first step is to substitute the differential impedance signal ΔZ into the above "pseudo-inverse solution formula with regularization" to obtain the response information (i.e. response value) corresponding to each volume element. The magnitude of the value reflects the degree of contact disturbance to the corresponding volume element. The larger the value, the closer the element is to the contact area. Finally, a response distribution covering the entire tactile sensor 1 volume element grid is formed, providing physical spatial clues for subsequent feature conversion.

[0095] The second step involves obtaining the initial response distribution of the volumetric unit space through a pseudo-inverse solution with regularization, since subsequent feature learning is based on surface vertices. This initial response distribution is then mapped to the surface vertices using area-weighted aggregation, forming the initial features of the surface vertices. Specifically, area-weighted averaging can be used to achieve information aggregation, with the aggregation formula as follows:

[0096] in, Let S be the response feature of the i-th surface vertex (i.e., the physical response feature in the initial features of the surface vertex), where the subscript v (i.e., vertex) represents the surface vertex in the triangular finite element mesh (i.e., volume element); the superscript (i) is the attribution marker, indicating that the response feature is the aggregation result of the i-th surface vertex itself; e This represents the area of ​​the corresponding volume element; This represents traversing all volume elements e containing the i-th surface vertex, where... Let e ​​represent the set of all volume elements associated with the i-th surface vertex. A specific unit within this; it should be noted that both e and e' are sets. Each element represents a single unit within the summation operation, but the two operations are performed at different levels and are independent of each other. Therefore, to avoid variable confusion, they are distinguished by a single quote ('). Here, 'e' is the loop variable for the outer summation, and 'e' is the loop variable for the inner summation. This represents the summation of the areas of all volume elements associated with the i-th surface vertex; This represents the response value of volume element e.

[0097] In other words, when using the area-weighted average method to achieve information aggregation, for each surface vertex, all volume elements containing that surface vertex are identified. The response information of the associated volume elements around each surface vertex is weighted and summed according to the ratio of the area of ​​each volume element to the total area of ​​the associated volume elements of that surface vertex. This ensures that the volume elements closer to the surface vertex and with larger areas contribute more to the surface vertex features. As a result, the response distribution of the volume element space is smoothly projected onto the surface vertex, forming the initial surface vertex features that correspond one-to-one with the surface vertex, thus completing the effective connection between physical information and surface features.

[0098] As described above, the differential impedance signal is mapped to the volume element response distribution by solving the pseudo-inverse with regularization, and the volume element information is mapped to the surface vertices by using area-weighted aggregation. This achieves smooth information transfer from the volume element to the surface vertices while suppressing noise amplification, providing reliable input features with physical meaning for subsequent neural networks.

[0099] (2) Surface graph neural network module (i.e., surface graph neural network feature extraction module) The input to the surface graph neural network module is the initial feature matrix H(0) of the surface vertices and the undirected graph structure constructed from the grid adjacency relationships. Correspondingly, the output of the surface graph neural network module is a vertex-level high-dimensional feature matrix H(L), where each surface vertex can correspond to 128-dimensional hidden features. These vertex-level high-dimensional features include contact region distribution, surface topology, and local geometric information. In practical applications, multi-layer graph convolution operators with residual connections can be used to propagate, fuse, and abstract vertex features layer by layer within the local neighborhood of the surface, capturing the spatial distribution pattern of contact perturbations and adapting to the feature learning needs of high curvature and irregular topological structures.

[0100] Specifically, Figure 9This is a schematic diagram of a surface graph neural network module disclosed in an embodiment of the present invention. It should be noted that the surface graph neural network module is used to achieve layer-by-layer abstraction and neighborhood fusion of vertex features from initial physical geometric features to high-dimensional discriminative features on a three-dimensional fingertip surface mesh, providing a feature basis for subsequent contact position decoding.

[0101] First, the 3D fingertip surface mesh (the triangular finite element mesh obtained by dividing the surface of the tactile sensor into triangular finite element meshes) is abstracted as an undirected graph structure. Specifically, each vertex of the 3D fingertip surface mesh is used as a graph node, and the mesh adjacency relationship between the vertices is used as the graph edge, so that the nodes are connected to each other according to the surface topology, thus forming an undirected graph structure. Based on this, an initial feature matrix is ​​constructed for each surface vertex. This initial feature matrix contains six dimensions, specifically composed of the vertex-level spatial response features output by the physical prior layer (one dimension each for positive and negative responses, for a total of two dimensions), the normalized 3D coordinates of the surface vertex (three dimensions), and the boundary markers of the surface vertex (one dimension), thereby realizing the fusion input of physical and geometric information.

[0102] The main body of the aforementioned surface graph neural network module adopts a multi-layer residual graph convolutional structure, such as... Figure 8 As shown, it contains multiple (e.g., four) graph convolutional modules. Each graph convolutional module sequentially includes a graph convolution operator, a ReLU activation function (i.e., a linear activation function), and residual connections. During feature propagation, the first-layer graph convolutional module takes the initial feature matrix and the adjacency relationship of the surface mesh as input. It aggregates the feature information of each surface vertex and its adjacent surface vertices through the graph convolution operator to obtain the output features of the first layer. Subsequent graph convolutional modules take the output features of the previous layer as input, first perform neighborhood feature aggregation through the graph convolution operator, then introduce a nonlinear transformation through the ReLU activation function, and finally directly superimpose the features of the previous layer onto the output of the current layer to form residual connections. This process is repeated for the second to fourth layers to complete the feature propagation, ultimately outputting a vertex-level high-dimensional feature matrix H(L).

[0103] It should be noted that the graph convolution operator aggregates vertex features based on the adjacency relationships of the surface mesh, enabling each surface vertex to fuse information from itself and its neighboring surface vertices, thereby expanding the receptive field layer by layer and capturing the spatial distribution patterns of contact perturbations. The residual connection directly superimposes the features from the previous layer onto the output of the current layer, effectively alleviating the feature degradation problem in deep network training. The ReLU activation function introduces a nonlinear transformation, enhancing the network's ability to fit complex contact patterns. After layer-by-layer propagation and fusion through multiple graph convolution modules, the surface graph neural network module finally outputs a high-dimensional feature matrix at the vertex level. Each surface vertex corresponds to a multi-dimensional (e.g., 128-dimensional) high-dimensional discriminative feature vector, providing core input for subsequent heatmap prediction and offset regression.

[0104] (3) Heatmap and Offset Module (i.e., Heatmap Branch and Offset Branch) The heatmap branch is a linear transformation branch in the haptic localization model used to predict the contact probability distribution of each surface vertex. Its input is the vertex-level high-dimensional feature matrix H(L), and its output is the surface vertex contact probability distribution Pi. In practical applications, the heatmap branch can obtain the surface vertex contact probability distribution Pi of each surface vertex by performing linear transformation and Softmax normalization operations on the vertex-level high-dimensional feature matrix, thus forming a surface contact heatmap. Specifically, firstly, the high-dimensional features of each surface vertex are mapped to a raw score of contact probability through a linear layer. Subsequently, the raw scores of each surface vertex are normalized by the Softmax function, so that the sum of the contact probabilities of all surface vertices equals 1, thereby obtaining the contact probability distribution defined on the surface mesh, and thus obtaining the surface vertex contact probability distribution Pi of each surface vertex, forming a contact heatmap, and achieving coarse localization of the contact area.

[0105] The offset branch is a linear transformation branch in the haptic localization model used to predict the 3D local offset vectors of each surface vertex. Its input is a high-dimensional feature matrix H(L) at the vertex level, and its output is a 3D offset vector Δi at the vertex level. In practical applications, the offset branch can predict the vertex-level 3D local correction vector Δi of each surface vertex through linear transformation, used for sub-vertex-level position refinement. Specifically, to overcome the limitations of discrete mesh resolution, the offset branch needs to predict the 3D local offset vector of each surface vertex based on the same high-dimensional feature matrix H(L) through linear transformation. This 3D local offset vector represents the spatial correction amount from the corresponding surface vertex to the actual contact point, used for continuous spatial refinement of the discrete surface vertex positions. To ensure training stability, the offset can be constrained by amplitude scaling and the hyperbolic tangent activation function to avoid excessively large correction amounts.

[0106] (4) Decoding output module The decoding output module takes as input the vertex contact probability distribution Pi, the vertex 3D offset vector Δi, and the 3D coordinates vi of the surface vertices, and outputs a high-precision, continuous 3D contact position. In practical applications, the contact probability distribution Pi output from the heatmap branch and the 3D local offset vector Δi output from the offset branch can be combined using a weighted summation formula for joint decoding. Specifically, using the contact probability of each surface vertex in the vertex contact probability distribution Pi as weight, the sum of the 3D coordinates of each surface vertex and its corresponding 3D local offset vector Δi is weighted and fused. This utilizes the heatmap to lock the core contact area and uses the offset to achieve sub-vertex level position refinement, ultimately outputting a high-precision, continuous 3D contact position. The weighted summation formula is as follows:

[0107] Where Ppred represents the three-dimensional contact position information; i represents the surface vertex in the triangular finite element mesh; N v P represents the number of surface vertices in the triangular finite element mesh after dividing the entire surface of the tactile sensor into triangular meshes; i V represents the contact probability of the i-th surface vertex; i Δi represents the three-dimensional coordinates of the i-th surface vertex; Δi represents the three-dimensional local offset vector corresponding to the i-th surface vertex.

[0108] In summary, this invention first utilizes a physical prior layer based on the Jacobian matrix to map low-dimensional impedance changes into the initial response distribution of the volume unit space, and then converts this into initial features of the surface vertices through area-weighted aggregation, providing reliable physical prior information for subsequent learning. Subsequently, the surface graph neural network module abstracts the irregular mesh of the three-dimensional fingertip surface into an undirected graph structure, using the mesh surface vertices as graph nodes and the connections between adjacent surface vertices as graph edges. Through multiple layers of graph convolution operators with residual connections, vertex features are propagated, fused, and abstracted layer by layer within the local neighborhood of the surface, enabling the network to effectively capture the spatial distribution pattern of contact disturbances on the surface and adapt to the feature learning requirements of high curvature and irregular topological structures. Finally, a joint decoding strategy of heatmap and offset is adopted to construct a dual-branch output based on the high-dimensional features at the vertex level: the heatmap branch outputs the probability distribution of each surface vertex belonging to the contact region, locking the contact core region; the offset regression branch predicts the three-dimensional local correction vector of each surface vertex, refining the discrete surface vertex positions in a sub-vertex continuous space. Finally, the probability-weighted surface vertex coordinates and probability-weighted offset correction are fused to output a high-precision, continuous 3D contact position, breaking through the limitation of grid discrete resolution and achieving sub-vertex level positioning accuracy.

[0109] As described above, by introducing the sensitivity information of the Jacobian matrix through the physical prior reconstruction layer to generate the initial features of the surface vertices, using the surface graph neural network module to process the irregular surface mesh topology, and combining the heat map and offset branch to achieve sub-vertex level decoding, the model can adapt to the surface structure, breaking through the problems of low positioning accuracy and artifact superposition in the boundary region of traditional algorithms, and realizing high-precision calculation of the three-dimensional contact position.

[0110] To make this application easier to understand, an exemplary application is provided below. This exemplary application provides a tactile sensing system that can be directly applied to fine manipulation tasks at the fingertips of a robot. This tactile sensing system mainly includes a tactile sensor 1, a data acquisition module, and a data processing module (such as a host computer). Based on the above tactile sensing system, the specific implementation method for tactile sensing by the robot can be as follows.

[0111] S1, the tactile sensor 1 of the present invention is installed at the fingertip of the robot end effector to form a curved fingertip. The tactile sensor 1 includes, from the inside to the outside, a 3D printed shell 11, 16 electrodes 12, a flexible sensing layer 13 and a flexible protective film.

[0112] S2. Before the robot begins its formal operation, it collects background impedance data using the data acquisition module. This module includes a main control unit, an impedance measurement module, a serial communication module, and two multiplexers. It should be noted that this background impedance data is collected when the tactile sensor 1 is in a non-contact reference state. The specific steps for collecting the background impedance data are as follows: The main control unit controls two multiplexers to sequentially select excitation and measurement electrodes from 16 electrodes 12 in a pairwise combination manner (a total of 120 groups). The main control unit also controls the impedance measurement module to apply the corresponding excitation signal and collect the response signal. The hardware DFT in the impedance measurement module calculates 120 sets of real and imaginary impedance parts based on the excitation and response signals and sends them to the main control unit. The main control unit obtains 120 sets of inter-electrode impedance data based on these 120 sets of real and imaginary impedance parts. The set of these 120 sets of inter-electrode impedance data is the background impedance data.

[0113] S3, when the robot is working, the curved surface of the curved fingertip makes light contact or gripping contact with the surface of an external object. The contact pressure causes a local change in the conductivity of the flexible sensing layer 13, which is made of carbon nanotube hydrogel, inside the tactile sensor 1, which in turn causes a change in the impedance between the 16 boundary electrodes.

[0114] S4, based on the data acquisition module, the contact impedance data is acquired and processed when the flexible sensing layer 13 in the tactile sensor 1 is pressed by an external object.

[0115] It should be noted that the specific acquisition operation of the contact impedance data is the same as that of the "specific acquisition operation of the background impedance data" described above. It will also obtain 120 sets of inter-electrode impedance data, and the collection of these 120 sets of inter-electrode impedance data is the contact impedance data. The main difference between the two lies in the time reference of the acquisition. The background impedance data is acquired when the tactile sensor 1 is in a non-contact reference state, while the contact impedance data is acquired when the flexible sensing layer 13 in the tactile sensor 1 is pressed by an external object.

[0116] S5, the main control unit uploads the background impedance data and contact impedance data to the host computer.

[0117] S6, the host computer calculates the differential impedance signal before and after contact, that is, the difference between the background impedance data and the contact impedance data to obtain the differential impedance signal.

[0118] S7, the host computer inputs the differential impedance signal into the tactile positioning model. In this tactile positioning model, the volume unit response is first reconstructed and mapped to the surface vertex through the physical prior layer based on the Jacobian matrix. Then, the high-dimensional features of the vertex are extracted through the surface graph neural network module. Finally, the heat map and offset module jointly decode and output the sub-vertex level three-dimensional contact position information.

[0119] Based on this real-time, high-precision contact position information, the robot's corresponding control system enables precise interactive tasks such as grasping flexible objects, precision assembly, and gentle handling of fragile items, effectively improving the robot's end-effector perception capabilities and operational stability.

[0120] Similarly, this invention can also be integrated into applications such as the fingertips of intelligent haptic feedback prostheses, which can reconstruct the tactile perception of residual limbs, enabling patients to perceive the contact position and intensity in real time, assisting in fine grasping and daily operations, and improving the rehabilitation experience.

[0121] It should be clearly stated that the core objective of this invention is to design a finger-sleeve-type EIT tactile sensor 1, a high-precision impedance acquisition system, and a tactile perception and localization method that integrates physical priors and graph neural networks (GNNs) for highly curved fingertips (such as bionic robot grippers and intelligent prosthetic fingertips). This enables real-time, high-precision calculation of the three-dimensional coordinates of the contact point from the sensor boundary impedance signal. In short, the core is to accurately obtain the three-dimensional spatial coordinates of the contact point between the finger sleeve surface and the object through the designed sensor and algorithm, providing a reliable positional perception foundation for precise robot manipulation and prosthetic tactile feedback.

[0122] In summary, according to the tactile sensing and positioning system of the present invention: (1) The present invention adopts a three-dimensional structure and surface electrode arrangement scheme of finger sleeve, and based on the principle of electrical impedance imaging, realizes the high integration of tactile sensor 1 on the high curvature surface of the fingertip, which effectively solves the problem of complex internal structure of traditional array sensors when applied to the fingertip of robots. (2) The carbon nanotube hydrogel sensing material used in the present invention simulates the biological characteristics of human skin. This material has excellent flexibility and piezoresistive response, and can capture tiny tactile stimuli in real time and sensitively, enhancing the reliability of tactile sensor 1 in human-computer interaction and medical rehabilitation. (3) The present invention uses a hybrid physical prior and graph neural network positioning method, which significantly improves the sensing accuracy and system robustness under complex curved surfaces. This algorithm solves the problems of low positioning accuracy and artifact superposition in the boundary area of ​​traditional algorithms by explicitly introducing physical sensitivity information and using graph convolution to process non-Euclidean space features. (4) To address the problems of existing tactile localization methods, such as the difficulty in mapping three-dimensional irregular topological structures, the incompatibility of traditional CNNs with non-Euclidean spaces, and the low localization accuracy in boundary regions, this invention provides a fingertip EIT tactile perception localization method that combines physical priors and graph neural networks. This method first uses a Jacobi matrix physical prior layer to map low-dimensional voltage to a spatial response distribution. Then, it uses a surface graph neural network to realize neighborhood feature propagation on irregular grids and combines a heat map and offset joint decoding strategy to achieve high-precision mapping of three-dimensional contact positions at the sub-vertex level. (5) This invention aims to achieve high-precision mapping from boundary impedance signals to surface contact positions. First, the differential impedance signals of the sensor before and after being compressed are obtained and input into the algorithm preprocessing module. Second, the algorithm introduces a physical prior reconstruction layer based on the Jacobi matrix, which explicitly uses the sensor's sensitivity matrix physical information to initially map low-dimensional electrical measurement values ​​to a spatial response feature distribution with physical meaning, thereby narrowing the mapping gap between the observation domain and the geometric space domain. Furthermore, a surface graph neural network extraction module was constructed, abstracting the fingertip surface mesh into an irregular undirected graph structure. Multi-layer graph convolution operators with residual connections were used to achieve feature propagation and deep fusion within the vertex neighborhood of the surface. Finally, a joint decoding strategy combining heatmap and offset modules was adopted. The heatmap branch was used to pinpoint the core region with the highest contact probability, and the offset branch was used to continuously correct the coordinates of discrete surface vertices at the sub-vertex level, ultimately outputting refined 3D coordinates of the contact points.

[0123] It should be understood that the terms "first," "second," etc., are used in this invention to describe various types of information, but these terms are not limited to them and are only used to distinguish information of the same type from each other. For example, without departing from the scope of this invention, "first" information can also be referred to as "second" information, and similarly, "second" information can also be referred to as "first" information.

[0124] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A tactile perception and localization method, characterized in that, It is applied to a tactile sensing and positioning system, including a tactile sensor and a data acquisition and processing module; The tactile sensor includes a housing, a flexible sensing layer, and multiple electrodes. The housing has a receiving cavity that is adapted to the shape of the target object. The electrodes are fixed to the housing, and one end of the electrode away from the housing is connected to the flexible sensing layer. The flexible sensing layer covers the housing and is made of carbon nanotube hydrogel. The flexible sensing layer is used to cause a change in the impedance data between at least some of the electrodes when pressed by an external object. The data acquisition and processing module is electrically connected to the plurality of electrodes. The data acquisition and processing module is used to apply an excitation signal to the flexible sensing layer through the electrodes and measure the response signal of the flexible sensing layer through the electrodes to obtain impedance data between the electrodes and determine the contact position information between the tactile sensor and the external object based on the impedance data. The method includes: When the flexible sensing layer in the tactile sensor is pressed by an external object, the data acquisition and processing module applies a first excitation signal to the flexible sensing layer through electrodes, and measures the first response signal of the flexible sensing layer through electrodes to obtain contact impedance data. The data acquisition and processing module obtains the background impedance data of the tactile sensor in a non-contact reference state; The data acquisition and processing module determines the contact position information between the tactile sensor and the external object based on the background impedance data and the contact impedance data, including: The differential impedance signal is determined based on the difference between the contact impedance data and the background impedance data; The contact position information is determined based on the differential impedance signal, specifically by inputting the differential impedance signal into a preset tactile positioning model to obtain the contact position information. The tactile localization model includes a physical prior reconstruction layer based on the Jacobian matrix, a surface graph neural network module, a heat map branch, and an offset branch. Correspondingly, the differential impedance signal is input into a preset tactile positioning model to obtain the contact position information, including: The differential impedance signal is input into the physical prior reconstruction layer, and the differential impedance signal is reconstructed into the initial features of the surface vertices in the physical prior reconstruction layer; The initial features of the surface vertices are input into the surface graph neural network module for neighborhood propagation and fusion to obtain high-dimensional features at the vertex level; The vertex-level high-dimensional features are input into the heatmap branch and the offset branch respectively to determine the contact probability distribution and three-dimensional local offset vector of each surface vertex. The contact position information is obtained by weighted summation of the contact probability distribution, the three-dimensional local offset vector, and the three-dimensional coordinates of the surface vertices.

2. The tactile perception and positioning method according to claim 1, characterized in that, The data acquisition and processing module includes a data acquisition module and a data processing module, and the data acquisition module and the data processing module are communicatively connected. The data acquisition module is electrically connected to the plurality of electrodes. The data acquisition module is used to apply an excitation signal to the flexible sensing layer through the electrodes and measure the response signal of the flexible sensing layer through the electrodes to obtain impedance data between the electrodes, and then send the impedance data to the data processing module. The data processing module is used to determine the contact position information between the tactile sensor and the external object based on the impedance data.

3. The tactile perception and positioning method according to claim 2, characterized in that, The data acquisition module includes a main control unit, a first multiplexer, and a second multiplexer; The main control unit is connected to the first multiplexer and the second multiplexer respectively, and the main control unit is used to control the channel selection of the first multiplexer and the second multiplexer; The first multiplexer is connected to multiple electrodes, and the first multiplexer is used to select the excitation electrode from the multiple electrodes under the control of the main control unit; The second multiplexer is connected to multiple electrodes and is used to select a measurement electrode from the multiple electrodes under the control of the main control unit.

4. The tactile perception and positioning method according to claim 1, characterized in that, It also includes a flexible protective film, which is sleeved on the outer surface of the flexible sensing layer.

5. The tactile perception and positioning method according to claim 1, characterized in that, The data acquisition and processing module applies a first excitation signal to the flexible sensing layer through electrodes, and measures the first response signal of the flexible sensing layer through electrodes to obtain contact impedance data, including: Two electrodes are randomly selected from the plurality of electrodes of the tactile sensor to form an electrode pair, thereby determining all possible electrode pairs; For each electrode pair, the first excitation signal is applied to one of the electrodes to inject into the flexible sensing layer, and the first response signal generated by the flexible sensing layer is measured through the other electrode; Based on the first excitation signal and the first response signal corresponding to each electrode pair, the inter-electrode impedance data of each electrode pair is calculated, thereby obtaining the contact impedance data of all electrode pair combinations.

6. The tactile perception and localization method according to claim 1, characterized in that, The process of reconstructing the differential impedance signal into initial features of the surface vertices based on the Jacobian matrix includes: The initial response distribution of the volume element space is obtained by solving the pseudo-inverse with regularization; The initial response distribution of the volume element space is mapped to the surface vertices by area-weighted aggregation, forming the initial features of the surface vertices.

Images