This invention discloses a visual-tactile fusion multimodal
tactile sensor and a three-dimensional pressure field
reconstruction method, relating to the technical field of
tactile perception. The sensor includes a visual acquisition unit, a tactile acquisition unit, a gel component, a
data processing unit, and a display and interaction unit, achieving dual-
modal collaborative
perception. The three-dimensional pressure field
reconstruction method sequentially executes steps such as force
system initialization, synchronous acquisition, image preprocessing, pressure calibration calculation, cross-
modal registration, dynamic
Gaussian convolution fusion, 2D-to-3D mapping,
point cloud and mesh reconstruction, real-time display and interaction,
data recording, and reset maintenance. It utilizes dynamic
Gaussian convolution kernels to simulate
pressure diffusion and generates a three-dimensional pressure field model based on the Open3D
library. This invention reduces the cost of the gel component, and the reconstruction
delay of the three-dimensional pressure field does not exceed 8 milliseconds, making it suitable for fields such as
robot operation, intelligent prosthetics, and human-computer interaction.