The invention discloses a multi-mode visual
tactile sensor and a sensing method thereof. The sensor is composed of a sensing cover and an imaging box body, the sensing cover is fixed on the imaging box body, the sensing cover is provided with a flexible sensing layer, an embedded mark point array combining
inert ink and thermochromic ink is arranged in the sensing layer, during tactile sensing, a
light source is provided through the imaging box body, and multi-mode
sensing data is obtained. Therefore, displacement tracking and
temperature sensing are realized at the same time. According to the invention, multi-mode sensing of five
modes of a two-dimensional texture pattern, a three-dimensional texture image, a three-dimensional
displacement field, a three-dimensional force field and a two-dimensional temperature field can be realized under a single optical frame, real-time co-registration measurement can be realized, and a high-integration-level solution is provided for a
robot touch
system. The problems that an existing visual
tactile sensor cannot sense multi-mode information synchronously, force field reconstruction calculation is complex, and
temperature sensing capacity is lacked are solved, and the visual
tactile sensor is suitable for application scenes such as
robot tactile sensing, man-
machine interaction, intelligent
artificial limbs and industrial detection.