A tactile feature extraction method and device based on a parallel symmetric tactile sensor

Abstract

The application relates to a tactile feature extraction method and device based on a parallel symmetrical visual tactile sensor, wherein the method comprises the following steps: acquiring a pair of tactile images output by the visual tactile sensor; performing air distortion correction and spatial consistency calibration on the pair of tactile images; respectively adopting an efficient filtering algorithm, adopting a layered Canny edge detection algorithm and adopting a Poisson method to perform feature extraction on the corrected tactile images, so as to respectively obtain a tactile feature optical flow field, obtain a tactile feature edge field and obtain a tactile feature depth field; establishing a tactile feature contact force field according to a nonlinear corresponding relation curve between a contact surface pressure and the tactile feature depth field; and linearly superimposing the tactile feature optical flow field, the edge field, the depth field and the contact force field to construct a four-dimensional tactile feature. Compared with the prior art, the method has the advantages of fast calculation speed, meeting the needs of rapid real-time performance, being capable of solving the problems of super-resolution, high precision and multi-dimensional tactile feature extraction, and the extracted tactile feature having a more comprehensive tactile feature representation. The device adopts an integrated modularization and has the advantages of small volume and the like.

Description

Technical Field

[0001] This invention relates to the field of robot tactile perception, and in particular to a method and apparatus for extracting tactile features based on a parallel symmetric visual-tactile sensor. Background Technology

[0002] Traditional visual grasping processes can observe the object being grasped using global or local RGB or RGB-D images, achieving stable grasping operations in structured environments. However, in unstructured environments, relying solely on visual perception is susceptible to limitations imposed by lighting and visual occlusion. It can only provide partial information about the object's surface texture and other surface features, failing to effectively perceive the object's physical properties, such as shape, stiffness, and texture, as well as the direct contact state with the object, such as pressure, contact area, and sliding. This leads to excessive contact force being applied to deformable objects during grasping, affecting the stability of the grasping task and significantly limiting the application range of robotic grasping.

[0003] Tactile feature extraction aims to explore how human and machine tactile systems acquire and interpret tactile features from their environment. These tactile features include information such as object shape, surface texture, and contact force characteristics, which help tactile systems model their environment. The performance of a tactile system depends not only on the sensing device but also on the learning method's interpretation of the information in the tactile data. Current vision-based tactile sensing offers new insights into tactile understanding. Chinese patent CN111291677A proposes a method for dynamic video tactile feature extraction and rendering. This method directly processes the input dynamic video to extract tactile features. By decompressing the received video, the shots are segmented based on inter-frame color histogram features. Spatiotemporal tactile saliency features are extracted from all frames of each segmented shot. Gaussian pyramids, LTTI algorithms, and other methods are used to extract optical flow sequences and calculate dynamic saliency. However, this method only extracts optical flow features and does not comprehensively consider multi-dimensional information such as contact edges, depth, and contact force, resulting in insufficiently comprehensive extracted tactile features. Therefore, the present invention provides a method and apparatus for extracting tactile features based on a parallel symmetric visual-tactile sensor. Summary of the Invention

[0004] The purpose of this invention is to overcome the defects of the prior art and provide a method and apparatus for extracting tactile features based on a parallel symmetric visual-tactile sensor.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] On one hand, this invention discloses a method for extracting tactile features based on a parallel symmetric visual-tactile sensor, comprising the following steps:

[0007] Step S1: Construct a tactile feature extractor with a parallel symmetric structure based on the visual-tactile sensor to obtain a pair of tactile images output by the visual-tactile sensor;

[0008] Step S2: Use a virtual plane and a virtual camera to perform spatial distortion correction and spatial consistency calibration on the pair of tactile images to obtain the corrected tactile images;

[0009] Step S3: Use an efficient filtering algorithm to extract features from the corrected tactile image to obtain the tactile feature optical flow field;

[0010] Step S4: Use the layered Canny edge detection algorithm and opening / closing operations to extract features from the corrected tactile image to obtain the tactile feature edge field;

[0011] Step S5: Reconstruct the corrected tactile image using the Poisson method and gradient difference to obtain the tactile feature depth field;

[0012] Step S6: Establish the tactile feature contact force field based on the nonlinear correspondence curve between the contact surface pressure and the tactile feature depth field;

[0013] Step S7: The tactile feature extractor linearly superimposes the tactile feature optical flow field, edge field, depth field and contact force field to construct a four-dimensional tactile feature.

[0014] Furthermore, the parallel symmetrical structure is a parallel two-finger gripper structure equipped with two visual-tactile sensors.

[0015] Furthermore, the virtual plane is an intersecting plane that is of common interest to two parallel and symmetrically mounted visual-tactile sensors.

[0016] Furthermore, the distortion correction and spatial consistency calibration include the following steps:

[0017] Step S201: Obtain the original tactile images output by two parallel and symmetrical visual-tactile sensors;

[0018] Step S202: Select the virtual plane and calculate the intrinsic and extrinsic parameters of the virtual camera;

[0019] Step S203: Perform distortion correction on the original tactile image based on the intrinsic and extrinsic parameters of the virtual camera;

[0020] Step S204: Select multiple corresponding marker points as projection points on the distortion-corrected tactile image, and project the projection points onto the distortion-corrected tactile image and the virtual camera image respectively to obtain the ori coordinates and vc coordinates;

[0021] Step S205: Calculate the homography matrix between the ori coordinates and vc coordinates using the least squares method;

[0022] Step S206: Perform perspective transformation based on the homography matrix to perform spatial consistency calibration.

[0023] Furthermore, the virtual camera is obtained by establishing unified and fixed intrinsic and extrinsic parameters for the internal cameras of the two visual-tactile sensors.

[0024] Furthermore, the spatial consistency calibration establishes data consistency between the two sensors by mapping the tactile images of the two visual-tactile sensors to the same position on a virtual plane using a virtual camera.

[0025] Furthermore, the efficient filtering algorithm filters the tactile image using multiple Gaussian filter kernels of different sizes and dimensions.

[0026] Furthermore, the layered Canny edge detection algorithm extracts edges by using Canny operators with different parameters for the center and edges of the tactile image, thus distinguishing between effective and invalid information regions.

[0027] Furthermore, the Poisson method rapidly solves the Poisson equation through gradient calculation, sine transformation, and inverse sine transformation.

[0028] Secondly, the present invention discloses a tactile feature extraction device based on a parallel symmetric visual-tactile sensor, the device comprising:

[0029] The tactile sensing unit includes a visual tactile sensor and a MEMS pressure sensor mounted in parallel and symmetrically. The tactile sensing of the tactile sensing unit includes tactile characteristic optical flow field, edge field, depth field, contact force field, and high-sensitivity digital pressure measurement.

[0030] The data acquisition unit includes a gripper-type two-finger clamp and a two-finger clamp fingertip. The data acquired by the data acquisition unit includes the raw tactile image output by the visual-tactile sensor and digital pressure readings.

[0031] The data processing unit processes the collected data to obtain four-dimensional tactile features;

[0032] The apparatus is used to implement any of the methods described above.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The tactile feature extraction method used in this invention can realize multi-dimensional perception of tactile features, providing tactile perception information from four dimensions: optical flow, contact edge, depth, and contact force, and can establish a unified and complete tactile feature representation.

[0035] (2) The data processing method used in this invention can perform multi-branch filtering on the tactile image output by the Gelsight visual tactile sensor, and the calculation speed is fast. Through different types of filtering and post-processing methods, it can obtain accurate four-dimensional features of contact optical flow, contact edge, depth and contact force, which meets the needs of fast real-time performance.

[0036] (3) The tactile feature extractor used in this invention is directly applicable to the commercial visual tactile sensor Gelsight through the proposed spatial consistency calibration method, and can be directly transplanted to any other type of visual tactile sensor using this method, thus having a wide range of applications.

[0037] (4) The tactile feature extractor used in this invention integrates three core unit modules: data acquisition unit, tactile perception unit, and data processing unit. It can realize the entire process of tactile perception, data acquisition, and data processing. It solves the problem of super-resolution, high-precision, and multi-dimensional tactile feature extraction in an integrated modular way. It is small in size and can be quickly assembled on the complex three-dimensional surface of the robot and the end connection. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method of the present invention;

[0039] Figure 2 This is a system structure diagram of an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the tactile feature extractor sensing an object in an embodiment of the present invention;

[0041] Figure 4 This is the original tactile map with marked points output by Gelsight in an embodiment of the present invention;

[0042] Figure 5 This is the output image of the left and right Gelsight tactile images after dot detection in an embodiment of the present invention;

[0043] Figure 6 These are the original tactile images, distortion correction images, and cropped, corrected images of the left and right Gelsight in this embodiment of the invention.

[0044] Figure 7 This is a schematic diagram of a virtual plane in an embodiment of the present invention;

[0045] Figure 8This is a diagram showing the horizontal correction effect of the left and right Gelsight after spatial consistency correction in an embodiment of the present invention.

[0046] Figure 9 This is a diagram showing the vertical correction effect of the left and right Gelsight after spatial consistency correction in an embodiment of the present invention.

[0047] Figure 10 This is a flowchart illustrating the calculation of the optical flow field for tactile features in an embodiment of the present invention;

[0048] Figure 11 The tactile feature optical flow field of the left and right Gelsight in the embodiment of the present invention;

[0049] Figure 12 This is a flowchart of the tactile feature edge field calculation in an embodiment of the present invention;

[0050] Figure 13 This refers to the tactile feature edge field of Gelsight in this embodiment of the invention;

[0051] Figure 14 This is a diagram showing the contact force calibration process and the nonlinear cubic fitting curve after calibration, as described in this embodiment of the invention.

[0052] Figure 15 This is a schematic diagram of the linear superposition of four-dimensional tactile features in an embodiment of the present invention.

[0053] Reference numerals: 1. Data processing unit; 2. Two-finger gripping side arm; 3. Two-finger gripping fingertip; 4. Gelsight visual-tactile sensor; 5. Tactile feature extractor; 6. Object under test in the environment; 7. Left Gelsight; 8. Right Gelsight; 9. Virtual plane; 10. Optical flow field; 11. Edge field; 12. Depth field; 13. Contact force field. Detailed Implementation

[0054] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0055] Example

[0056] This embodiment provides a tactile feature extraction method based on the parallel symmetric visual-tactile sensor Gelsight. This method is implemented using a tactile feature extraction device based on the parallel symmetric visual-tactile sensor Gelsight, such as... Figure 2 As shown, the device includes a tactile sensing unit, a data acquisition unit, and a data processing unit.

[0057] Specifically, the tactile sensing unit includes a Gelsight visual tactile sensor and a MEMS pressure sensor mounted in parallel and symmetrically, enabling super-resolution, high-precision, and high-sensitivity tactile sensing. The tactile sensing of the unit includes tactile characteristic optical flow field, edge field, depth field, contact force field, and high-sensitivity digital pressure measurement.

[0058] The Gelsight visual-tactile sensor in this embodiment uses the Gelsight mini, a commercially available visual-tactile sensor developed by MIT. This sensor features super-resolution, real-time performance, and a wide range of applications. Compared to traditional tactile sensors, it can capture minute details on object surfaces, leveraging the advantages of visual information to provide higher resolution and super-resolution tactile information far exceeding that of human fingertips.

[0059] The aforementioned MEMS six-axis accelerometer uses the BOSCH BMI160 six-axis accelerometer, which can accurately measure the acceleration and angular velocity of an object and characterize the vibration information of the object's surface.

[0060] The data acquisition unit includes a two-finger clamping mechanical structure and a two-finger clamping fingertip. The data acquired by the data acquisition unit includes the raw tactile image output by the visual tactile sensor and digital pressure readings.

[0061] The data processing unit processes the collected data to obtain four-dimensional tactile features.

[0062] like Figure 1 As shown, a tactile feature extraction method based on the parallel symmetric visual-tactile sensor Gelsight includes the following steps:

[0063] Step 1: Construct a tactile feature extractor with a parallel symmetric structure based on the Gelsight visual-tactile sensor to obtain a pair of tactile images output by the Gelsight visual-tactile sensor.

[0064] The parallel symmetrical structure is a parallel two-finger gripper-type mechanical structure equipped with the Gelsight visual-tactile sensor, used to collect parallel symmetrical tactile images in real time.

[0065] Step 2: Use a virtual plane and a virtual camera to perform spatial consistency calibration and distortion correction on the pair of tactile images to obtain a pair of tactile images with consistent spatiotemporal correspondence.

[0066] Specifically, the virtual plane is the intersecting plane that is of common interest to two parallel and symmetrically mounted visual-tactile sensors, Gelsight.

[0067] The virtual camera is obtained by establishing unified and fixed intrinsic and extrinsic parameters for the two Gelsight internal cameras, thus achieving rapid unification of the intrinsic and extrinsic parameters of each Gelsight output image.

[0068] A basic schematic diagram of the tactile feature extractor in this embodiment, which senses external objects and extracts tactile sensations, is shown below. Figure 3 The diagram shows a schematic of the tactile feature extractor in detecting an external spherical object.

[0069] In the Gelsight spatial consistency calibration process, the first step is to perform intrinsic parameter calculations and distortion corrections on the individual Gelsight internal cameras. During the intrinsic parameter calculation, the static marker array provided by Gelsight is utilized, such as... Figure 4 As shown, the symmetrical solid circular calibration plate is calibrated using the Zhang Zhengyou calibration method. When using this method for calibration, the marked point region is first detected using an edge extraction algorithm. Then, the roundness, eccentricity, and convexity features of the circular marked points are detected to extract the circular feature contour. Finally, the camera's intrinsic parameter matrix is ​​calculated to obtain the transformation from the camera coordinate system to the pixel coordinate system.

[0070] The roundness C of the marker point represents how close it is to the theoretical circle, and can be expressed by the following formula:

[0071]

[0072] Where A is the area enclosed by the edge contour, and P is the perimeter of the contour. When the roundness C approaches 1, the shape tends to be an ideal circle, which is the feature point that needs to be extracted.

[0073] The eccentricity E of the marked point represents the degree of deviation between the elliptical orbit and the ideal circle. The eccentricity can be obtained by the inertia rate I, as shown in Formula 1.2.

[0074] E 2 +I 2 =1(0.2)

[0075] Where I represents the inertia rate, the closer the eccentricity rate is to 0, the closer it is to the ideal circle.

[0076] The convexity V of the marker point represents the degree of concavity or convexity of the planar convex diagram, and is expressed by the following formula:

[0077]

[0078] Where H is the area of ​​the convex shell enclosed by the edge contour, and S is the area enclosed by the edge contour. The closer the convexity is to 1, the closer it is to the ideal circle.

[0079] The above three metrics were used to detect the markers in Gelsight, and the results are as follows: Figure 5As shown.

[0080] Calculate the intrinsic parameter matrices of the two Gelsights respectively. as follows:

[0081]

[0082]

[0083] In the distortion correction process, the main issues to be addressed are radial and tangential distortion of the camera. Radial distortion can be expressed by the following formula:

[0084]

[0085] Where r represents the radius of curvature, k1, k2, k3 are the radial distortion coefficients, x, y are the coordinates of the image points after radial distortion, and x', y' are the coordinates of the image points after radial distortion is removed.

[0086] Tangential distortion can be expressed by the formula:

[0087]

[0088] Where r represents the radius of curvature, p1 and p2 represent the tangential distortion correction coefficients, x and y are the coordinates of the image points after tangential distortion occurs, and x' and y' are the coordinates of the image points after tangential distortion is removed.

[0089] The lens distortion coefficients are calculated as follows:

[0090]

[0091]

[0092]

[0093]

[0094] The result obtained after distortion correction is as follows: Figure 6 As shown.

[0095] The virtual plane in this embodiment is as follows: Figure 7 As shown, this is the intersecting plane of interest for two parallel and symmetrically mounted visual-tactile sensors, Gelsight.

[0096] The intrinsic parameter M of the virtual camera in this embodiment v and distortion coefficient k v Both are fixed, consisting of two Gelsight intrinsic parameters M. i and distortion coefficient k iThe average value is calculated. During preprocessing, the homography matrix H is calculated based on the intrinsic parameters of the two current Gelsight sensors, as well as the distortion coefficients and the intrinsic parameters and distortion coefficients of the virtual camera. i,v By selecting 63 corresponding marker points p on Gelsight k =(x k ,y k ,0) T k = 0, 1, 2, ..., 63 are used as projection points, and then they are projected onto the current Gelsight and the virtual camera respectively to obtain... and Finally, the homography matrix H is obtained using the least squares method. i,v As shown in Formula 1.6:

[0097]

[0098] The two calculated H matrices are shown below:

[0099]

[0100]

[0101] The results after spatial consistency calibration are as follows: Figure 8 , Figure 9 As shown, the horizontal and vertical spatial errors of the two Gelsight output images are significantly corrected after calibration.

[0102] Step 3: Extract the tactile feature optical flow field from the corrected tactile image using a high-efficiency filtering algorithm based on Gaussian blur. This high-efficiency filtering algorithm filters the tactile image using multiple Gaussian filter kernels of different sizes and dimensions, effectively filtering out interference information. The tactile feature optical flow field is the flow formed by the displacement changes of the marker points in the Gelsight tactile image as the contact surface is subjected to force or pressure, including the direction and distance of movement of the marker points.

[0103] Specifically, the optical flow field calculation process in this embodiment is as follows: Figure 10As shown, a high-efficiency filtering algorithm based on Gaussian blur is used to extract the optical flow field of tactile features. The corrected Gelsight image is sequentially input into Gaussian filters of size and . The pixel values ​​of the output image are limited to between 140 and 255 based on a threshold derived from empirical parameters for initial dot filtering. Then, a Gaussian kernel of size and a standard deviation of 3 is created as a template for marker points. Normalized cross-correlation is used to find positions in the output image that match the created template. Finally, values ​​less than zero in the result are set to zero to eliminate machine precision errors, and the final mask image is returned. The output image dynamically tracks the mask image for each frame, marking the displacement of the marker points in the mask to obtain the optical flow field. The results are shown in the figure. Figure 11 As shown.

[0104] Step 4: Extract the tactile feature edge field of the target object using a layered Canny edge detection algorithm and opening / closing operations on the corrected tactile image. The layered Canny edge detection algorithm extracts edges by applying Canny operators with different parameters to the center and edges of the tactile image, distinguishing between effective and ineffective information regions.

[0105] Specifically, the edge field calculation process in this embodiment is as follows: Figure 12 As shown, the layered Canny edge detection algorithm and opening / closing operations are used to extract the tactile feature edge field of the target object from the corrected tactile image. The corrected Gelsight image is smoothed and filtered, and the red channel of the filtered image is extracted as the new target image. The layered Canny edge detection algorithm is then used to extract the edges of the target image, followed by multiple opening / closing operations, consisting of one opening operation and one closing operation. The processed image is then input into a mean filter to calculate the contact area and contact edge. The results are shown below. Figure 13 As shown.

[0106] Step 5: Reconstruct the tactile feature depth field of the corrected tactile image using gradient differencing and the Poisson method. The Poisson method rapidly solves the Poisson equation through gradient calculation, sine transformation, and inverse sine transformation.

[0107] Specifically, the depth field calculation in this embodiment is mainly achieved through gradient difference. First, the absolute difference between the images obtained in contact and non-contact situations is calculated. Then, this difference image is used to calculate the gradient images in the x and y directions, as shown in the formula. Finally, the depth field of the contact surface is reconstructed using the Poisson method.

[0108]

[0109]

[0110] Where x and y are the horizontal and vertical coordinates in the image, I(x,y) represents the grayscale value of the image, and I... x (x,y) and I y (x, y) represent the gradients of the image in the x and y directions, respectively.

[0111] Step 6: Use BMI160 to calibrate the correspondence between Gelsight contact surface pressure and tactile image; based on the correspondence, establish a nonlinear curve of contact surface pressure for the tactile feature depth field, and reconstruct the tactile feature contact force field. The tactile feature contact force field is obtained based on the contact surface pressure curve and reflects the actual contact force magnitude corresponding to the pixels of the Gelsight tactile image.

[0112] Specifically, in this embodiment, the contact force field calculation establishes the relationship between contact depth and pressure by calibrating the contact force using tactile images. Under a pressure testing instrument, the contact depth and the readings of the BMI160 pressure sensor are recorded, and the recorded data are shown in Table 1.

[0113] Table 1. Relationship between contact depth and contact force

[0114]

[0115] Cubic curve fitting was used on the calibration data to obtain the nonlinear relationship between contact depth and contact force, such as... Figure 14 As shown, this enables the creation of a contact force field based on the depth field.

[0116] Step 7: Linearly superimpose the tactile feature optical flow field, edge field, depth field, and contact force field to construct a unified and complete tactile feature, such as... Figure 15 As shown.

[0117] By adopting the tactile feature extraction method based on the parallel symmetric visual-tactile sensor Gelsight provided in this embodiment, the response speed can be improved, and a unified representation of super-resolution, high-precision tactile features and efficient extraction of tactile features can be achieved.

[0118] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for extracting tactile features based on a parallel symmetric visual-tactile sensor, characterized in that, Includes the following steps: Step S1: Construct a tactile feature extractor with a parallel symmetric structure based on the visual-tactile sensor to obtain a pair of tactile images output by the visual-tactile sensor; Step S2: Use a virtual plane and a virtual camera to perform distortion correction and spatial consistency calibration on the pair of tactile images to obtain the corrected tactile images; Step S3: Use an efficient filtering algorithm to extract features from the corrected tactile image to obtain the tactile feature optical flow field; Step S4: Use the layered Canny edge detection algorithm and opening / closing operations to extract features from the corrected tactile image to obtain the tactile feature edge field; Step S5: Reconstruct the corrected tactile image using the Poisson method and gradient difference to obtain the tactile feature depth field; Step S6: Establish the tactile feature contact force field based on the nonlinear correspondence curve between the contact surface pressure and the tactile feature depth field; Step S7: The tactile feature extractor linearly superimposes the tactile feature optical flow field, edge field, depth field and contact force field to construct a four-dimensional tactile feature.

2. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 1, characterized in that, The parallel symmetrical structure is a parallel two-finger gripper structure equipped with two visual-tactile sensors.

3. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 1, characterized in that, The virtual plane is the intersecting plane that is of common interest to two parallel and symmetrically mounted visual-tactile sensors.

4. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 1, characterized in that, The distortion correction and spatial consistency calibration process includes the following steps: Step S201: Obtain the original tactile images output by two parallel and symmetrical visual-tactile sensors; Step S202: Select the virtual plane and calculate the intrinsic and extrinsic parameters of the virtual camera; Step S203: Perform distortion correction on the original tactile image based on the intrinsic and extrinsic parameters of the virtual camera; Step S204: Select multiple corresponding marker points as projection points on the distortion-corrected tactile image, and project the projection points onto the distortion-corrected tactile image and the virtual camera image respectively to obtain the ori coordinates and vc coordinates; Step S205: Calculate the homography matrix between the ori coordinates and vc coordinates using the least squares method; Step S206: Perform perspective transformation based on the homography matrix to perform spatial consistency calibration.

5. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 1, characterized in that, The virtual camera is obtained by establishing unified and fixed intrinsic and extrinsic parameters for the internal cameras of the two visual-tactile sensors.

6. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 5, characterized in that, The spatial consistency calibration establishes data consistency between the two sensors by mapping the tactile images of the two visual-tactile sensors to the same position on a virtual plane using a virtual camera.

7. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 1, characterized in that, The efficient filtering algorithm filters the tactile image using multiple Gaussian filter kernels of different sizes and dimensions.

8. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 1, characterized in that, The layered Canny edge detection algorithm extracts edges by using Canny operators with different parameters for the center and edges of the tactile image, distinguishing between effective and invalid information regions.

9. The tactile feature extraction method based on a parallel symmetric visual-tactile sensor according to claim 1, characterized in that, The Poisson method is used to quickly solve the Poisson equation through gradient calculation, sine transformation, and inverse sine transformation.

10. A tactile feature extraction device based on a parallel symmetric visual-tactile sensor, characterized in that, include: The tactile sensing unit includes a visual tactile sensor and a MEMS pressure sensor mounted in parallel and symmetrically. The tactile sensing of the tactile sensing unit includes tactile characteristic optical flow field, edge field, depth field, contact force field, and high-sensitivity digital pressure measurement. The data acquisition unit includes a gripper-type two-finger clamp and a two-finger clamp fingertip. The data acquired by the data acquisition unit includes the raw tactile image output by the visual-tactile sensor and digital pressure readings. The data processing unit processes the collected data to obtain four-dimensional tactile features; The apparatus is used to implement the method as described in any one of claims 1-9.

Images