A three-dimensional contact force sensing device and method for fingertips that retain the sense of touch

Abstract

A three-dimensional contact force sensing device and method for fingertips that retains tactile sensation, the device has a symmetrical structure, including a base, an adaptive pad installed on the upper part of the base inner cavity, side plates connected to both sides of the base, and radial thin film sensors installed on the inner sides of the side plates; sensor grooves are symmetrically provided on the lower part of the base, axial thin film sensors are installed in the sensor grooves, and the axial thin film sensors correspond to contact heads symmetrically installed at the front part of the base inner cavity; the method is to apply pressure to a three-axis pressure sensing device in different directions after wearing the device, collect four-way thin film sensor signals and three-way three-axis pressure sensor signals; then send them into a multivariate regression learning system, and establish a fingertip contact force estimation model through the exponential GPR; finally, the human finger wearing the device applies pressure to the object, and the four thin film sensor signals are sent to the fingertip contact force estimation model to obtain an estimated three-dimensional contact force; the present invention has the characteristics of being lightweight, fitting the human body, and taking a short calibration time.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a fingertip three-dimensional contact force sensing device and method that retains tactile sensation. Background Art

[0002] With the continuous development of mechatronic interaction, intelligent control, and robotics, advanced contact force feedback and control technologies are being increasingly incorporated into the human-robot interaction between rehabilitation equipment and patients. In hand function rehabilitation training involving complex grasping movements, measuring the contact force between fingertips and grasped objects provides crucial feedback for improving the compliance of human-robot interaction and coordinating human-robot motion.

[0003] However, existing methods for measuring fingertip contact force mostly rely on force sensors located on the wearer's fingertips. This can affect fingertip tactile sensation, interfere with fingertip movement, and be incapable of measuring contact forces in different directions, making it difficult to meet the needs of fine-tuned hand rehabilitation training. On the one hand, human grip force control relies heavily on touch, so tactile impairments caused by sensors can lead to reduced dexterity. Specifically, a weakened sense of touch can cause excessive grip force, resulting in damage to objects, or insufficient grip force, leading to slippage and even secondary hand injury. On the other hand, in hand rehabilitation training, tactile stimulation of the cerebral cortex can promote the central nervous system's ability to restore perception and control of the limbs, thereby gradually activating or reconstructing neural conduction pathways, achieving "motor cognitive relearning," and improving rehabilitation outcomes.

[0004] In recent years, fingertip contact force estimation methods based on fingertip deformation have attracted the attention of researchers. In 2019, Junghoon Park et al. (Park J, Heo P, Kim J, et al. A Finger Grip Force Sensor with an Open-Pad Structure for Glove-Type Assistive Devices [J]. Sensors, 2019, 20(1).) from South Korea attempted to use customized capacitive sensors for glove-type assistive devices with open finger pad structures. They used a quasi-exponential function to selectively calibrate the sensor. This method had a small error (0.843 N) when the contact force was below 5 N. However, this method can only measure the magnitude of the contact force in a single direction and cannot provide high-quality three-dimensional force feedback signals for rehabilitation training of fine hand movements. In 2019, Ayane Saito et al. (Saito A, Kuno W, Kawai W, et al. Estimation of Fingertip Contact Force by Measuring Skin Deformation and Posture with Photo-reflective Sensors[C] / / the 10th Augmented Human International Conference 2019.2019.) from Japan estimated the three-dimensional contact force of the fingertip by using a light reflective sensor to measure the distance between the side of the fingertip and the sensor. The average contact force estimation errors in the three directions reached 0.661N, 1.002N, and 1.071N, respectively. This method is easily affected by ambient light and the estimation accuracy is not high. Summary of the Invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a three-dimensional contact force sensing device and method for fingertips that retains tactile sensation, which can provide reliable feedback signals for the grip force control of hand function rehabilitation robots, and has the characteristics of being lightweight, fits the human body, has a short calibration time, and has good rehabilitation effects.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is:

[0007] A three-dimensional contact force sensing device for fingertips that retains tactile sensation has a symmetrical structure and includes a base 1. An adaptive pad 2 is installed on the upper part of the inner cavity of the base 1, and the adaptive pad 2 contacts the upper surface of a human finger 7; side plates 3 are connected to both sides of the base 1, and radial film sensors 4 are installed on the inner sides of the side plates 3; sensor grooves 10 are symmetrically provided on the lower part of the base 1, and axial film sensors 6 are installed in the sensor grooves 10. The axial film sensors 6 correspond to the contact heads 5 symmetrically installed on the front part of the inner cavity of the base 1.

[0008] The adaptive pad 2 is individually designed according to the size and curved surface characteristics of the human finger 7 .

[0009] By changing the thickness of the gasket between the side plate 3 and the base 1, the preload force between the human finger 7 and the radial film sensor 4 can be adjusted to meet the usage requirements of different patients.

[0010] The width of the sensor slot 10 is the same as the thickness of the axial thin film sensor 6 .

[0011] The radial thin film sensor 4 and the axial thin film sensor 6 are FSR400 thin film pressure sensors, whose resistance value decreases as the pressure in the sensing area increases. The sensing area has a diameter of 5 mm, a thickness of 0.3 mm, and a sensitivity range of 0.2N-20N.

[0012] The protruding end of the contact head 5 contacts the front end of the human finger 7. The model of the contact head 5 can be replaced according to different usage requirements. The protruding end of the contact head 5 adopts a spherical head design.

[0013] The radial film sensor 4 is adhered to the inner side of the side plate 3 , and the inner side of the side plate 3 is configured as a concave surface to ensure that the radial film sensor 4 is always in contact with the human finger 7 .

[0014] The fingertip three-dimensional contact force sensing device with tactile retention is worn on the upper part of a human finger 7, with the fingertip portion open.

[0015] The base 1, the adaptable pad 2, the side panel 3 and the contact head 5 are all manufactured by 3D printing technology; the base 1, the side panel 3 and the contact head 5 are made of 8200p resin material, and the adaptable pad 2 is made of soft rubber material.

[0016] A method for utilizing a fingertip three-dimensional contact force sensing device that preserves tactile sensation comprises the following steps:

[0017] The first step is force signal acquisition: a tactile 3D contact force sensing device is worn on a human finger 7. Pressure is applied to a 3D pressure sensing device along different directions of the finger 7. Seven pressure signals, including four thin film sensor signals and three 3D pressure sensor signals, are collected.

[0018] The second step is to establish a fingertip contact force estimation model: the seven pressure signals are fed into a multivariate regression learning system, and five cross-validations are set to avoid overfitting. The fingertip contact force estimation model is established using the exponential GPR.

[0019] The third step is three-dimensional contact force estimation: the human finger 7 wears a three-dimensional contact force sensing device on the fingertip that retains tactile sensation, applies any amount of pressure to the object in any direction, collects four pressure signals from four thin film sensors, and sends them to the fingertip contact force estimation model established in the second step. The estimated three-dimensional contact force is calculated through the exponential GPR, which serves as the feedback signal provided by the hand function rehabilitation robot for grip force control.

[0020] The beneficial effects of the present invention are:

[0021] The device of the present invention opens the fingertip part of the human finger 7, retains the tactile sensation of the fingertip when the fingertip contacts an object, and measures the axial and radial interaction forces of the fingertip of the human finger 7 through four thin film sensors respectively, and sends them to the multivariate regression learning system to obtain a fingertip contact force estimation model, thereby realizing high-precision estimation of the three-dimensional contact force of the fingertip.

[0022] The device of the present invention has a compact structure and lightweight materials, and is suitable for being worn on the fingertips of human fingers; the base 1 and the contact head 5 can be personalized according to the shape and size of human fingers, thereby enhancing the adaptability of the product and improving the rehabilitation effect of the product.

[0023] The method of the present invention adopts a Gaussian process regression model with an exponential kernel for multivariate regression learning. The exponential GPR ensures the boundedness of the prediction error with a certain probability. The mean errors of the contact force estimation in the three directions are 0.4360N, 0.3698N and 0.4654N, respectively, which is 47% lower than the mean error in the study by Ayane Saito et al. using a light reflection sensor for estimation. Therefore, it can be applied to the fingertip contact force feedback scenario of rehabilitation robots with higher safety requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the overall structure of the device of the present invention.

[0025] Figure 2 Schematic diagram of the finger wearing of the device of the present invention.

[0026] Figure 3 It is an exploded view of the device of the present invention.

[0027] Figure 4 It is a structural schematic diagram of the base of the device of the present invention.

[0028] Figure 5 It is a structural schematic diagram of the contact head of the device of the present invention.

[0029] Figure 6 It is a structural schematic diagram of the side panel of the device of the present invention.

[0030] Figure 7 It is a flow chart of the method of the present invention.

[0031] Figure 8 Schematic diagram of the finger coordinate system of the method of the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0033] Reference Figure 1-Figure 2 A three-dimensional contact force sensing device for fingertips that retains tactile sensation has a symmetrical structure and includes a base 1, an adaptation pad 2, a side plate 3, a radial film sensor 4, a contact head 5, and an axial film sensor 6;

[0034] An adaptive pad 2 is installed on the upper part of the inner cavity of the base 1. The adaptive pad 2 contacts the upper surface of a human finger 7. The human finger 7 is scanned by a Revopoint handheld 3D scanner to obtain the size and surface characteristics of the human finger 7. The adaptive pad 2 is personalized according to the size and surface characteristics of the human finger 7.

[0035] Reference Figure 3 , mounting holes for side panels 3 are respectively provided on both sides of the base 1, and the side panels 3 are fixed to the base 1 by a pair of screws, namely a first screw 8 and a second screw 9. A radial film sensor 4 is installed on the inner side of the side panel 3; by changing the thickness of the gasket between the side panel 3 and the base 1, the preload force under the contact between the human finger 7 and the radial film sensor 4 can be adjusted to meet the usage needs of different patients.

[0036] Reference Figure 3 、 Figure 4 A sensor groove 10 is symmetrically provided at the lower part of the base 1, in which an axial film sensor 6 is installed. The width of the sensor groove 10 is the same as the thickness of the axial film sensor 6; the axial film sensor 6 and the contact head 5 symmetrically installed at the front part of the inner cavity of the base 1 correspond to each other.

[0037] The radial film sensor 4 and the axial film sensor 6 are FSR400 film pressure sensors. The resistance value decreases as the pressure in the sensing area increases. They are often used in medical equipment to detect the degree of pressure on the human body and can sense tiny pressure and tactile signals. The sensing area has a diameter of 5mm and a thickness of 0.3mm. They are light, thin and bend-resistant, with a sensitivity range of 0.2N-20N.

[0038] Reference Figure 5The protruding end of the contact head 5 contacts the front end of the human finger 7. The model of the contact head 5 is replaced according to different usage requirements, thereby changing the size of the mechanism to meet the usage needs of different patients; the protruding end of the contact head 5 adopts a spherical head design to increase wearing comfort.

[0039] Reference Figure 1 、 Figure 6 The radial film sensor 4 is pasted on the inner side of the side plate 3, and the inner side of the side plate 3 is set to a concave surface to ensure that the radial film sensor 4 is always in contact with the human finger 7.

[0040] The fingertip three-dimensional contact force sensing device with tactile sense preservation is worn on the upper part of a human finger 7, with the fingertip portion open, to ensure the tactile sense of the fingertip.

[0041] The base 1, adaptive pad 2, side panel 3 and contact head 5 are all manufactured through 3D printing technology; the base 1, side panel 3 and contact head 5 are made of 8200p resin material to ensure the toughness and lightness of the structure; the adaptive pad 2 is made of soft rubber material to ensure wearing comfort.

[0042] Reference Figure 7-Figure 8 A tactile-preserving three-dimensional fingertip contact force sensing device is used. Four thin-film sensor signals and three-axis pressure sensor signals are simultaneously collected and processed with low-pass filtering, zero-bias removal, and voltage-to-force conversion. The seven signals are then fed into a multivariate regression learning system to establish a fingertip contact force estimation model and estimate the three-dimensional fingertip contact force. The specific steps are as follows:

[0043] The first step is force signal acquisition: a three-dimensional contact force sensing device with tactile retention is worn on the fingertip of the human finger 7. The device applies arbitrary pressure within the range of the LZ-SWF46 three-axis pressure sensor, which has a range of 10N in the X-axis direction, 10N in the Y-axis direction, and 20N in the Z-axis direction, in six directions, namely +X, -X, +Y, -Y, +Z, and the +X+Y bisecting line of the human finger 7. The sensor is pressed ten times in each direction, with each press lasting 5 seconds. At the same time, the pressure signal acquisition device collects seven pressure signals from the four thin film sensors and the LZ-SWF46 three-axis pressure sensor.

[0044] The second step is to establish a fingertip contact force estimation model: the seven pressure signals are collected and transmitted through the Smacq3020 data acquisition card and sent to the multivariate regression learning system. Five cross-validations are set to avoid overfitting, and the fingertip contact force estimation model is established through the exponential GPR (Gaussian process regression);

[0045] The third step is three-dimensional contact force estimation: the human finger 7 wears a three-dimensional contact force sensing device on the fingertip that retains tactile sensation, and applies pressure to the object in any direction. At the same time, the pressure signal acquisition device collects four pressure signals from four thin film sensors and sends the four pressure signals to the fingertip contact force estimation model established in the second step. The estimated three-dimensional contact force is calculated through the exponential GPR (Gaussian process regression), which then provides a reliable feedback signal as the grasping force control of the hand function rehabilitation robot.

[0046] GPR (Gaussian Process Regression) is a non-parametric modeling method suitable for modeling nonlinear systems. The complexity of the model can be adjusted according to the training data. Assume that the data set {X, y} consists of n pairs of data, where the input set is X and the output set is y. For a query point x, its corresponding output f(x) can be expressed as The prior distribution of the output set y and the joint distribution of y and f(x) can be expressed as:

[0047] where K(X,X) is the n×n dimensional GP covariance matrix, consisting of the elements K(x i ,x j ), i, j = 1, …, n; K(X, x) is the covariance vector composed of the covariances of all elements in the input set X and the query point. Similarly, K(x, X) and K(x, x) can be obtained; and the posterior of f(x) also follows a Gaussian distribution: There are many choices for GP kernel functions, such as linear kernel function, polynomial kernel function, exponential kernel function, quadratic rational kernel function, etc.

[0048] Exponential GPR is a Gaussian process regression model using an exponential kernel. Since exponential GPR ensures the boundedness of the prediction error with a certain probability, it can be applied to fingertip contact force feedback scenarios of rehabilitation robots with high safety requirements.

[0049] The above embodiments are only used to illustrate the present invention, rather than to limit the present invention. Any equivalent transformations or modifications made by those skilled in the relevant technical field based on the essence of the present invention should be included in the scope of protection of the present invention.

Claims

1. A three-dimensional contact force sensing device for fingertips that retains tactile sensation, having a symmetrical structure, comprising a base (1), characterized in that: An adaptable pad (2) is installed on the upper part of the inner cavity of the base (1), and the adaptable pad (2) contacts the upper surface of the human finger (7); side plates (3) are connected to both sides of the base (1), and radial film sensors (4) are installed on the inner sides of the side plates (3); sensor grooves (10) are symmetrically provided on the lower part of the base (1), and axial film sensors (6) are installed in the sensor grooves (10), and the axial film sensors (6) and contact heads (5) symmetrically installed on the front part of the inner cavity of the base (1) are correspondingly matched; The fingertip three-dimensional contact force sensing device with tactile retention is worn on the upper part of a human finger (7), with the fingertip portion open; The device is a fingertip three-dimensional contact force sensing method, The following steps are involved: The first step is force signal acquisition: a human finger (7) is equipped with a three-dimensional contact force sensing device for retaining tactile sensation, and pressure is applied to a three-axis pressure sensing device in different directions along the human finger (7), and seven pressure signals, including four-channel thin film sensor signals and three-channel three-axis pressure sensor signals, are acquired from the three-dimensional contact force sensing device for retaining tactile sensation; The second step is to establish a fingertip contact force estimation model: the seven pressure signals are fed into a multivariate regression learning system, and five cross-validations are set to avoid overfitting. The fingertip contact force estimation model is established using a Gaussian process regression model with an exponential kernel. The third step is three-dimensional contact force estimation: the human finger (7) is equipped with a three-dimensional contact force sensing device that retains tactile sensation. It applies any amount of pressure to the object in any direction. The four pressure signals from the four thin film sensors are collected and sent to the fingertip contact force estimation model established in the second step. The estimated three-dimensional contact force is calculated by the exponential GPR and used as the feedback signal for the grip force control of the hand function rehabilitation robot.

2. The device according to claim 1, characterized in that: The adaptive pad (2) is individually designed according to the size and curved surface characteristics of the human finger (7).

3. The device according to claim 1, characterized in that: By changing the thickness of the gasket between the side plate (3) and the base (1), the preload force under contact between the human finger (7) and the radial film sensor (4) is adjusted to meet the use requirements of different patients.

4. The device according to claim 1, characterized in that: The width of the sensor slot (10) is the same as the thickness of the axial thin film sensor (6).

5. The device according to claim 1, characterized in that: The radial thin film sensor (4) and the axial thin film sensor (6) are FSR400 thin film pressure sensors, whose resistance value decreases as the pressure in the sensing area increases. The sensing area has a diameter of 5 mm, a thickness of 0.3 mm, and a sensitivity range of 0.2N-20N.

6. The device according to claim 1, characterized in that: The protruding end of the contact head (5) contacts the front end of the human finger (7), and the model of the contact head (5) is replaced according to different usage requirements; the protruding end of the contact head (5) adopts a spherical head design.

7. The device according to claim 1, characterized in that: The radial film sensor (4) is adhered to the inner side of the side plate (3), and the inner side of the side plate (3) is configured as a concave surface to ensure that the radial film sensor (4) is always in contact with the human finger (7).

8. The device according to claim 1, characterized in that: The base (1), the adaptable pad (2), the side plate (3) and the contact head (5) are all manufactured by 3D printing technology; the base (1), the side plate (3) and the contact head (5) are made of 8200p resin material, and the adaptable pad (2) is made of soft rubber material.

Images