Multi-mode three-dimensional tactile sensing device, method and system based on light reflection and resistance

Abstract

The invention discloses a multi-mode three-dimensional tactile sensing device, method and system based on light reflection and resistance, and the device comprises a hemispherical flexible layer, four silicone rubber columns, a thermistor layer, a piezoresistive film layer composed of four piezoresistive sensing units, a light reflection film layer, an optical fiber sleeve, a transmitting optical fiber, and a receiving optical fiber. The top ends of the silicone rubber columns are connected to the lower portion of the hemispherical flexible layer and evenly distributed in four quadrant areas in the hemispherical flexible layer, the thermistor layer is located below the silicone rubber columns, and all the piezoresistive sensing units of the piezoresistive film layer are arranged below the thermistor layer and correspond to the silicone rubber columns in a one-to-one mode. The light reflecting film layer is located below the piezoresistive film layer, the optical fiber sleeve is arranged below the light reflecting film layer, and terminals of the transmitting optical fiber and the receiving optical fiber are coaxially arranged in the optical fiber sleeve at intervals, so that an optical fiber bundle is formed. The system is low in cost, wide in application scene and capable of achieving three-dimensional force and multi-mode sensing.

Description

Technical Field

[0001] The present invention relates to tactile sensing technology, and in particular, to a multimodal three-dimensional tactile sensing device, method and system based on light reflection and resistive type. Background Art

[0002] Tactile perception is a basic ability of the human skin, which is an embodiment of the response to external stimuli and is crucial in human-computer interaction. It mainly realizes by evaluating the intensity, distribution and pattern of the stimulation events applied to the skin. For example, the human skin can feel information such as the temperature, humidity, material type, surface texture, hardness, vibration of an object through touch, but specific parameter values cannot be obtained quantitatively. The tactile sensing device detects other information such as pressure, strain, temperature, vibration, etc. by imitating the function of the human skin, converts the external stimulus into an electrical signal, and then quantitatively obtains specific parameter values through signal processing. In recent years, with the rapid development of flexible electronics, flexible tactile sensing technology has been widely applied in the fields of intelligent robots, human-computer interaction and medical health monitoring. The tactile sensor based on the principle of light reflection converts pressure into light intensity change, thereby realizing pressure measurement. The resistive type is a very typical and commonly used electrical tactile sensing principle, mainly by measuring the change of the resistance of the sensing element to obtain the external stimulus, and can measure physical quantities such as temperature, pressure, deformation, etc. The thermistor is also a kind of resistive sensing principle, mainly by converting temperature into resistance change to realize temperature measurement, which is a very simple temperature measurement device. In order to more accurately imitate the ability of the human skin, the tactile sensing device that can only measure one physical quantity no longer meets the needs, so a flexible tactile sensing device that can measure multiple physical quantities is required.

[0003] In summary, the current difficulties of tactile sensors are as follows: First, the sensitivity and response speed of flexible tactile sensors are usually limited by the viscoelasticity of the elastomeric substrate, while rigid tactile sensors will limit the usage scenarios and surface structures; Second, the current tactile sensors still cannot meet the simultaneous realization of multimodality and three-dimensional forces; Third, the manufacturing of tactile sensing devices is complex and the cost is relatively high. Summary of the Invention

[0004] Aiming at the problems existing in the prior art, the purpose of the present invention is to provide a multimodal three-dimensional tactile sensing device, method and system based on light reflection and resistive type with low cost and wide usage scenarios.

[0005] In order to achieve the above invention purpose, the present invention provides the following technical solutions:

[0006] A multi-modal three-dimensional tactile sensing device based on light reflection and resistance type, comprising a hemispherical flexible layer, four silicone rubber columns, a thermistor layer, a piezoresistive film layer composed of four piezoresistive sensing units, a light reflection film layer, an optical fiber sleeve, a transmitting optical fiber, and a receiving optical fiber. The top ends of the silicone rubber columns are connected below the hemispherical flexible layer and are evenly distributed in four quadrant regions within the hemispherical flexible layer. The thermistor layer is located below the silicone rubber columns. Each piezoresistive sensing unit of the piezoresistive film layer is disposed below the thermistor layer and corresponds to the position of the silicone rubber column one by one. The light reflection film layer is located below the piezoresistive film layer. The optical fiber sleeve is disposed below the light reflection film layer. The terminals of the transmitting optical fiber and the receiving optical fiber are arranged coaxially and at intervals within the optical fiber sleeve, thereby forming an optical fiber bundle. The central axes of the hemispherical flexible layer, the thermistor layer, the piezoresistive film layer, the optical fiber bundle, the light reflection film layer, and the optical fiber sleeve are located on the same vertical line.

[0007] Further, the centers of the end faces of all the silicone rubber columns are equidistant from the center of the sphere of the hemispherical flexible layer.

[0008] Further, the thermistor layer includes a flexible base layer, electrodes, and a protective layer. The electrodes are uniformly laid on the flexible base layer in an arch-shaped path, and the protective layer covers the electrodes.

[0009] Further, the piezoresistive sensing unit is specifically a 1 / 4 circular sector and is symmetrically distributed around the center.

[0010] Further, the light reflection film layer includes a flexible sub-layer and a silver powder sub-layer.

[0011] A multi-modal three-dimensional tactile sensing method based on light reflection and resistance type, which is based on the above multi-modal three-dimensional tactile sensing device, specifically includes the following steps:

[0012] Measure the resistance change amounts of the four piezoresistive sensing units of the piezoresistive film layer, and based on the resistance change amounts, calculate the magnitude of the three-dimensional force applied to the hemispherical flexible layer according to the pre-established relationship between the magnitude of the applied force and the resistance change amounts of the four piezoresistive sensing units. Then, judge the direction of the applied three-dimensional force according to whether the resistance of the piezoresistive sensing unit increases or decreases, and output the magnitude and direction of the applied three-dimensional force as the sensed three-dimensional force.

[0013] Measure the real-time resistance of the thermistor layer, and query the pre-established relationship between the resistance of the thermistor layer and the temperature according to the real-time resistance to obtain the real-time temperature of the thermistor layer, which is output as the sensed temperature.

[0014] Emit laser light to the transmitting optical fiber and record the light spots of the reflected laser light in the receiving optical fiber at different times. Among them, the reflected laser light in the receiving optical fiber is the laser light in the transmitting optical fiber reflected back by the light reflection film.

[0015] Based on the light spots at different recorded times, a curve of the change in the gray value of the light spot with respect to time is obtained, and by performing discrete wavelet transform on the curve, the vibration frequency is obtained and output as the sensing vibration frequency.

[0016] Further, the relationship between the magnitude of the applied force and the resistance change of the four piezoresistive sensing units established in advance is specifically obtained by the following method:

[0017] Apply several three-dimensional forces with different magnitudes to the hemispherical flexible layer, and measure the resistance of each piezoresistive sensing unit when applying three-dimensional forces with different magnitudes.

[0018] According to the magnitude of the applied three-dimensional force and the corresponding resistance change of each piezoresistive sensing unit, the following formula is obtained by fitting:

[0019] F1 = f1(ΔR1, ΔR2, ΔR3, ΔR4)

[0020] F2 = f2(ΔR1, ΔR2, ΔR3, ΔR4)

[0021] F3 = f3(ΔR1, ΔR2, ΔR3, ΔR4)

[0022] ΔR1 = R1 - R 10 、ΔR2 = R2 - R 20 、ΔR3 = R3 - R 30 、ΔR4 = R4 - R 40

[0023] In the formula, F1, F2, and F3 respectively represent the force components of the three-dimensional force in the x, y, and z directions, f1, f2, and f3 respectively represent the relationships between F1, F2, and F3 and the resistance change, ΔR1, ΔR2, ΔR3, and ΔR4 respectively represent the resistance changes of the four piezoresistive sensing units, and R 10 、R 20 、R 30 、R 40 represents the resistance of the four piezoresistive sensing units when no three-dimensional force is applied.

[0024] Further, the relationship between the resistance of the thermistor layer and the temperature established in advance is specifically:

[0025]

[0026] In the formula, R(T) represents the resistance at temperature T, ρ0 is the resistivity at the reference temperature, α represents the temperature coefficient, which can be obtained through a temperature calibration experiment, T0 represents the reference temperature, L represents the length of the upper electrode of the thermistor layer, and A is the cross-sectional area of the electrode.

[0027] Further, obtaining a curve of the change in the spot gray value varying with time based on the spots at different recorded times specifically includes:

[0028] Solving for the gray values of the spots at different times based on the spots at different recorded times:

[0029] G = ∫ S g(x, y)dS

[0030] In the formula, G represents the gray value of the spot, g(x, y) is the gray value distribution of the spot, and S is the area of the entire spot;

[0031] Subtracting the gray value of the reference spot from the gray values of the spots at different times to obtain the change in the spot gray value:

[0032] ΔG i = G i - G0

[0033] In the formula, G i is the gray value of the spot recorded at the i-th time, and G0 is the gray value of the reference spot when no external force is applied;

[0034] Obtaining a curve of the change in the spot gray value varying with time based on the change in the spot gray value and the corresponding time.

[0035] A multi-modal three-dimensional tactile sensing system based on optical reflection and resistive includes the above multi-modal three-dimensional tactile sensing device, and further includes:

[0036] A resistance measuring instrument, which is respectively connected to the thermistor layer and the piezoresistive film layer, and is used to measure the real-time resistance of the thermistor layer, query the relationship between the resistance and temperature of the thermistor layer established in advance according to the real-time resistance, obtain the real-time temperature of the thermistor layer, and output it as the sensing temperature; and measure the change in the resistance of the four piezoresistive sensing units of the piezoresistive film layer, and calculate the magnitude of the three-dimensional force applied to the hemispherical flexible layer based on the relationship between the magnitude of the applied force and the change in the resistance of the four piezoresistive sensing units established in advance, and then judge the direction of the applied three-dimensional force according to whether the resistance of the piezoresistive sensing unit increases or decreases, and output the magnitude and direction of the applied three-dimensional force as the sensing three-dimensional force;

[0037] An optical fiber light source, which is used to emit laser light to the transmitting optical fiber;

[0038] A camera, which is used to record the spots of the reflected laser light in the receiving optical fiber at different times, where the reflected laser light in the receiving optical fiber is the laser light reflected back by the laser light in the transmitting optical fiber through the optical reflection film;

[0039] A calculation module, configured to obtain a curve of the change in the spot gray value over time based on the spots at different times recorded, and perform discrete wavelet transform on the curve to obtain the vibration frequency, which is output as the sensed vibration frequency;

[0040] A base, configured to place the fiber optic light source, the camera, the resistance measuring instrument, and the calculation module on the base.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. By combining the hemispherical flexible layer of the silicone rubber column and the piezoresistive film layer, the direction of the applied force is obtained through the change direction of the resistance of the four piezoresistive sensing units, and the magnitude of the force value is obtained through the change amount of the resistance, realizing three-dimensional force measurement;

[0043] 2. A thermistor layer is adopted, and temperature measurement is realized by establishing a curve between temperature and resistance;

[0044] 3. Based on the principle of the light reflection film, when the contact target comes into contact with the hemispherical flexible layer, the deformation is transmitted to the light reflection film layer through the silicone rubber column. Due to the local deformation caused by the force, the direction of the reflected light is changed, which macroscopically shows a change in the intensity of the reflected light. The vibration signal can be detected through the light intensity;

[0045] 4. Compared with the traditional tactile sensing device, the present invention can realize three-dimensional multimodal tactile measurement through a layered structure, including temperature, three-dimensional force, and vibration measurement, and the measurements between various parts are relatively independent, which can reduce the crosstalk and coupling between multimodals, that is, various physical quantities can be solved without complex algorithms;

[0046] 5. The layered structure adopted in the present invention is simple to manufacture and has a low cost;

[0047] 6. The present invention can increase the sensing area through an array, and has biocompatibility, and can be integrated into the tool instruments for minimally invasive surgery, with a wide range of application scenarios. Description of the Drawings

[0048] Figure 1 is a schematic structural diagram of a multimodal three-dimensional tactile sensing device based on light reflection and resistive type provided by an embodiment of the present invention;

[0049] Figure 2 is an enlarged schematic diagram of a hemispherical flexible layer with 4 silicone rubber columns;

[0050] Figure 3 is a schematic structural diagram of the thermistor layer;

[0051] Figure 4 is a schematic structural diagram of the piezoresistive film layer;

[0052] Figure 5 It is a schematic diagram of an optical fiber bundle (transmitting optical fiber and receiving optical fiber) and an optical fiber sleeve;

[0053] Figure 6 It is a schematic diagram of the deformation of the optical reflection film layer;

[0054] Figure 7 It is a schematic diagram of the deformation generated by the thermistor layer, piezoresistive film layer, and optical reflection film layer under force;

[0055] Figure 8 It is a schematic diagram of the relationship between the three-dimensional force direction and the resistance change;

[0056] Figure 9 It is a schematic diagram of the slippage of the curve L(t);

[0057] Figure 10 It is a schematic diagram of the structure of the multi-modal three-dimensional tactile sensing system based on optical reflection and resistive type provided by the embodiment of the present invention. Detailed implementation manners

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0059] Embodiment 1

[0060] The embodiment of the present invention provides a multi-modal three-dimensional tactile sensing device based on optical reflection and resistive type, as Figure 1 shown, including a hemispherical flexible layer 1, four silicone rubber columns 2, a thermistor layer 3, a piezoresistive film layer 4 composed of four piezoresistive sensing units, an optical reflection film layer 5, an optical fiber sleeve 6, a transmitting optical fiber 7, and a receiving optical fiber 8. The central axes of the hemispherical flexible layer 1, the four silicone rubber columns 2, the thermistor layer 3, the piezoresistive film layer 4 composed of four piezoresistive sensing units, the optical reflection film layer 5, and the optical fiber sleeve 6 are located on the same vertical line.

[0061] As Figure 2 shown, the top end of the silicone rubber column 2 is an arc-shaped end face, which is connected to the lower part of the hemispherical flexible layer 1. The center points of the end faces of all the silicone rubber columns 2 are equidistant from the center of the sphere of the hemispherical flexible layer 1 and are evenly distributed in the four quadrant areas inside the hemispherical flexible layer 1. The silicone rubber columns 2 can improve the performance of the tactile sensing device, such as sensitivity, etc. Specifically, the material of the hemispherical flexible layer 1 can be set as PDMS, which has biocompatibility, with a radius of 10 mm. The radius of the four silicone rubber columns 2 is 1.5 mm, the radius of the arc-shaped upper surface is 4.5 mm, and the distance from the center of the silicone rubber column 2 to the center of the sphere of the hemispherical flexible layer 1 is The hemispherical mold and the column mold can be obtained by 3D printing. The material of the silicone column is silastic 75u from Dow Corning. After mixing the silicone raw material and the vulcanizing agent evenly, put them into the column mold, carry out high-temperature and high-pressure vulcanization, and finally demold after cooling. First place the silicone rubber column in the hemispherical mold, then mix the basic component and the curing agent of DC184 from Dow Corning at a ratio of 10:1, and pour the mixture into the hemispherical mold. After curing, peel off the hemispherical mold to obtain the hemispherical flexible layer 1 embedded with 4 silicone rubber columns 2. It can be understood that in other embodiments, the top of the silicone rubber column 2 and the hemispherical flexible layer 1 can also be connected together by means of pasting or the like.

[0062] As Figure 3 shown, the thermistor layer 3 is located below the silicone rubber column 2 and includes a flexible base layer 3-1, electrodes 3-2 and a protective layer. The flexible substrate is PI (polyimide). The electrodes 3-2 are uniformly laid on the flexible base layer 3-1 in an arch-shaped path, which can increase the electrode length. The material of the electrodes 3-2 is copper. In order to protect the copper from oxidation, a thin layer of gold is electroplated on the copper surface as a protective layer, covering the electrodes 3-2. Specifically, during implementation, the thickness of the flexible substrate can be set to 1 mm. The manufacturing path of the electrodes starts from the electrode resistance measurement terminal a, makes the electrodes in area 1 in an arch-shaped path, connects to the electrodes in area 2 through point b, makes the electrodes in area 2 in an arch-shaped path from the center to the edge direction, connects to the electrodes in area 3 through point c, makes the electrodes in area 3 in an arch-shaped path from the edge to the center direction, connects to the electrodes in area 4 through point d, makes the electrodes in area 4 in an arch-shaped path from the center to the edge direction, and finally ends at the electrode resistance measurement terminal e. The diameter of the outermost layer of electrodes is 8.5 mm, the diameter of the innermost layer of electrodes is 0.5 mm, and the angle between the concentric electrodes in adjacent areas is 10°.

[0063] The arc length calculation formula for the fan-shaped area is:

[0064]

[0065] where l is the arc length of the fan-shaped area, n is the central angle of the fan-shaped area, π is the pi, and r is the radius.

[0066] Then the total length L of the electrodes can be obtained as:

[0067]

[0068] The resistance of the electrode layer is:

[0069]

[0070] where ρ is the resistivity, A is the cross-section of the conductor, L is the length of the conductor. At room temperature, the resistivity of pure copper is 1.68×10-8 Ω·m. As the temperature increases, the kinetic energy of electrons in copper increases, leading to an increase in resistivity. Therefore, when L, A, and R are known, the temperature can be obtained by measuring the resistance of the thermistor layer 3.

[0071] As Figure 4 shown, each piezoresistive sensing unit 4-1 of the piezoresistive film layer 4 is disposed below the thermistor layer 3 and corresponds to the position of the silicone rubber column 2 one by one. The piezoresistive sensing unit 4-1 is specifically a 1 / 4 circular sector and is symmetrically distributed around the center. The piezoresistive sensing unit 4-1 uses graphene and carbon black as sensitive materials, with resistances R1, R2, R3, and R4 respectively. Its radius is 5 mm and the spacing is 0.4 mm. When a three-dimensional force is applied to the hemispherical flexible layer 1, the piezoresistive sensing unit 4-1 will be subjected to compressive stress and the resistance will change. According to the change amount of the resistance, the magnitude of the applied force can be obtained.

[0072] The light reflection film layer 5 includes a flexible sub-layer and a silver powder sub-layer. The optical fiber sleeve 6 is disposed below the light reflection film layer 5, and the silver powder sub-layer faces the optical fiber sleeve 6. The terminals of the transmitting optical fiber 7 and the receiving optical fiber 8 are arranged coaxially and spaced apart within the optical fiber sleeve 6, thus forming an optical fiber bundle. As Figure 5 shown, specifically: from the inside to the outside are a layer of transmitting optical fiber end faces, a layer of receiving optical fiber end faces, a layer of transmitting optical fiber end faces..., arranged in sequence. As Figure 6 shown in (a) of, after the transmitting laser 7a emitted from the end face of the transmitting optical fiber 7 reaches the light reflection film layer 5, it is reflected to form a reflected laser 8a. The reflected laser 8a reaches the receiving optical fiber 8. When an external force is applied to the light reflection film layer 5, it will deform, causing a change in the local slope of the light reflection film layer 5. As Figure 6 shown in (b) of, thus changing the intensity of the reflected laser. By detecting the light intensity in the receiving optical fiber 8, the detection of the slip signal (vibration signal) can be achieved.

[0073] As Figure 7 shown, the multi-modal three-dimensional tactile sensing device of the embodiment of the present invention mainly has three sensing principles. The first is based on the piezoresistive principle. When an external force is applied to the hemispherical flexible layer 1 and the deformation is transmitted to the piezoresistive film layer 4 through the silicone rubber column 2, the three-dimensional force value is calculated according to the change amount of the resistance on the piezoresistive film layer 4, and the direction of the three-dimensional force is obtained according to the change direction of the resistance. The second is based on the principle that the metal resistance changes with temperature, so the temperature value is calculated according to the resistance of the thermistor layer 3. The third is based on the light reflection film principle. The deformation of the applied external force is transmitted to the light reflection film layer 5, resulting in local deformation of the light reflection film layer 5, and thus the intensity of the reflected light changes. The vibration signal is detected by detecting the change in the light intensity of the receiving optical fiber, thereby realizing the slip detection during the contact process.

[0074] Embodiment Two

[0075] An embodiment of the present invention provides a multi-modal three-dimensional tactile sensing method based on optical reflection and resistive type. This method is based on the multi-modal three-dimensional tactile sensing device of Embodiment 1, and specifically includes the following steps:

[0076] S1. Measure the resistance change amounts of the four piezoresistive sensing units of the piezoresistive film layer, and based on the relationship established in advance between the magnitude of the applied force and the resistance change amounts of the four piezoresistive sensing units, calculate the magnitude of the three-dimensional force applied to the hemispherical flexible layer. Then, determine the direction of the applied three-dimensional force according to whether the resistance of the piezoresistive sensing unit increases or decreases, and output the magnitude and direction of the applied three-dimensional force as the sensed three-dimensional force.

[0077] Among them, the relationship established in advance between the magnitude of the applied force and the resistance change amounts of the four piezoresistive sensing units is specifically obtained through the following method: Apply several three-dimensional forces with different magnitudes to the hemispherical flexible layer, and measure the resistance of each piezoresistive sensing unit when applying three-dimensional forces with different magnitudes; according to the magnitude of the applied three-dimensional force and the corresponding resistance change amount of each piezoresistive sensing unit, fit to obtain the following formula:

[0078] F1 = f1(ΔR1, ΔR2, ΔR3, ΔR4)

[0079] F2 = f2(ΔR1, ΔR2, ΔR3, ΔR4)

[0080] F3 = f3(ΔR1, ΔR2, ΔR3, ΔR4)

[0081] ΔR1 = R1 - R 10 、ΔR2 = R2 - R 20 、ΔR3 = R3 - R 30 、ΔR4 = R4 - R 40

[0082] In the formula, F1, F2, and F3 respectively represent the force components of the three-dimensional force in the x, y, and z directions, f1, f2, and f3 respectively represent the relationships between F1, F2, and F3 and the resistance change amounts, ΔR1, ΔR2, ΔR3, and ΔR4 respectively represent the resistance change amounts of the four piezoresistive sensing units, and R 10 、R 20 、R 30 、R 40 represent the resistances of the four piezoresistive sensing units when no three-dimensional force is applied.

[0083] As Figure 8 shown, under the action of F3, the four silicone rubber columns are compressed, and the four piezoresistive sensing units are subjected to the same compressive stress. Then, the resistances R1, R2, R3, and R4 of the four piezoresistive sensing units decrease simultaneously, and the changing amplitudes are equal, that is:

[0084] ΔR1 = ΔR2 = ΔR3 = ΔR4

[0085] When subjected to the shear forces F1 or F2, two piezoresistive sensing units on one side are subjected to compressive stress, while two piezoresistive sensing units on the other side are subjected to tensile stress. Then, the changes in the resistance values of the sensing units on both sides are opposite. Therefore, the normal force and the shear force can be calculated based on the average value and the difference of the resistance values of the four sensing units. Then, F1, F2, and F3 can be expressed as:

[0086]

[0087] By judging whether the resistance of the four piezoresistive sensing units increases or decreases, the direction of the applied force can be judged. For example, when a force in the z direction is applied, the resistance changes of the four piezoresistive sensing units are consistent. When a force in the x direction is applied, the resistance changes of R1 and R4 are consistent, and the resistance changes of R2 and R3 are consistent. When a force in the y direction is applied, the resistance changes of R1 and R2 are consistent, and the resistance changes of R3 and R4 are consistent.

[0088] S2. Measure the real-time resistance of the thermistor layer, query the relationship between the resistance and the temperature of the thermistor layer established in advance according to the real-time resistance, and obtain the real-time temperature of the thermistor layer as the sensed temperature output.

[0089] Formula In the formula, the resistivity ρ increases with the increase of temperature, and its expression is:

[0090] ρ(T) = ρ0(1 + α(T - T0))

[0091] Among them, ρ(T) represents the resistivity at temperature T, ρ0 is the resistivity at the reference temperature (usually room temperature), α represents the temperature coefficient, which can be obtained through temperature calibration experiments, and T0 represents the reference temperature.

[0092] Therefore, the relationship between the resistance and the temperature of the thermistor layer can be specifically obtained as:

[0093]

[0094] In the formula, R(T) represents the resistance at temperature T.

[0095] S3. Emit laser light to the transmitting optical fiber and record the spots of the reflected laser light in the receiving optical fiber at different times.

[0096] Among them, the reflected laser light in the receiving optical fiber is the laser light in the transmitting optical fiber reflected back by the optical reflection film.

[0097] The recording of the spot of the reflected laser can be carried out by a high-speed CCD. For example, for a high-speed CCD with a frame rate of 6400 fps, 6400 frame images can be recorded in 1 second. If the time of the first frame of the spot image is t1, then the time ti of the i-th frame of the spot image is i as follows:

[0098]

[0099] S4. According to the spots at different recorded times, obtain the curve of the change of the spot gray value with time, and perform discrete wavelet transform on the curve to obtain the vibration frequency, which is output as the sensing vibration frequency.

[0100] Among them, the method for obtaining the curve of the change of the spot gray value with time specifically includes:

[0101] According to the spots at different recorded times, solve the gray values of the spots at different times, as follows:

[0102] G = ∫ s g(x, y)dS

[0103] In the formula, G represents the gray value of the spot, g(x, y) is the gray value distribution of the spot, and S is the area of the entire spot; subtract the gray value of the reference spot from the gray values of the spots at different times to obtain the change amount of the spot gray value:

[0104] ΔG i = G i - G0

[0105] In the formula, G i is the gray value of the spot recorded at the i-th time, and G0 is the gray value of the reference spot when no external force is applied;

[0106] According to the change amount of the spot gray value and the corresponding time, obtain the curve of the change of the spot gray value with time.

[0107] As Figure 9 shown, the position where the curve L(t) of the change of the spot gray value with time changes violently indicates that slippage has occurred. Since slippage is a high-frequency signal, discrete wavelet transform is performed on the curve L(t) to obtain different frequency components in the signal. The formula for the discrete wavelet transform of the curve L(t) is:

[0108]

[0109] Among them, is the scale factor, is the translation factor, where j ∈ Z, k ∈ Z. After discrete wavelet transform, the high-frequency components in the signal can be obtained, so that the vibration frequency at the time of slippage can be obtained.

[0110] Environmental control during measurement by the tactile measurement device;

[0111] For the measurement of the reflected light intensity, the difference between the reflected light spot image of the i-th frame and the reference reflected light spot image is used to obtain the curve L(t). Then, the change of the external light source will affect the measurement result. This embodiment controls from two aspects: vibration isolation and light source:

[0112] ① Install the device on a platform with vibration isolation measures.

[0113] ② During the experiment, light shielding treatment is required to reduce the influence of environmental stray light on the experimental results.

[0114] Embodiment 3

[0115] The embodiment of the present invention provides a multimodal three-dimensional tactile sensing system based on light reflection and resistive type, as Figure 10 shown, including the multimodal three-dimensional tactile sensing device of Embodiment 1, and further including:

[0116] A resistance measuring instrument 9, which is respectively connected to the thermistor layer and the piezoresistive film layer, is used to measure the real-time resistance of the thermistor layer, query the relationship between the resistance and temperature of the thermistor layer established in advance according to the real-time resistance, obtain the real-time temperature of the thermistor layer, and output it as the sensing temperature; and measure the resistance change of the four piezoresistive sensing units of the piezoresistive film layer, and calculate the magnitude of the three-dimensional force applied to the hemispherical flexible layer based on the relationship between the magnitude of the applied force and the resistance change of the four piezoresistive sensing units established in advance. Then, judge the direction of the applied three-dimensional force according to whether the resistance of the piezoresistive sensing unit increases or decreases, and output the magnitude and direction of the applied three-dimensional force as the sensing three-dimensional force;

[0117] An optical fiber light source 10, which is used to emit laser light to the transmitting optical fiber;

[0118] A camera 11, which is used to record the light spots of the reflected laser in the receiving optical fiber at different times. Among them, the reflected laser in the receiving optical fiber is the laser reflected back by the laser in the transmitting optical fiber through the light reflection film; a calculation module 12, which is used to obtain the curve of the change of the light spot gray value with time according to the recorded light spots at different times, and perform discrete wavelet transform on the curve to obtain the vibration frequency, and output it as the sensing vibration frequency;

[0119] A base 13, which is used to place the optical fiber light source 10, the camera 11, the resistance measuring instrument 9 and the calculation module 12 on the base 13.

[0120] Among them, the resistance measuring instrument 9 can be the UT3516 of UNI-T, with a measurement range of 1 μΩ to 2.2 MΩ, an accuracy of 0.05%, and a minimum resolution of 1 μΩ.

[0121] The camera 11 can be a high-speed CCD, with the model of FASTCAM Mini AX200, a size of 20.48×20.48 mm, a resolution of 1024×1024, and a frame rate of 6400 fps.

[0122] The calculation module 12 can specifically be a computer or a device or module with calculation functions.

[0123] The system provided by the embodiment of the present invention can be used to execute the method provided by the second embodiment of the present invention, and has the corresponding functions and beneficial effects for executing the method. For the details not described in detail, refer to the second embodiment and will not be repeated here.

[0124] It should be noted that in the embodiment of the present invention, the included units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0125] The embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented only by hardware as long as the functions or effects can be achieved.

[0126] It should be understood that the above embodiments and the descriptions in the specification are only the principles, main features and advantages of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the protection scope of the present invention.

Claims

1. A multi-modal three-dimensional tactile sensing device based on light reflection and resistance, characterized in that: It includes a hemispherical flexible layer, four silicone rubber columns, a thermistor layer, a piezoresistive film layer composed of four piezoresistive sensing units, a light reflecting film layer, an optical fiber sleeve, a transmitting optical fiber, and a receiving optical fiber. The top of the silicone rubber column is connected to the bottom of the hemispherical flexible layer and is evenly distributed in four quadrants in the hemispherical flexible layer. The thermistor layer is located below the silicone rubber column. Each piezoresistive sensing unit of the piezoresistive film layer is arranged below the thermistor layer and corresponds to the position of the silicone rubber column one by one. The light reflecting film layer is located below the piezoresistive film layer. The optical fiber sleeve is arranged below the light reflecting film layer. The terminals of the transmitting optical fiber and the receiving optical fiber are coaxially arranged at intervals in the optical fiber sleeve to form an optical fiber bundle. The center lines of the hemispherical flexible layer, the thermistor layer, the piezoresistive film layer, the optical fiber bundle, the light reflecting film layer, and the optical fiber sleeve are located on the same vertical line.

2. The multi-modal three-dimensional tactile sensing device based on light reflection and resistance according to claim 1, characterized in that: The end centers of all silicone rubber columns are equidistant from the center of the hemispherical flexible layer.

3. The multi-modal three-dimensional tactile sensing device based on light reflection and resistance according to claim 1, characterized in that: The thermistor layer includes a flexible base layer, an electrode and a protective layer. The electrode is evenly laid on the flexible base layer in an arc-shaped path, and the protective layer covers the electrode.

4. The multi-modal three-dimensional tactile sensing device based on light reflection and resistance according to claim 1, characterized in that: The piezoresistive sensing units are specifically in the shape of a 1 / 4 circle sector and are symmetrically distributed around the center.

5. The multi-modal three-dimensional tactile sensing device based on light reflection and resistance according to claim 1, characterized in that: The light reflective film layer includes a flexible sublayer and a silver powder sublayer.

6. A multimodal three-dimensional tactile sensing method based on light reflection and resistance, characterized in that: The method is based on the multimodal three-dimensional tactile sensing device according to claim 1, and specifically comprises the following steps: The resistance change of the four piezoresistive sensing units of the piezoresistive film layer is measured, and the magnitude of the three-dimensional force applied to the hemispherical flexible layer is calculated based on the resistance change and the relationship between the applied force and the resistance change of the four piezoresistive sensing units established in advance, and the direction of the applied three-dimensional force is determined according to whether the resistance of the piezoresistive sensing unit increases or decreases, and the magnitude and direction of the applied three-dimensional force are used as the sensing three-dimensional force output; Measuring the real-time resistance of the thermistor layer, and querying the pre-established relationship between the resistance and temperature of the thermistor layer according to the real-time resistance to obtain the real-time temperature of the thermistor layer as the sensing temperature output; Emitting laser light to the transmitting optical fiber, and recording the light spots of the reflected laser light in the receiving optical fiber at different times, wherein the reflected laser light in the receiving optical fiber is the laser light in the transmitting optical fiber reflected back by the light reflecting film; According to the light spots recorded at different times, a curve showing the change of the light spot gray value over time is obtained, and the vibration frequency is obtained by performing discrete wavelet transform on the curve as the sensing vibration frequency output.

7. The multimodal three-dimensional tactile sensing method based on light reflection and resistance according to claim 6, characterized in that: The pre-established relationship between the magnitude of the applied force and the resistance change of the four piezoresistive sensing units is specifically obtained by the following method: Applying a plurality of three-dimensional forces of different magnitudes to the hemispherical flexible layer, and measuring the resistance of each piezoresistive sensing unit when the three-dimensional forces of different magnitudes are applied; According to the magnitude of the applied three-dimensional force and the corresponding resistance change of each piezoresistive sensing unit, the following equation is fitted: F1=f1(ΔR1,ΔR2,ΔR3,ΔR4) F2=f2(ΔR1,ΔR2,ΔR3,ΔR4) F3=f3(ΔR1,ΔR2,ΔR3,ΔR4) ΔR1=R1-R 10 、ΔR2=R2-R 20 、ΔR3=R3-R 30 、ΔR4=R4-R 40 Where F1, F2, and F3 represent the force components of the three-dimensional force in the x, y, and z directions, respectively; f1, f2, and f3 represent the relationship between F1, F2, and F3 and the resistance change, respectively; ΔR1, ΔR2, ΔR3, and ΔR4 represent the resistance changes of the four piezoresistive sensing units, respectively; R 10 , R 20 , R 30 , R 40 represents the resistance of the four piezoresistive sensing units when no three-dimensional force is applied.

8. The multi-modal three-dimensional tactile sensing method based on light reflection and resistance according to claim 6, characterized in that: The relationship between the resistance and temperature of the thermistor layer established in advance is specifically: Where R(T) represents the resistance at temperature T, ρ0 is the resistivity at the reference temperature, α represents the temperature coefficient, which can be obtained through temperature calibration experiments, T0 represents the reference temperature, L represents the length of the electrode on the thermistor layer, and A is the cross-sectional area of ​​the electrode.

9. The multimodal three-dimensional tactile sensing method based on light reflection and resistance according to claim 6, characterized in that: The curve of the change of the gray value of the light spot over time is obtained according to the light spots recorded at different times, specifically including: According to the light spots recorded at different times, the grayscale values ​​of the light spots at different times are solved: G=∫ s g(x,y)dS In the formula, G represents the gray value of the light spot, g(x,y) is the gray value distribution of the light spot, and S is the area of ​​the entire light spot; Subtract the grayscale value of the reference light spot from the grayscale value of the light spot at different times to get the change in the grayscale value of the light spot: ΔG i =G i -G0 In the formula, G i is the gray value of the light spot recorded at the i-th time, and G0 is the gray value of the reference light spot when no external force is applied; According to the change amount of the gray value of the light spot and the corresponding time, a curve of the change amount of the gray value of the light spot over time is obtained.

10. A multi-modal three-dimensional tactile sensing system based on light reflection and resistance, characterized in that: The multimodal three-dimensional tactile sensing device according to claim 1 further comprises: A resistance measuring instrument is connected to the thermistor layer and the piezoresistive film layer respectively, and is used to measure the real-time resistance of the thermistor layer, and to query the pre-established relationship between the resistance and temperature of the thermistor layer according to the real-time resistance, and obtain the real-time temperature of the thermistor layer as the sensing temperature output; and to measure the resistance change of the four piezoresistive sensing units of the piezoresistive film layer, and to calculate the size of the three-dimensional force applied to the hemispherical flexible layer according to the resistance change based on the pre-established relationship between the applied force and the resistance change of the four piezoresistive sensing units, and then to judge the direction of the applied three-dimensional force according to whether the resistance of the piezoresistive sensing unit increases or decreases, and to output the size and direction of the applied three-dimensional force as the sensing three-dimensional force; A fiber optic light source, used for emitting laser light to a transmitting optical fiber; A camera, used to record the light spots of the reflected laser in the receiving optical fiber at different times, wherein the reflected laser in the receiving optical fiber is the laser in the transmitting optical fiber reflected back by the light reflecting film; The calculation module is used to obtain a curve of the change of the gray value of the light spot over time according to the light spots recorded at different times, and obtain the vibration frequency as the sensing vibration frequency output by performing discrete wavelet transform on the curve; A base is used to place the optical fiber light source, camera, resistance measuring instrument and computing module on the base.

Images