Self-decoupling three-dimensional force multi-mode sensing device, system and method based on radial magnetization

Abstract

The invention discloses a self-decoupling three-dimensional force multi-mode sensing device, system and method based on radial magnetization, the device comprises a flexible magnetic film with radial magnetization, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB, a common camera and an infrared camera, the flexible magnetic film, the microstructure layer, the optical waveguide layer and the shell are stacked in sequence from top to bottom, and the Hall element is arranged in the shell. The PCB, the common camera and the infrared camera are all located on the bottom face of the shell, and the Hall element is installed on the PCB. The system comprises a self-decoupling three-dimensional force multi-mode sensing device and a signal processing device, and the signal processing device can realize three-dimensional force decoupling, infrared distance sensing and force application position measurement according to data of a PCB, a common camera and an infrared camera. According to the invention, the resolution is high, three-dimensional force decoupling can be realized, and multi-modal sensing can be realized.

Description

Technical Field

[0001] The present invention relates to pressure sensing technology, and particularly to a self-decoupling three-dimensional force multi-modal sensing device, system and method based on radial magnetization. Background Art

[0002] The human hand can recognize texture, shape and size, can achieve fine tactile feedback and daily operations, and can also distinguish normal force and shear force and achieve ultra-high resolution perception. Through tactile feedback, the finger posture and contact force can also be dynamically adjusted. In recent years, tactile sensors based on various sensing principles have been proposed, including capacitive, resistive, piezoelectric, magnetoelectric, optical, etc. Optical-based tactile sensors can achieve super resolution comparable to that of human skin, but they are relatively large in structure and most can only achieve normal force sensing. With the development of materials science, the use of new materials to manufacture electrodes can achieve shear force measurement. However, there is a coupling relationship between the normal force and the shear force. Although a decoupling model of the normal force and the shear force can be established through calibration experiments, there are still certain challenges in practical applications. First, calibration experiments need to be carried out, and second, a decoupling model needs to be fitted. The whole process is very complex and cumbersome. Moreover, existing three-dimensional force sensing cannot achieve multi-modal data sensing.

[0003] In summary, the current difficulties of tactile sensors are as follows: First, the resolution is relatively low and cannot reach the resolution of human hand skin; second, there is a coupling between the normal force and the shear force; third, multi-modal sensing cannot be achieved. Summary of the Invention

[0004] Aiming at the problems existing in the prior art, the purpose of the present invention is to provide a high-resolution self-decoupling three-dimensional force multi-modal sensing device, system and method based on radial magnetization.

[0005] In order to achieve the above invention purpose, the present invention provides the following technical solutions:

[0006] A self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization, comprising a flexible magnetic film with radial magnetization, a microstructure layer, an optical waveguide layer, a housing, a Hall element, a PCB board, an ordinary camera and an infrared camera. Among them, the flexible magnetic film, the microstructure layer, the optical waveguide layer and the housing are stacked in sequence from top to bottom. The PCB board, the ordinary camera and the infrared camera are all located on the bottom surface of the housing, and the Hall element is installed on the PCB board.

[0007] Further, the centers of the flexible magnetic film, the microstructure layer, the optical waveguide layer and the Hall element are located on the same vertical line.

[0008] Further, the ordinary camera and the infrared camera are symmetrically placed on both sides of the Hall element.

[0009] Further, the material of the flexible magnetic film is PDMS and NdFeB magnetic powder.

[0010] Further, the material of the micro-structure layer is silicone rubber, and its shape is an inverted pyramid.

[0011] Further, the material of the optical waveguide layer is PMMA.

[0012] A three-dimensional force multi-modal sensing system based on radial magnetization includes the above-mentioned self-decoupling three-dimensional force multi-modal sensing device and a signal processing device. The signal processing device specifically includes:

[0013] A data acquisition module, which is used to acquire the photos of leakage light spots taken by an ordinary camera and the infrared light spot photos taken by an infrared camera;

[0014] A force position acquisition module, which is used to calculate the position of the center point of the leakage light spot on the imaging plane of the ordinary camera in the photo of the leakage light spot, and substitute the position of the current center point of the leakage light spot on the imaging plane of the ordinary camera into the relationship curve between the position of the center point of the light spot and the force loading position stored in advance to obtain the current force loading position;

[0015] An infrared distance sensing module, which is used to calculate the size of the infrared light spot in the infrared light spot photo, and substitute the size of the current infrared light spot into the relationship curve between the size of the infrared light spot and the infrared sensing distance stored in advance to obtain the current infrared sensing distance;

[0016] A three-dimensional force decoupling module, which is used to decouple the three-dimensional force according to the following formula:

[0017]

[0018] In the formula, F x , F y , F z respectively represent the forces in the x, y, and z directions of the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, k = 2π / T represents the wave number, T is the period, a 1 , a 2 and b 0 , b 1 , b 2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x 0 ) respectively represent the ratios of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the flexible magnetic film deforms to the x i position under the action of the force in the x direction and when it is not stressed, R yz (y i ), R yz(y 0 ) represent the ratio of the magnetic flux density in the y - direction to the magnetic flux density in the z - direction when the flexible magnetic film deforms to the y i position under the action of a force in the y - direction, and when not under force. B(z i ), B(z 0 ) represent the total magnetic flux density when the flexible magnetic film deforms to the z i position under the action of a force in the z - direction, and when not under force. (x 0 , y 0 , z 0 ) represent the original coordinate values of the center point of the flexible magnetic film when not under force, and (x i , y i , z i ) represent the coordinate values of the center point of the flexible magnetic film after deforming under the action of a three - dimensional force.

[0019] Furthermore, the relationship curve between the position of the center point of the light spot and the force loading position is obtained by the following method:

[0020] Taking the center position of the upper surface of the flexible magnetic film as the coordinate origin, establish a coordinate system x′y′;

[0021] Taking the center position of the imaging plane of the ordinary camera as the coordinate origin, establish a coordinate system x″′y″′;

[0022] Calculate the coordinate transformation curve between the coordinate system x″′y″′ and the coordinate system x′y′, and use it as the relationship curve between the position of the center point of the light spot and the force loading position.

[0023] Furthermore, the relationship curve between the size of the infrared light spot and the infrared sensing distance is obtained by the following method:

[0024] Place the same infrared sensing object at different distances from the upper surface of the flexible magnetic film, and obtain infrared light spot photos when placing the infrared sensing object;

[0025] Calculate the size of the infrared light spot in the infrared light spot photos at different distances;

[0026] Fit according to the size of the infrared light spot in the infrared light spot photos at different distances to obtain the relationship curve between the size of the infrared light spot and the infrared sensing distance.

[0027] A self - decoupling three - dimensional force multi - modal sensing method based on radial magnetization, which is realized based on the above self - decoupling three - dimensional force multi - modal sensing device. This method includes:

[0028] Obtain the leaked light spot photos taken by an ordinary camera and the infrared light spot photos taken by an infrared camera;

[0029] Calculate the position of the center point of the leakage light spot in the leakage light spot photo on the imaging plane of the ordinary camera, and substitute the position of the current center point of the leakage light spot on the imaging plane of the ordinary camera into the relationship curve between the position of the center point of the light spot and the force loading position stored in advance to obtain the current force loading position;

[0030] Calculate the size of the infrared light spot in the infrared light spot photo, and substitute the size of the current infrared light spot into the relationship curve between the size of the infrared light spot and the infrared sensing distance stored in advance to obtain the current infrared sensing distance;

[0031] Decouple the three-dimensional force according to the following formula:

[0032]

[0033] In the formula, F x , F y , F z respectively represent the forces in the x, y, and z directions in the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, k = 2π / T represents the wave number, T is the period, a 1 , a 2 and b 0 , b 1 , b 2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x 0 ) respectively represent the ratios of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the flexible magnetic film deforms to the x i position under the action of the force in the x direction and when not under force, R yz (y i ), R yz (y 0 ) respectively represent the ratios of the magnetic flux density in the y direction to the magnetic flux density in the z direction when the flexible magnetic film deforms to the y i position under the action of the force in the y direction and when not under force, B(z i ), B(z 0 ) respectively represent the total magnetic flux densities when the flexible magnetic film deforms to the z i position and when not under force under the action of the force in the z direction, (x 0 , y 0 , z 0 ) represents the original coordinate value of the center point of the flexible magnetic film when not under force, (x i , y i , z i ) represents the coordinate value of the center point of the flexible magnetic film after deformation under the action of the three-dimensional force.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] 1. In the present invention, the normal force and the shear force are self-decoupled through the radially magnetized flexible magnetic film, and no complex calibration experiment is required to fit the decoupling model. By establishing a mathematical model between the magnetic flux density on the magnetic film and the normal force and the shear force, the decoupling of the normal force and the shear force is realized.

[0036] 2. In the present invention, the sensing device can realize three-dimensional force measurement. When the normal force is applied, local deformation will occur on the surface of the flexible magnetic film, and when the tangential force is applied, the flexible magnetic film will generate tangential displacement.

[0037] 3. In the present invention, different measurement ranges and sensitivities can also be realized by replacing the microstructure layer with different elastic moduli and the thickness of the flexible magnetic film.

[0038] 4. In the present invention, the detection of the acting force position is realized based on the total internal reflection method. The infrared distance measurement of the human hand is realized by using an infrared camera.

[0039] 5. In the present invention, multi-modal measurement of three-dimensional force, acting force position and infrared distance can be realized.

[0040] 6. In the present invention, different measurement units are adopted for multi-modal measurement, so no complex multi-modal decoupling algorithm is required, and the structure is simple and the application range is wide. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic structural diagram of a self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization provided by an embodiment of the present invention;

[0042] Figure 2 is a top view of the radially magnetized flexible magnetic film of the present invention;

[0043] Figure 3 is a schematic magnetic field diagram of the flexible magnetic film of the present invention;

[0044] Figure 4 is a schematic diagram of the normal force applied to the self-decoupling three-dimensional force multi-modal sensing device of the present invention;

[0045] Figure 5 is a schematic diagram of the shear force applied to the self-decoupling three-dimensional force multi-modal sensing device of the present invention;

[0046] Figure 6 is a schematic diagram of total internal reflection in the optical waveguide layer of the present invention;

[0047] Figure 7 is a schematic diagram of the acting force position detection of the present invention;

[0048] Figure 8It is a schematic diagram of coordinate transformation of the present invention;

[0049] Figure 9 It is a schematic diagram of infrared distance measurement of the present invention.

[0050] Reference numerals: 1 - flexible magnetic film, 2 - microstructure layer, 3 - optical waveguide layer, 4 - housing, 5 - LED lamp, 6 - ordinary camera, 7 - PCB, 8 - infrared camera, 9 - signal processing device, 1-1: S pole, 1-2: N pole, 1-3: magnetic induction line, 1-4: upper surface of the flexible magnetic film, 3-1: lower surface of the optical waveguide layer; 5-1: light ray, 5-2: leaked light ray, 6-1: leaked light spot, 6-2: imaging surface, 8-1: infrared light spot. Specific embodiments

[0051] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0052] Embodiment 1

[0053] The embodiment of the present invention provides a self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization, as Figure 1 shown, including a flexible magnetic film 1 with radial magnetization, a microstructure layer 2, an optical waveguide layer 3, a housing 4, an ordinary camera 6, a Hall element (not shown in the figure), a PCB board 7, and an infrared camera 8. Among them, the flexible magnetic film 1, the microstructure layer 2, and the optical waveguide layer 3 are stacked in sequence from top to bottom. The PCB board 7, the ordinary camera 6, and the infrared camera 8 are all located on the bottom surface of the housing 4, and the Hall element is installed on the PCB board 7. The LED lamp 5 is used as a light source, and the light ray 5-1 emitted by the LED lamp 5 undergoes total internal reflection inside the optical waveguide layer 3. The centers of the flexible magnetic film 1, the microstructure layer 2, and the optical waveguide layer 3 are located on the same vertical line as the center of the Hall element. The ordinary camera 6 and the infrared camera 8 are symmetrically placed on both sides of the Hall element.

[0054] In specific implementation, the flexible magnetic film 1 can be made of a mixture of PDMS (polydimethylsiloxane) and NdFeB (neodymium iron boron) magnetic powder, with a thickness of 1 mm. After curing, a pulse magnetizer is used to radially magnetize the film. After radial magnetization, the flexible magnetic film 1 is as Figure 2 shown, the S pole 1-1 and the N pole 1-2 are spaced apart and partitioned from the inside to the outside, and the magnetic induction lines 1-3 of the flexible magnetic film 1 are distributed as Figure 3As shown in the figure. The material of the micro-structure layer 2 is silicone rubber, with a Shore A hardness of 70, and its shape is an inverted pyramid with a height of 1 mm. The material of the optical waveguide layer 3 is PMMA (polymethyl methacrylate), and its thickness is 1 mm. The flexible film 1 and the micro-structure layer 2 are assembled using an adhesive. The Hall element uses a commercial MLX90393, and its sensitivity is adjustable. To achieve the same measurement range in the x and y directions, the magnetic pole center of the flexible magnetic film 1 needs to be aligned with the center of the Hall element. The material of the housing 4 is stainless steel, with a height of 5 mm. An ordinary camera 6 is used to detect the position of the acting force, and an infrared camera 8 is used to achieve infrared distance measurement.

[0055] The working principle of the embodiment of the present invention is as follows:

[0056] 1. Magnetize the flexible magnetic film 1 in a radial magnetization manner. When an external force is applied to the flexible magnetic film 1, the flexible magnetic film 1 will produce local deformation, and the magnetic flux density below it will change. Use the Hall element to measure it, and convert it into an electrical signal through the PCB board 7, and then analyze and process the electrical signal to decouple and obtain the magnitude and direction of the three-dimensional force;

[0057] 2. After the flexible magnetic film 1 is radially magnetized, the change in its total magnetic flux density B(z) is only related to the force in the z direction. The ratio R xz of the magnetic flux density in the x direction to the magnetic flux density in the z direction is only related to the force in the x direction, and the ratio R yz of the magnetic flux density in the y direction to the magnetic flux density in the z direction is only related to the force in the y direction;

[0058] 3. When a force in the z direction is applied, the flexible magnetic film 1 generates a pit in the contact area, and the magnetic flux density at the corresponding position will change due to local deformation, as Figure 4 shown. Establish a mathematical model between the force in the z direction, the displacement in the z direction, and the total magnetic flux density B(z). When a shear force in the x or y direction is applied, as Figure 5 shown, the flexible magnetic film 1 will generate a corresponding displacement in the x or y direction, and the center position of the flexible magnetic film 1 will shift. Establish a mathematical model between the x direction and the displacement in the x direction and the ratio R xz of the magnetic flux density, and a mathematical model between the y direction and the displacement in the y direction and the ratio R yz of the magnetic flux density, so as to achieve three-dimensional force decoupling;

[0059] 4. After the flexible magnetic film 1 is stressed, the force value is transmitted to the micro-structure layer 2, which causes the micro-structure layer 3 to come into contact with the optical waveguide layer 3. A light spot will be generated in the contact area. Capture the light spot through the ordinary camera 6, and establish the corresponding relationship between the center position of the light spot and the position of the acting force, so as to achieve the detection of the position of the acting force;

[0060] V: When located at different positions in the z direction of the flexible magnetic film 1, the images captured by the infrared camera 8 are different. By establishing the correspondence between the size of the infrared image and different positions in the z direction, infrared distance measurement can be achieved.

[0061] Embodiment 2

[0062] The embodiment of the present invention provides a self-decoupling three-dimensional force multi-modal sensing system based on radial magnetization. As Figure 1 shown, in addition to the self-decoupling three-dimensional force multi-modal sensing device described in Embodiment 1, it further includes a signal processing device 9. The signal processing device 9 is connected to the ordinary camera 6, the PCB board 7, and the infrared camera 8. The signal processing device specifically includes:

[0063] A data acquisition module for acquiring the leakage spot photos taken by the ordinary camera and the infrared spot photos taken by the infrared camera;

[0064] A force position acquisition module for calculating the position of the center point of the leakage spot in the leakage spot photo on the imaging plane of the ordinary camera, and substituting the position of the current center point of the leakage spot on the imaging plane of the ordinary camera into the relationship curve between the position of the center point of the spot and the force loading position stored in advance to obtain the current force loading position;

[0065] An infrared distance sensing module for calculating the size of the infrared spot in the infrared spot photo, and substituting the size of the current infrared spot into the relationship curve between the size of the infrared spot and the infrared sensing distance stored in advance to obtain the current infrared sensing distance;

[0066] A three-dimensional force decoupling module for decoupling the three-dimensional force according to the following formula:

[0067]

[0068] In the formula, F x , F y , F z respectively represent the forces in the x, y, and z directions of the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, k = 2π / T represents the wave number, T is the period, a 1 , a 2 and b 0 , b 1 , b 2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x 0 ) respectively represent the ratio of the magnetic flux density in the x direction to the magnetic flux density in the z direction of the flexible magnetic film when deformed to the x i position under the action of the force in the x direction and when not subjected to force, Ryz (y i )、R yz (y 0 ) respectively represent the ratio of the magnetic flux density in the y - direction to the magnetic flux density in the z - direction when the flexible magnetic film deforms to the y i position under the action of a force in the y - direction and when it is not under force. B(z i )、B(z 0 ) respectively represent the total magnetic flux density when the flexible magnetic film deforms to the z i position under the action of a force in the z - direction and when it is not under force. (x 0 ,y 0 ,z 0 ) represents the original coordinate value of the center point of the flexible magnetic film when it is not under force, and (x i ,y i ,z i ) represents the coordinate value of the center point of the flexible magnetic film after deformation under the action of a three - dimensional force.

[0069] Among them, the derivation process of the decoupling formula of the three - dimensional force decoupling module is as follows:

[0070] In the x - y plane, the thickness of the flexible magnetic film 1 is d. It is assumed that the upper and lower surfaces of the flexible magnetic film 1 are z = 0 and z = d respectively. The ideal flexible magnetic film 1 is radially magnetized as the superposition of two orthogonal sine curves on the x - y plane. The magnetic field components in the x, y, and z directions are:

[0071] M x = M 0 sin(kx)

[0072] M y = M 0 sin(ky) (1)

[0073] M z = 0

[0074] Among them, k = 2π / T is the wave number, T is the period, and M 0 is the maximum amplitude of each component. M x 、M y 、M z represent the magnetic field components at the coordinate (x,y,z).

[0075] The field in the plane from the Poisson equation and Laplace equation above and below is:

[0076]

[0077] Among them, represents the scalar potential of the magnetic field above the flexible magnetic film, represents the scalar potential of the magnetic field inside the flexible magnetic film, Represents the scalar potential of the magnetic field below the flexible magnetic film.

[0078] The magnetic flux density B is proportional to the sum of the magnetic field H and the magnetization M. The calculation formula for the magnetic flux density B can be obtained as:

[0079] B = μ 0 (H + M) (3)

[0080] where μ 0 is the magnetic permeability of free space, with a magnitude of 4π×10 -7 H·m -1 . According to the magnetization intensity outside the magnetic material being 0, the magnetic flux densities below the flexible magnetic film 1 can be obtained as:

[0081]

[0082] where B x , B y , B z respectively represent the magnetic flux densities at the coordinates (x, y, z) below the flexible magnetic film 1.

[0083] According to formula (4), calculate the magnetic flux density B(x, y, z) below the Halbach plane, and the magnetic field ratios R xz and R yz The calculation formulas are:

[0084]

[0085] When the displacement generated by the applied force < 1 mm, formula (5) can be approximated as:

[0086] B(x, y, z) = a 0 μ 0 M 0 (1 - e kd )e kz + b 0

[0087] R xz = a 1 tan(kx) + b 1 (6)

[0088]

[0089] where a 0 , a 1 , a 2 and b 0 , b 1 , b 2is a pre-calibrated fitting coefficient obtained by fitting multiple data pairs. Then, according to Equation (6), the expressions for x, y, and z can be obtained as follows:

[0090]

[0091] When the flexible magnetic film 1 is under an external force, the center point coordinates deform from the position (x 0 , y 0 , z 0 ) to the position (x i , y i , z i ). According to Hooke's law, the forces F x , F y , F z in the x, y, and z directions can be calculated as follows:

[0092]

[0093] where S is the contact area, and γ x γ y , ε, Δx, Δy, Δz, G, E, h are the strain in the x direction, the strain in the y direction, the strain in the z direction, the deformation in the x direction, the deformation in the y direction, the deformation in the z direction, the shear modulus, the elastic modulus, and the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, respectively. The expression between the shear modulus and the elastic modulus is:

[0094]

[0095] υ is the Poisson's ratio, then we can obtain:

[0096]

[0097] From this, it can be concluded that F x is only related to the deformation in the x direction and R xz , F y is only related to the deformation in the y direction and R yz , and F z is only related to the deformation in the z direction and the total magnetic flux density B(z). Given S, G, k, and h, the present invention can achieve the three-dimensional force decoupling of F x , F y , F z .

[0098] As Figure 6 shown, when no external force is applied, the LED is the light source, and total internal reflection occurs inside the optical waveguide layer 3. The refractive index n 1 of the optical waveguide layer is 1.49, and the refractive index of air n 2 is 1. At point B, according to the refraction law, we can obtain:

[0099] n 1 sinθ i = n 2 sinθ t

[0100] where θ i is the incident angle, and θ t is the refraction angle when the refraction angle θ t = 90°, the critical angle of the incident angle can be obtained as:

[0101]

[0102] As Figure 7 shown, when a force is applied to the flexible magnetic film 1, the microstructure layer 2 deforms and comes into contact with the optical waveguide layer 3. Since the total internal reflection condition is not satisfied in the contact area, leakage light is generated at point D. By using an ordinary camera 6 to obtain a photo of the leakage spot, the position of the acting force can be deduced based on the center position of the leakage spot 6-1.

[0103] As Figure 8 shown, assuming that the center position of the upper surface 1-4 of the flexible magnetic film 1 is the coordinate origin, a coordinate system x′y′ is established. Assuming that the coordinate of the force application point on the upper surface 1-4 of the flexible magnetic film 1 is (x′ 1 , y′ 1 ), since the optical waveguide layer 3 has the same length and width as the flexible magnetic film 1 and their centers are on the same straight line, similarly, taking the center position of the lower surface 3-1 of the optical waveguide layer 3 as the coordinate origin, a coordinate system x″y″ is established. Then, leakage light is generated at the corresponding position (x″ 1 , y″ 1 ) of the optical waveguide layer 3, forming a leakage spot on the ordinary camera 6. Taking the center position of the imaging surface 6-2 of the ordinary camera 6 as the coordinate origin, a coordinate system x″′y″′ is established. By processing the leakage spot image and locating the center point position of the leakage spot, the coordinates (x″′ 1 , y″′ 1 ) of the center point in the coordinate system x″′y″′ can be obtained. Calculate the coordinate conversion curve between the coordinate system x″′y″′ and the coordinate system x′y′, which is used as the relationship curve f between the center point position of the spot and the force application position. Then, according to the curve, when the coordinates (x″′ 1 , y″′ 1 ) are known, the coordinates (x′ 1 , y′ 1 ) of the force application point position can be obtained as (x′ 1 , y′ 1 ) = f(x″′ 1 , y″′ 1 ).

[0104] All objects with a temperature higher than absolute zero (-273.15 °C) emit electromagnetic radiation, and this phenomenon is called thermal radiation. The normal body temperature is about 36 - 37 °C (about 300 Kelvin). According to the blackbody radiation law, at such a temperature, the electromagnetic waves mainly radiated by the human body are in the infrared band (wavelength about 700 nm to 1 mm). When a human hand or other object with temperature approaches the upper surface of the flexible magnetic film 1, as Figure 9 shown in (a) of Figure 9 , the infrared light energy radiated can be captured by the infrared camera 8, as Figure 9 shown in (b) of Figure 9 , which is an infrared spot pattern of the human hand at two different positions. By establishing a mathematical model between the size of the infrared spot and the distance, infrared distance measurement can be achieved. Specifically, the size of the infrared spot in the infrared spot photos at different distances can be calculated; fitting is performed based on the size of the infrared spot in the infrared spot photos at different distances to obtain the relationship curve between the size of the infrared spot and the infrared sensing distance. For example, when the distance between the human hand and directly above the flexible magnetic film is d 1 , and the diameter of the infrared spot is L, the relationship curve G between them is: d 1 = G(L).

[0105] It should be noted that in the embodiments of the above system, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for easy distinction from each other and do not limit the protection scope of the present invention.

[0106] The embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented only by hardware, as long as the functions or effects can be achieved.

[0107] Embodiment Three

[0108] The embodiment of the present invention provides a self-decoupling three-dimensional force multi-modal sensing method based on radial magnetization, which is implemented based on the self-decoupling three-dimensional force multi-modal sensing device of the above Embodiment One. The method includes:

[0109] Obtain the leakage spot photos taken by an ordinary camera and the infrared spot photos taken by an infrared camera;

[0110] Calculate the position of the center point of the leakage light spot in the leakage light spot photo on the imaging surface of the ordinary camera, and substitute the position of the current center point of the leakage light spot on the imaging surface of the ordinary camera into the relationship curve between the position of the center point of the light spot and the force loading position stored in advance to obtain the current force loading position;

[0111] Calculate the size of the infrared light spot in the infrared light spot photo, and substitute the size of the current infrared light spot into the relationship curve between the size of the infrared light spot and the infrared sensing distance stored in advance to obtain the current infrared sensing distance;

[0112] Decouple the three-dimensional force according to the following formula:

[0113]

[0114] In the formula, F x , F y , F z respectively represent the forces in the x, y, and z directions in the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, k = 2π / T represents the wave number, T is the period, a 1 , a 2 and b 0 , b 1 , b 2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x 0 ) respectively represent the ratio of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the flexible magnetic film deforms to the x i position under the action of the force in the x direction and when it is not stressed, R yz (y i ), R yz (y 0 ) respectively represent the ratio of the magnetic flux density in the y direction to the magnetic flux density in the z direction when the flexible magnetic film deforms to the y i position under the action of the force in the y direction and when it is not stressed, B(z i ), B(z 0 ) respectively represent the total magnetic flux density when the flexible magnetic film deforms to the z i position and when it is not stressed under the action of the force in the z direction, (x 0 , y 0 , z 0 ) represents the original coordinate value of the center point of the flexible magnetic film when it is not stressed, (x i , y i , z i ) represents the coordinate value of the center point of the flexible magnetic film after being deformed by the three-dimensional force.

[0115] This method corresponds one by one to the system in Embodiment 2. For details not described here, please refer to Embodiment 1 and will not be elaborated further.

[0116] It should be understood that the above embodiments and the descriptions in the specification only illustrate the principles, main features and advantages of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and all these changes and improvements fall within the protection scope of the present invention.

Claims

1. A self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization, characterized in that: The invention comprises a flexible magnetic film with radial magnetization, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB board, an ordinary camera and an infrared camera, wherein the flexible magnetic film, the microstructure layer, the optical waveguide layer and the shell are stacked in sequence from top to bottom, the PCB board, the ordinary camera and the infrared camera are all located on the bottom surface of the shell, and the Hall element is mounted on the PCB board.

2. The self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization according to claim 1, characterized in that: The centers of the flexible magnetic film, the microstructure layer, the optical waveguide layer and the center of the Hall element are located on the same vertical line.

3. The self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization according to claim 1, characterized in that: The ordinary camera and the infrared camera are symmetrically placed on both sides of the Hall element.

4. The self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization according to claim 1, characterized in that: The materials of the flexible magnetic film are PDMS and NdFeB magnetic powder.

5. The self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization according to claim 1, characterized in that: The material of the microstructure layer is silicone rubber and its shape is an inverted pyramid.

6. The self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization according to claim 1, characterized in that: The material of the optical waveguide layer is PMMA.

7. A self-decoupling three-dimensional force multimodal sensing system based on radial magnetization, characterized in that: The invention comprises the self-decoupling three-dimensional force multimodal sensing device and the signal processing device according to claim 1, wherein the signal processing device specifically comprises: A data acquisition module, used to acquire leaked light spot photos taken by an ordinary camera and infrared light spot photos taken by an infrared camera; A force position acquisition module is used to calculate the position of the center point of the leaked light spot in the leaked light spot photo on the imaging surface of the ordinary camera, and substitute the current position of the center point of the leaked light spot on the imaging surface of the ordinary camera into the pre-stored relationship curve between the center point position of the light spot and the force loading position to obtain the current force loading position; The infrared distance sensing module is used to calculate the size of the infrared spot in the infrared spot photo, substitute the current infrared spot size into the pre-stored relationship curve between the infrared spot size and the infrared sensing distance, and obtain the current infrared sensing distance; The 3D force decoupling module is used to decouple the 3D force according to the following equation: In the formula, F x 、F y 、F z They represent the forces in the x, y, and z directions in the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, k=2π / T represents the wave number, T is the period, a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients, and R xz (x i ), R xz (x0) respectively represent the deformation of the flexible magnetic film to x under the action of the x direction force i The ratio of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the magnetic flux density is in the position and no force is applied, R yz (y i ), R yz (y0) respectively represents the deformation of the flexible magnetic film to y direction under the action of y direction force i The ratio of the magnetic flux density in the y direction to the magnetic flux density in the z direction when the magnetic flux density is in the y direction and not under force, B(z i ), B(z0) respectively represent the deformation of the flexible magnetic film to z direction under the action of z direction force i The total magnetic flux density when the flexible magnetic film is in position and not under force, (x0, y0, z0) represents the original coordinate value of the center point when the flexible magnetic film is not under force, (x i ,y i ,z i ) represents the coordinate value of the center point of the flexible magnetic film after being deformed by three-dimensional force.

8. The radial magnetization-based self-decoupling three-dimensional force multi-modal sensing system according to claim 7, characterized in that: The relationship curve between the center point position of the light spot and the force loading position is obtained by the following method: Taking the center position of the upper surface of the flexible magnetic film as the coordinate origin, a coordinate system x′y′ is established; Take the center position of the imaging surface of the ordinary camera as the coordinate origin and establish the coordinate system x″′y″′; The coordinate conversion curve between the coordinate system x″′y″′ and the coordinate system x′y′ is calculated as the relationship curve between the position of the center point of the light spot and the force loading position.

9. The radial magnetization-based self-decoupling three-dimensional force multi-modal sensing system according to claim 7, characterized in that: The relationship curve between the infrared spot size and the infrared sensing distance is obtained by the following method: The same infrared sensing object is placed at different distances from the upper surface of the flexible magnetic film, and infrared spot photos are obtained when the infrared sensing object is placed; Calculate the size of infrared spots in infrared spot photos at different distances; The infrared spot size in the infrared spot photos at different distances is fitted to obtain the relationship curve between the infrared spot size and the infrared sensing distance.

10. A self-decoupling three-dimensional force multimodal sensing method based on radial magnetization, characterized in that: The method is implemented based on the self-decoupling three-dimensional force multimodal sensing device according to claim 1, and the method comprises: Obtain leaked light spot photos taken by a normal camera and infrared light spot photos taken by an infrared camera; Calculate the position of the center point of the leaked light spot in the leaked light spot photo on the imaging surface of the ordinary camera, substitute the current position of the center point of the leaked light spot on the imaging surface of the ordinary camera into the pre-stored relationship curve between the center point position of the light spot and the force loading position, and obtain the current force loading position; Calculate the size of the infrared spot in the infrared spot photo, substitute the current infrared spot size into the pre-stored relationship curve between the infrared spot size and the infrared sensing distance, and obtain the current infrared sensing distance; The three-dimensional forces are decoupled according to the following equation: In the formula, F x 、F y 、F z They represent the forces in the x, y, and z directions in the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, k=2π / T represents the wave number, T is the period, a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients, and R xz (x i ), R xz (x0) respectively represent the deformation of the flexible magnetic film to x under the action of the x direction force i The ratio of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the magnetic flux density is in the position and no force is applied, R yz (y i ), R yz (y0) respectively represents the deformation of the flexible magnetic film to y direction under the action of y direction force i The ratio of the magnetic flux density in the y direction to the magnetic flux density in the z direction when the magnetic flux density is in the y direction and not under force, B(z i ), B(z0) respectively represent the deformation of the flexible magnetic film to z direction under the action of z direction force i The total magnetic flux density when the flexible magnetic film is in position and not under force, (x0, y0, z0) represents the original coordinate value of the center point when the flexible magnetic film is not under force, (x i ,y i ,z i ) represents the coordinate value of the center point of the flexible magnetic film after being deformed by three-dimensional force.

Images