Self-decoupled three-dimensional force multi-modal sensing device, system and method based on radial magnetization

Abstract

The application discloses a self-decoupling three-dimensional force multi-modal sensing device, system and method based on radial magnetization, the device comprises a flexible magnetic film with radial magnetization, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB board, a general camera and an infrared camera, wherein the flexible magnetic film, the microstructure layer, the optical waveguide layer and the shell are stacked in sequence from top to bottom, the PCB board, the general camera and the infrared camera are located on the bottom surface of the shell, and the Hall element is installed on the PCB board. The system comprises the self-decoupling three-dimensional force multi-modal sensing device and a signal processing device, the signal processing device can realize three-dimensional force decoupling, infrared distance sensing and force exertion position measurement according to the data of the PCB board, the general camera and the infrared camera. The application has high resolution, can realize three-dimensional force decoupling, and can realize multi-modal sensing.

Description

TECHNICAL FIELD

[0001] The present application relates to pressure sensing technology, and in particular to a self-decoupling three-dimensional force multi-modal sensing device, system and method based on radial magnetization. BACKGROUND

[0002] Human hands can recognize texture, shape and size, can achieve fine tactile feedback and daily operation, and can also distinguish normal force and shear force and achieve ultra-high resolution perception. Through tactile feedback, the posture and contact force of the fingers can also be dynamically adjusted. In recent years, various tactile sensors with different sensing principles have been proposed, including capacitive, resistive, piezoelectric, magneto-electric, optical, etc. The tactile sensor based on optics can achieve ultra-resolution comparable to the skin, but the structure is relatively large, and most of them can only achieve normal force sensing. With the development of material science, the use of new materials to manufacture electrodes can achieve shear force measurement. However, there is a coupling relationship between the normal force and the shear force, although a decoupling model of the normal force and the shear force can be established through calibration experiments, but there are still some challenges in practical application. First, calibration experiments are needed, and second, a decoupling model needs to be fitted. The whole process is very complex and tedious. Moreover, the existing three-dimensional force sensor cannot achieve multi-modal data sensing.

[0003] In summary, the current tactile sensor has the following difficulties: first, the resolution is low and cannot reach the resolution of human hand skin; second, there is a coupling between the normal force and the shear force; third, it cannot achieve multi-modal sensing. SUMMARY

[0004] In view of the problems existing in the prior art, the purpose of the present application is to provide a self-decoupling three-dimensional force multi-modal sensing device, system and method based on radial magnetization with high resolution.

[0005] In order to achieve the above-mentioned purpose of the application, the present application provides the following technical solutions:

[0006] A self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization, comprising a flexible magnetic film with radial magnetization, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB board, a normal camera and an infrared camera, wherein the flexible magnetic film, the microstructure layer, the optical waveguide layer and the shell are stacked in order from top to bottom, the PCB board, the normal camera and the infrared camera are located on the bottom surface of the shell, and the Hall element is installed on the PCB board.

[0007] Further, the center of the flexible magnetic film, the microstructure layer and the optical waveguide layer is located on the same vertical line as the center of the Hall element.

[0008] Further, the normal camera and the infrared camera are symmetrically placed on both sides of the Hall element.

[0009] Furthermore, the flexible magnetic film is made of PDMS and NdFeB magnetic powder.

[0010] Furthermore, the microstructure layer is made of silicone rubber and is shaped like an inverted pyramid.

[0011] Furthermore, the material of the optical waveguide layer is PMMA.

[0012] A self-decoupled three-dimensional force multimodal sensing system based on radial magnetization includes the aforementioned self-decoupled three-dimensional force multimodal sensing device and a signal processing device, wherein the signal processing device specifically includes:

[0013] The data acquisition module is used to acquire leaked light spot photos taken by ordinary cameras and infrared light spot photos taken by infrared cameras;

[0014] The force position acquisition module is used to calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera. The current position of the center point of the leaked spot on the imaging plane of the ordinary camera is substituted into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position.

[0015] The infrared distance sensing module is used to calculate the size of the infrared spot in the infrared spot photo, and substitute the current size of the infrared spot into the pre-stored relationship curve between the size of the infrared spot and the infrared sensing distance to obtain the current infrared sensing distance.

[0016] The three-dimensional force decoupling module is used to decouple three-dimensional forces according to the following formula:

[0017]

[0018] In the formula, F x F y F z Let S represent the force in the x, y, and z directions of the three-dimensional force, respectively; S represent the contact area; G represent the shear modulus; E represent the elastic modulus; h represent the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board; k = 2π / T represents the wave number; T is the period; a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients; R xz (x i ), R xz (x0) represent the deformation of the flexible magnetic membrane to x0 under the action of force in the x direction. i R is the ratio of the magnetic flux density in the x-direction to the magnetic flux density in the z-direction when the object is in a certain position and is not under force. yz (y i ), R yz (y0) represents the deformation of the flexible magnetic film under the action of force in the y direction to the y direction. iThe ratio of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction when the object is in its current position and is not under force, B(z) i B(z0) and B(z0) represent the deformation of the flexible magnetic film to z0 under the action of force in the z direction, respectively. i The total magnetic flux density at position and without force, where (x0, y0, z0) represents the original coordinates of the center point of the flexible magnetic membrane when it is not under force. i ,y i ,z i () represents the coordinates of the center point of the flexible magnetic membrane after it has been deformed by a three-dimensional force.

[0019] Furthermore, the relationship curve between the center point of the light spot and the position of the force application is obtained in the following way:

[0020] A coordinate system x′y′ is established with the center of the upper surface of the flexible magnetic film as the origin;

[0021] Establish a coordinate system x″′y″′ with the center of the imaging plane of a regular camera as the origin;

[0022] Calculate the coordinate transformation curve between coordinate system x′′′y′′′ and coordinate system x′y′, and use it as the relationship curve between the position of the center point of the light spot and the position of the force loading.

[0023] Furthermore, the relationship curve between the infrared spot size and the infrared sensing distance is obtained in the following way:

[0024] The same infrared sensing object was placed at different distances from the upper surface of the flexible magnetic film, and infrared spot photographs were obtained when the infrared sensing object was placed.

[0025] Calculate the size of the infrared spot in infrared spot photos at different distances;

[0026] By fitting the infrared spot size in infrared spot photos at different distances, a curve showing the relationship between the infrared spot size and the infrared sensing distance is obtained.

[0027] A self-decoupled three-dimensional force multimodal sensing method based on radial magnetization, the method being implemented using the aforementioned self-decoupled three-dimensional force multimodal sensing device, the method comprising:

[0028] Obtain leaked light spot photos taken by ordinary cameras and infrared light spot photos taken by infrared cameras;

[0029] Calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera, and substitute the current position of the center point of the leaked spot on the imaging plane of the ordinary camera into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position.

[0030] The size of the infrared light spot in the infrared light spot photo is calculated, and the size of the current infrared light spot is substituted into a relationship curve between the size of the infrared light spot and the infrared sensing distance to obtain the current infrared sensing distance.

[0031] Decouple the three-dimensional force according to the following formula:

[0032]

[0033] In the formula, F x , F y , F z respectively represent the forces in the x, y and z directions of the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB, k = 2pi / T represents the wave number, T is the period, a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x0) respectively represent the ratio of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the flexible magnetic film is deformed to the x i position under the action of the force in the x direction and when the flexible magnetic film is not forced, R yz (y i ), R yz (y0) respectively represent the ratio of the magnetic flux density in the y direction to the magnetic flux density in the z direction when the flexible magnetic film is deformed to the y i position under the action of the force in the y direction and when the flexible magnetic film is not forced, B(z i ), B(z0) respectively represent the total magnetic flux density when the flexible magnetic film is deformed to the z i position under the action of the force in the z direction and when the flexible magnetic film is not forced, (x0, y0, z0) represents the original coordinate value of the center point of the flexible magnetic film when the flexible magnetic film is not forced, (x i , y i , z i ) represents the coordinate value of the center point of the flexible magnetic film after being deformed by the three-dimensional force.

[0034] Compared with the prior art, the present application has the following advantages:

[0035] 1. In the present application, the normal force and shear force are self-decoupled by the radially magnetized flexible magnetic film, and complex calibration experiments are not needed to fit the decoupling model. The mathematical model of the magnetic flux density on the magnetic film and the normal force and shear force is established to realize the decoupling of the normal force and shear force.

[0036] 2. In the present application, the sensing device can realize three-dimensional force measurement. The surface of the flexible magnetic film will produce local deformation when the normal force is applied, and the flexible magnetic film will produce tangential displacement when the tangential force is applied.

[0037] 3. In the present application, different measurement ranges and sensitivities can be achieved by replacing the microstructure layer and the flexible magnetic film with different elastic modulus and thickness.

[0038] 4. In the present application, the force position detection is realized based on the total internal reflection method. The infrared distance measurement of the human hand is realized by using an infrared camera.

[0039] 5. In the present application, three-dimensional force, force position and infrared distance multi-modal measurement can be realized.

[0040] 6. In the present application, different measurement units are used for multi-modal measurement, so that complex multi-modal decoupling algorithms are not needed, the structure is simple, and the application range is wide. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is a structural schematic diagram of a self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization provided by an embodiment of the present application;

[0042] Figure 2 is a top view of a flexible magnetic film of the present application;

[0043] Figure 3 is a magnetic field schematic diagram of the flexible magnetic film of the present application;

[0044] Figure 4 is a schematic diagram of the application of a normal force to the self-decoupling three-dimensional force multi-modal sensing device;

[0045] Figure 5 is a schematic diagram of the application of a shear force to the self-decoupling three-dimensional force multi-modal sensing device;

[0046] Figure 6 is a schematic diagram of total internal reflection in the optical waveguide layer of the present application;

[0047] Figure 7 is a force position detection schematic diagram of the present application;

[0048] Figure 8 is a coordinate conversion schematic diagram of the present application;

[0049] Figure 9 is an infrared distance measurement schematic diagram of the present application.

[0050] FIG. 1 is a flexible magnetic film, 2 is a microstructure layer, 3 is an optical waveguide layer, 4 is an outer shell, 5 is an LED lamp, 6 is a general camera, 7 is a PCB, 8 is an infrared camera, 9 is a signal processing device, 1-1: S pole, 1-2: N pole, 1-3: magnetic induction line, 1-4: upper surface of the flexible magnetic film, 3-1: lower surface of the optical waveguide layer; 5-1: light, 5-2: leaked light, 6-1: leaked light spot, 6-2: imaging surface, 8-1: infrared light spot. Detailed Implementation

[0051] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0052] Example 1

[0053] This invention provides a self-decoupling three-dimensional force multimodal sensing device based on radial magnetization, such as... Figure 1 As shown, the system includes a radially magnetized flexible magnetic film 1, a microstructure layer 2, an optical waveguide layer 3, a housing 4, a conventional camera 6, a Hall element (not shown), a PCB board 7, and an infrared camera 8. The flexible magnetic film 1, microstructure layer 2, optical waveguide layer 3, and housing 4 are stacked sequentially from top to bottom. The PCB board 7, conventional camera 6, and infrared camera 8 are all located on the bottom surface of the housing 4. The Hall element is mounted on the PCB board 7. An LED lamp 5 serves as the light source, and the light emitted by the LED lamp 5 undergoes total internal reflection within the optical waveguide layer 3. The centers of the flexible magnetic film 1, microstructure layer 2, and optical waveguide layer 3 are aligned vertically with the center of the Hall element. The conventional camera 6 and infrared camera 8 are symmetrically placed on either side of the Hall element.

[0054] In practical implementation, the flexible magnetic film 1 can be made from a mixture of PDMS (polydimethylsiloxane) and NdFeB (neodymium iron boron) magnetic powder, with a thickness of 1 mm. After curing, the film is radially magnetized using a pulse magnetizer. The radially magnetized flexible magnetic film 1 is as follows: Figure 2 As shown, the S pole 1-1 and N pole 1-2 are spaced apart from the inside to the outside, and the magnetic field lines 1-3 of the flexible magnetic film 1 are distributed as follows. Figure 3 As shown. The microstructure layer 2 is made of silicone rubber with a Shore A70 hardness, and is shaped like an inverted pyramid with a height of 1 mm. The optical waveguide layer 3 is made of PMMA (polymethyl methacrylate) with a thickness of 1 mm. The flexible film 1 and microstructure layer 2 are assembled using an adhesive. The Hall element is a commercially available MLX90393 with adjustable sensitivity. To ensure consistent measurement ranges in the x and y directions, the center of the magnetic poles of the flexible magnetic film 1 must be aligned with the center of the Hall element. The outer casing 4 is made of stainless steel with a height of 5 mm. A standard camera 6 is used to detect the location of the applied force, and an infrared camera 8 is used for infrared distance measurement.

[0055] The working principle of this invention embodiment is as follows:

[0056] I. The flexible magnetic film 1 is magnetized in a radial magnetization mode, when an external force is applied to the flexible magnetic film 1, the flexible magnetic film 1 will produce local deformation, and the magnetic flux density below the flexible magnetic film 1 will change, the Hall element is used for measurement, and the PCB board 7 is used for converting into an electric signal, and then the electric signal is analyzed and processed to decouple the size and direction of the three-dimensional force;

[0057] II: The change of the total magnetic flux density B(z) of the flexible magnetic film 1 after radial magnetization is only related to the z-direction force, the change of the ratio R xz of the x-direction magnetic flux density to the z-direction magnetic flux density is only related to the x-direction force, and the change of the ratio R yz of the y-direction magnetic flux density to the z-direction magnetic flux density is only related to the y-direction force;

[0058] III: When subjected to the z-direction force, the flexible magnetic film 1 produces a pit in the contact area, and the magnetic flux density at the corresponding position changes due to local deformation, as shown in Figure 4 , a mathematical model is established between the z-direction force and the z-direction displacement and the total magnetic flux density B(z). When subjected to the x-direction or y-direction shear force, as shown in Figure 5 , the flexible magnetic film 1 produces corresponding x-direction or y-direction displacement, and the center position of the flexible magnetic film 1 is offset, and mathematical models are respectively established between the x-direction and the x-direction displacement and the magnetic flux density ratio R xz and between the y-direction and the y-direction displacement and the magnetic flux density ratio R yz , so as to realize three-dimensional force decoupling;

[0059] IV: After the flexible magnetic film 1 is subjected to force, the force value is transmitted to the microstructure layer 2, so that the microstructure layer 3 and the optical waveguide layer 3 are in contact, and a light spot is generated in the contact area, the light spot is captured by the ordinary camera 6, the corresponding relationship between the light spot center position and the force position is established, and the force position detection can be realized;

[0060] V: When the flexible magnetic film 1 is located at different positions in the z-direction, the infrared camera 8 captures different images, and the corresponding relationship between the infrared image size and the different positions in the z-direction is established, so that the infrared distance measurement can be realized.

[0061] Embodiment II

[0062] The embodiment of the application provides a self-decoupling three-dimensional force multi-modal sensing system based on radial magnetization, as shown in Figure 1 , in addition to the self-decoupling three-dimensional force multi-modal sensing device based on radial magnetization in the embodiment I, the signal processing device 9 is further included, the signal processing device 9 is connected with the ordinary camera 6, the PCB board 7 and the infrared camera 8, and the signal processing device specifically includes:

[0063] a data acquisition module configured to acquire a leaked light spot photo taken by a common camera and an infrared light spot photo taken by an infrared camera;

[0064] a force position acquisition module configured to calculate a position of a leaked light spot center point in an imaging plane of the common camera in the leaked light spot photo, and substitute the position of the current leaked light spot center point in the imaging plane of the common camera into a pre-stored relationship curve between the position of the light spot center point and the force loading position to obtain the current force loading position;

[0065] an infrared distance sensing module configured to calculate a size of an infrared light spot in the infrared light spot photo, and substitute the size of the current infrared light spot into a pre-stored relationship curve between the size of the infrared light spot and the infrared sensing distance to obtain the current infrared sensing distance;

[0066] a three-dimensional force decoupling module configured to decouple the three-dimensional force according to the following formula:

[0067]

[0068] wherein, F x , F y , and F z respectively represent forces in x, y, and z directions in the three-dimensional force, S represents a contact area, G represents a shear modulus, E represents an elastic modulus, h represents a distance between an upper surface of the flexible magnetic film and an upper surface of a PCB, k = 2π / T represents a wave number, T is a period, a1, a2, and b0, b1, b2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x0) respectively represent a ratio of a magnetic flux density in the x direction to a magnetic flux density in the z direction when the flexible magnetic film is deformed to an x i position under the action of a force in the x direction and when the flexible magnetic film is not subjected to force, R yz (y i ), R yz (y0) respectively represent a ratio of a magnetic flux density in the y direction to a magnetic flux density in the z direction when the flexible magnetic film is deformed to a y i position under the action of a force in the y direction and when the flexible magnetic film is not subjected to force, B(z i ), and B(z0) respectively represent a total magnetic flux density when the flexible magnetic film is deformed to a z i position under the action of a force in the z direction and when the flexible magnetic film is not subjected to force, (x0, y0, z0) represents original coordinate values of a center point of the flexible magnetic film when the flexible magnetic film is not subjected to force, and (x i , y i , and z i ) represent coordinate values of the center point of the flexible magnetic film after the flexible magnetic film is deformed by the three-dimensional force.

[0069] wherein, a derivation process of the formula of the three-dimensional force decoupling module is as follows:

[0070] In the x-y plane, the thickness of the flexible magnetic film 1 is d, and the upper and lower surfaces of the flexible magnetic film 1 are set as z=0 and z=d respectively. The ideal flexible magnetic film 1 is radially magnetized and arranged on the superposition of two orthogonal sinusoidal curves in the x-y plane. The magnetic field components in the x, y, and z directions are respectively:

[0071] M x = M0sin(kx)

[0072] M y = M0sin(ky) (1)

[0073] M z = 0

[0074] where k=2π / T is the wave number, T is the period, and M0 is the maximum amplitude of each component. x , M y , M z represent the magnetic field components at coordinates (x, y, z).

[0075] The field in the x-y plane is derived from the Poisson equation and Laplace equation above and below:

[0076]

[0077] where represents the scalar potential of the upper magnetic field of the flexible magnetic film, represents the scalar potential of the internal magnetic field of the flexible magnetic film, represents the scalar potential of the lower magnetic field of the flexible magnetic film.

[0078] The magnetic flux density B is proportional to the sum of the magnetic field H and the magnetization M. The calculation formula of the magnetic flux density B can be obtained as:

[0079] B=μ0(H+M) (3)

[0080] where μ0 is the permeability of free space, with a size of 4π×10 -7 H·m -1 . According to the magnetization outside the magnetic material being 0, the magnetic flux density below the flexible magnetic film 1 is respectively:

[0081]

[0082] where B x , B y , B z represent the magnetic flux density at coordinates (x, y, z) below the flexible magnetic film 1.

[0083] According to formula (4), the magnetic flux density B(x, y, z) below the Halbach plane is calculated, and the magnetic field ratio Rxz and R yz The calculation formula is:

[0084]

[0085] When the displacement caused by the load force is <1mm, formula (5) can be approximated as:

[0086] B(x,y,z)=a0μ0M0(1-e kd )e kz +b0

[0087] R xz =a1tan(kx)+b1 (6)

[0088]

[0089] Where a0, a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients, which are obtained by fitting a plurality of data pairs. Then according to formula (6), the expressions of x, y, z are respectively:

[0090]

[0091] When the flexible magnetic film 1 is under the action of external force, the center point coordinate is deformed from (x0, y0, z0) to (x i ,y i ,z i ) position, according to Hooke's law, the forces F x , F y , F z in x, y, z directions can be obtained, and the calculation formula is:

[0092]

[0093] Where S is the contact area, γ x γ y , ε, Δx, Δy, Δz G, E, h are respectively the x direction strain, y direction strain, z direction strain, x direction deformation, y direction deformation, z direction deformation, shear modulus, elastic modulus, the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB. The expression between shear modulus and elastic modulus is:

[0094]

[0095] υ is the Poisson's ratio, then:

[0096]

[0097] Therefore, F x only with the deformation in x direction and R xzRelated to, F y Only with the deformation of y direction and R yz Related to, F z Only with the deformation of z direction and total magnetic flux density B(z), in the known S, G, k and h, the present application can realize F x , F y , F z Three-dimensional force decoupling.

[0098] As Figure 6 shown, when no external force is applied, the LED is a light source, total internal reflection occurs inside the optical waveguide layer 3, the refractive index n1 of the optical waveguide layer is 1.49, the refractive index of air n2 is 1, at point B, according to the law of refraction, we can get:

[0099] n1sinθ i =n2sinθ t

[0100] Where θ i is the angle of incidence, θ t is the angle of refraction, when the angle of refraction θ t =90°, the critical angle of incidence can be obtained:

[0101]

[0102] As Figure 7 shown, when the force is applied on the flexible magnetic film 1, the microstructure layer 2 is deformed and contacts with the optical waveguide layer 3, because the total internal reflection condition is not met in the contact area, then the leakage light will be generated at point D, the leakage light spot photo is obtained through the ordinary camera 6, and the position of the acting force can be deduced according to the center position of the leakage light spot 6-1.

[0103] As Figure 8As shown, assuming the center position of the upper surface 1-4 of the flexible magnetic film 1 is taken as the origin, a coordinate system x′y′ is established. Assuming the coordinates of the force loading point on the upper surface 1-4 of the flexible magnetic film 1 are (x′1, y′1), since the optical waveguide layer 3 has the same length and width as the flexible magnetic film 1 and the center is located on the same straight line, similarly, assuming the center position of the lower surface 3-1 of the optical waveguide layer 3 is taken as the origin, a coordinate system x″y″ is established. Then, leakage light will be generated at the corresponding position (x″1, y″1) of the optical waveguide layer 3, forming a leakage light spot on the ordinary camera 6. assuming the center position of the imaging surface 6-2 of the ordinary camera 6 is taken as the origin, a coordinate system x″′y″′ is established. By processing the image of the leakage light spot, the center point of the leakage light spot can be located, and the coordinates of the center point in the coordinate system x″′y″′ (x″′1, y″′1) can be obtained. Calculate the coordinate transformation curve between coordinate system x″′y″′ and coordinate system x′y′, and use it as the relationship curve f between the center point of the light spot and the position of the force loading point. Based on the curve, when the coordinates (x″′1, y″′1) are known, the coordinates (x′1, y′1) of the position of the force loading point can be obtained.

[0104] All objects with a temperature above absolute zero (-273.15℃) emit electromagnetic radiation; this phenomenon is called thermal radiation. The normal human body temperature is approximately 36-37℃ (about 300 Kelvin). According to the blackbody radiation law, at this temperature, the electromagnetic waves mainly radiated by the human body are in the infrared band (wavelength approximately 700nm to 1mm). When a human hand or other warm object approaches the upper surface of the flexible magnetic film 1, such as... Figure 9 As shown in (a), the radiated infrared light energy is captured by infrared camera 8, as... Figure 9 As shown in (b), infrared spot images of a hand at two different locations are displayed. Infrared distance measurement can be achieved by establishing a mathematical model relating the infrared spot size to the distance. Specifically, the size of the infrared spot in the images at different distances can be calculated; by fitting the infrared spot sizes in the images at different distances, a curve relating the infrared spot size to the infrared sensing distance can be obtained. For example, if the distance between the hand and directly above the flexible magnetic film is d1, and the diameter of the infrared spot is L, then the relationship curve G is: d1 = G(L).

[0105] It is worth noting that in the embodiments of the above system, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of the present invention.

[0106] The embodiments described above are only illustrative, wherein the modules illustrated as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, i.e., can be located in one place or distributed to multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments. Those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course, can also be realized only by hardware, as long as the function or effect can be realized.

[0107] Embodiment three

[0108] The embodiment of the present application provides a self-decoupling three-dimensional force multi-modal sensing method based on radial magnetization, which is realized based on the self-decoupling three-dimensional force multi-modal sensing device in the above embodiment one, and the method comprises the following steps:

[0109] acquiring a leakage light spot photo taken by an ordinary camera and an infrared light spot photo taken by an infrared camera;

[0110] calculating the position of a leakage light spot center point in the imaging plane of the ordinary camera in the leakage light spot photo, substituting the position of the current leakage light spot center point in the imaging plane of the ordinary camera into a pre-stored relationship curve of the position of the light spot center point and the force loading position, and obtaining the current force loading position;

[0111] calculating the size of the infrared light spot in the infrared light spot photo, substituting the size of the current infrared light spot into a pre-stored relationship curve between the size of the infrared light spot and the infrared sensing distance, and obtaining the current infrared sensing distance;

[0112] decoupling the three-dimensional force according to the following formula:

[0113]

[0114] In the formula, F x , F y , and F z respectively represent the forces in the x, y and z directions of the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB, k = 2π / T represents the wave number, T is the period, a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients, R xz (x i ), and R xz (x0) respectively represent the ratio of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the flexible magnetic film is deformed to the x i position under the action of the force in the x direction and when the flexible magnetic film is not subjected to the force, respectively. yz (y i ), and R yz(y0) respectively represent the magnetic flux density in the y direction when the flexible magnetic film is deformed to the y i position under the action of the force in the y direction, and the ratio of the magnetic flux density in the z direction to the magnetic flux density in the y direction when the flexible magnetic film is not subjected to force, B(z i ), B(z0) respectively represent the magnetic flux density in the z direction when the flexible magnetic film is deformed to the z i position under the action of the force in the z direction, and the total magnetic flux density when the flexible magnetic film is not subjected to force, (x0, y0, z0) represents the original coordinate value of the center point of the flexible magnetic film when the flexible magnetic film is not subjected to force, (x i , y i , z i ) represents the coordinate value of the center point of the flexible magnetic film after the flexible magnetic film is deformed by the three-dimensional force.

[0115] The method corresponds to the system of Embodiment Two on a one-to-one basis, and details not described in Embodiment One will not be described again.

[0116] It should be understood that the above embodiments and the description in the specification are only the principles, main features and advantages of the present application, and various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the protection scope of the present application.

Claims

1. A self-decoupled three-dimensional force multimodal sensing system based on radial magnetization, characterized in that: Includes a self-decoupled three-dimensional force multimodal sensing device and a signal processing device; The self-decoupled three-dimensional force multimodal sensing device includes a radially magnetized flexible magnetic film, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB board, a conventional camera, and an infrared camera. The flexible magnetic film, microstructure layer, optical waveguide layer, and shell are stacked sequentially from top to bottom. The PCB board, conventional camera, and infrared camera are all located on the bottom surface of the shell. The Hall element is mounted on the PCB board. The light emitted by the light source undergoes total internal reflection inside the optical waveguide layer. The signal processing device specifically includes: The data acquisition module is used to acquire leaked spot photos taken by ordinary cameras and infrared spot photos taken by infrared cameras; The force position acquisition module is used to calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera. The current position of the center point of the leaked spot on the imaging plane of the ordinary camera is substituted into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position. The infrared distance sensing module is used to calculate the size of the infrared spot in the infrared spot photo, and substitute the current size of the infrared spot into the pre-stored relationship curve between the size of the infrared spot and the infrared sensing distance to obtain the current infrared sensing distance. The three-dimensional force decoupling module is used to decouple three-dimensional forces according to the following formula: ， ， ， In the formula, , , These represent the forces in the x, y, and z directions in a three-dimensional force, respectively. G represents the contact area, and G represents the shear modulus. Indicates the elastic modulus. This represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, where k = 2π / T represents the wave number and T is the period. , and , , These are pre-calibrated fitting coefficients. , These represent the deformation of the flexible magnetic film under the action of force in the x-direction, respectively. The ratio of the magnetic flux density in the x-direction to the magnetic flux density in the z-direction when the object is in a fixed position and is not under force. These represent the deformation of the flexible magnetic film under the action of force in the y direction, respectively. The ratio of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction when the object is in a fixed position and is not under force. , These represent the deformation of the flexible magnetic film under the action of force in the z-direction, respectively. The total magnetic flux density at position and without force, ( , () represents the original coordinates of the center point of the flexible magnetic membrane when it is not under force. , () represents the coordinates of the center point of the flexible magnetic membrane after it has been deformed by three-dimensional force.

2. The self-decoupling three-dimensional force multimodal sensing system based on radial magnetization according to claim 1, characterized in that: The relationship curve between the center point of the light spot and the position of the force application was obtained in the following way: Establish a coordinate system with the center of the upper surface of the flexible magnetic film as the origin. ; Establish a coordinate system with the center of the image plane of a standard camera as the origin. ; Calculate coordinate system With coordinate system The coordinate transformation curve serves as the relationship curve between the center point of the light spot and the position of the force application.

3. The self-decoupling three-dimensional force multimodal sensing system based on radial magnetization according to claim 1, characterized in that: The relationship curve between the size of the infrared spot and the infrared sensing distance was obtained in the following way: The same infrared sensing object was placed at different distances from the upper surface of the flexible magnetic film, and infrared spot photographs were obtained when the infrared sensing object was placed. Calculate the size of the infrared spot in infrared spot photos at different distances; By fitting the infrared spot size in infrared spot photos at different distances, a curve showing the relationship between the infrared spot size and the infrared sensing distance is obtained.

4. The self-decoupling three-dimensional force multimodal sensing system based on radial magnetization according to claim 1, characterized in that: The centers of the flexible magnetic film, microstructure layer, and optical waveguide layer are located on the same vertical line as the center of the Hall element.

5. The self-decoupling three-dimensional force multimodal sensing system based on radial magnetization according to claim 1, characterized in that: The conventional camera and the infrared camera are symmetrically placed on both sides of the Hall element.

6. The self-decoupling three-dimensional force multimodal sensing system based on radial magnetization according to claim 1, characterized in that: The flexible magnetic film is made of PDMS and NdFeB magnetic powder.

7. The self-decoupling three-dimensional force multimodal sensing system based on radial magnetization according to claim 1, characterized in that: The microstructure layer is made of silicone rubber and is shaped like an inverted pyramid.

8. The self-decoupling three-dimensional force multimodal sensing system based on radial magnetization according to claim 1, characterized in that: The material of the optical waveguide layer is PMMA.

9. A self-decoupled three-dimensional force multimodal sensing method based on radial magnetization, characterized in that: This method is based on a self-decoupled three-dimensional force multimodal sensing device, which includes a radially magnetized flexible magnetic film, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB board, a conventional camera, and an infrared camera. The flexible magnetic film, microstructure layer, optical waveguide layer, and shell are stacked sequentially from top to bottom. The PCB board, conventional camera, and infrared camera are all located on the bottom surface of the shell. The Hall element is mounted on the PCB board, and the light emitted by the light source undergoes total internal reflection inside the optical waveguide layer. The method includes: Obtain leaked light spot photos taken by ordinary cameras and infrared light spot photos taken by infrared cameras; Calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera, and substitute the current position of the center point of the leaked spot on the imaging plane of the ordinary camera into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position. Calculate the size of the infrared spot in the infrared spot image, substitute the current size of the infrared spot into the pre-stored relationship curve between the size of the infrared spot and the infrared sensing distance, and obtain the current infrared sensing distance; The three-dimensional forces are decoupled according to the following formula: ， ， ， In the formula, , , These represent the forces in the x, y, and z directions in a three-dimensional force, respectively. G represents the contact area, and G represents the shear modulus. Indicates the elastic modulus. This represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, where k = 2π / T represents the wave number and T is the period. , and , , These are pre-calibrated fitting coefficients. , These represent the deformation of the flexible magnetic film under the action of force in the x-direction, respectively. The ratio of the magnetic flux density in the x-direction to the magnetic flux density in the z-direction when the object is in a fixed position and is not under force. These represent the deformation of the flexible magnetic film under the action of force in the y direction, respectively. The ratio of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction when the object is in a fixed position and is not under force. , These represent the deformation of the flexible magnetic film under the action of force in the z-direction, respectively. The total magnetic flux density at position and without force, ( , () represents the original coordinates of the center point of the flexible magnetic membrane when it is not under force. , () represents the coordinates of the center point of the flexible magnetic membrane after it has been deformed by three-dimensional force.

Images