Self-decoupling three-dimensional force multi-modal sensing apparatus, system and method based on radial magnetization

Abstract

A self-decoupling three-dimensional force multi-modal sensing apparatus, system and method based on radial magnetization. The apparatus comprises a radially magnetized flexible magnetic film (1), a microstructure layer (2), an optical waveguide layer (3), a housing (4), a Hall element, a PCB (7), a standard camera (6) and an infrared camera (8), wherein the flexible magnetic film (1), the microstructure layer (2), the optical waveguide layer (3) and the housing (4) are sequentially stacked from top to bottom, the PCB (7), the standard camera (6) and the infrared camera (8) are all located on a bottom face of the housing (4), and the Hall element is mounted on the PCB (7). The system comprises a self-decoupling three-dimensional force multi-modal sensing apparatus and a signal processing apparatus (9), wherein the signal processing apparatus (9) can realize three-dimensional force decoupling, force application position measurement and infrared distance sensing on the basis of data of a PCB (7), a standard camera (6) and an infrared camera (8).

Description

Self-decoupled three-dimensional force multimodal sensing device, system, and method based on radial magnetization Technical Field

[0001] This invention relates to pressure sensing technology, and more particularly to a self-decoupled three-dimensional force multimodal sensing device, system, and method based on radial magnetization. Background Technology

[0002] Human hands can recognize texture, shape, and size, enabling precise tactile feedback and everyday operations. They can also distinguish between normal and shear forces and achieve ultra-high resolution perception. Tactile feedback allows for dynamic adjustment of finger posture and contact force. In recent years, various tactile sensors based on different sensing principles have been proposed, including capacitive, resistive, piezoelectric, magnetoelectric, and optical sensors. Optical tactile sensors can achieve super-resolution comparable to skin, but their structures are relatively large, and most can only sense normal forces. With the development of materials science, shear force measurement can be achieved using electrodes made from novel materials. However, there is a coupling relationship between normal and shear forces. Although a decoupling model of normal and shear forces can be established through calibration experiments, its practical application still presents challenges. First, calibration experiments are required, and second, the decoupling model needs to be fitted, making the entire process very complex and tedious. Moreover, existing three-dimensional force sensors cannot achieve multimodal data sensing.

[0003] In summary, the current challenges of tactile sensors are: firstly, their resolution is low and cannot reach the resolution of human hand skin; secondly, there is coupling between normal force and shear force; and thirdly, they cannot achieve multimodal sensing. Summary of the Invention

[0004] To address the problems existing in the prior art, the purpose of this invention is to provide a high-resolution, radially magnetized, self-decoupled three-dimensional force multimodal sensing device, system, and method.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0006] A self-decoupled three-dimensional force multimodal sensing device based on radial magnetization includes a flexible magnetic film with radial magnetization, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB board, a conventional camera, and an infrared camera. The flexible magnetic film, microstructure layer, optical waveguide layer, and shell are stacked sequentially from top to bottom. The PCB board, conventional camera, and infrared camera are all located on the bottom surface of the shell, and the Hall element is mounted on the PCB board.

[0007] Furthermore, the centers of the flexible magnetic film, microstructure layer, and optical waveguide layer are located on the same vertical line as the center of the Hall element.

[0008] Furthermore, the conventional camera and the infrared camera are symmetrically placed on both sides of the Hall element.

[0009] Furthermore, the flexible magnetic film is made of PDMS and NdFeB magnetic powder.

[0010] Furthermore, the microstructure layer is made of silicone rubber and is shaped like an inverted pyramid.

[0011] Furthermore, the material of the optical waveguide layer is PMMA.

[0012] A self-decoupled three-dimensional force multimodal sensing system based on radial magnetization includes the aforementioned self-decoupled three-dimensional force multimodal sensing device and a signal processing device, wherein the signal processing device specifically includes:

[0013] The data acquisition module is used to acquire leaked spot photos taken by ordinary cameras and infrared spot photos taken by infrared cameras;

[0014] The force position acquisition module is used to calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera. The current position of the center point of the leaked spot on the imaging plane of the ordinary camera is substituted into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position.

[0015] The infrared distance sensing module is used to calculate the size of the infrared spot in the infrared spot photo, and substitute the current size of the infrared spot into the pre-stored relationship curve between the size of the infrared spot and the infrared sensing distance to obtain the current infrared sensing distance.

[0016] The three-dimensional force decoupling module is used to decouple three-dimensional forces according to the following formula:

[0017] In the formula, F x F y F z Let S represent the force in the x, y, and z directions of the three-dimensional force, respectively; S represent the contact area; G represent the shear modulus; E represent the elastic modulus; h represent the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board; k = 2π / T represents the wave number; T is the period; a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients; R xz (x i ), R xz (x0) represent the deformation of the flexible magnetic membrane under the action of force in the x direction to x. i R is the ratio of the magnetic flux density in the x-direction to the magnetic flux density in the z-direction when the object is in a certain position and is not under force. yz (y i ), R yz (y0) represents the deformation of the flexible magnetic film under the action of force in the y direction to the y direction. iThe ratio of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction when the object is in its current position and is not under force, B(z) i B(z0) and B(z0) represent the deformation of the flexible magnetic film to z0 under the action of force in the z direction, respectively. i The total magnetic flux density at position and without force, where (x0, y0, z0) represents the original coordinates of the center point of the flexible magnetic membrane when it is not under force. i ,y i ,z i () represents the coordinates of the center point of the flexible magnetic membrane after it has been deformed by a three-dimensional force.

[0018] Furthermore, the relationship curve between the center point of the light spot and the position of the force application is obtained in the following way:

[0019] A coordinate system x′y′ is established with the center of the upper surface of the flexible magnetic film as the origin;

[0020] Establish a coordinate system x″′y″′ with the center of the imaging plane of a regular camera as the origin;

[0021] Calculate the coordinate transformation curve between coordinate system x″′y″′ and coordinate system x′y′, and use it as the relationship curve between the position of the center point of the light spot and the position of the force loading.

[0022] Furthermore, the relationship curve between the infrared spot size and the infrared sensing distance is obtained in the following way:

[0023] The same infrared sensing object was placed at different distances from the upper surface of the flexible magnetic film, and infrared spot photographs were obtained when the infrared sensing object was placed.

[0024] Calculate the size of the infrared spot in infrared spot photos at different distances;

[0025] By fitting the infrared spot size in infrared spot photos at different distances, a curve showing the relationship between the infrared spot size and the infrared sensing distance is obtained.

[0026] A self-decoupled three-dimensional force multimodal sensing method based on radial magnetization, the method being implemented using the aforementioned self-decoupled three-dimensional force multimodal sensing device, the method comprising:

[0027] Obtain leaked light spot photos taken by ordinary cameras and infrared light spot photos taken by infrared cameras;

[0028] Calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera, and substitute the current position of the center point of the leaked spot on the imaging plane of the ordinary camera into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position.

[0029] Calculate the size of the infrared spot in the infrared spot image, substitute the current size of the infrared spot into the pre-stored relationship curve between the size of the infrared spot and the infrared sensing distance, and obtain the current infrared sensing distance;

[0030] The three-dimensional forces are decoupled according to the following formula:

[0031] In the formula, F x F y F z Let S represent the force in the x, y, and z directions of the three-dimensional force, respectively; S represent the contact area; G represent the shear modulus; E represent the elastic modulus; h represent the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board; k = 2π / T represents the wave number; T is the period; a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients; R xz (x i ), R xz (x0) represent the deformation of the flexible magnetic membrane under the action of force in the x direction to x. i R is the ratio of the magnetic flux density in the x-direction to the magnetic flux density in the z-direction when the object is in a certain position and is not under force. yz (y i ), R yz (y0) represents the deformation of the flexible magnetic film under the action of force in the y direction to the y direction. i The ratio of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction when the object is in its current position and is not under force, B(z) i B(z0) and B(z0) represent the deformation of the flexible magnetic film to z0 under the action of force in the z direction, respectively. i The total magnetic flux density at position and without force, where (x0, y0, z0) represents the original coordinates of the center point of the flexible magnetic membrane when it is not under force. i ,y i ,z i () represents the coordinates of the center point of the flexible magnetic membrane after it has been deformed by a three-dimensional force.

[0032] Compared with the prior art, the beneficial effects of this invention are:

[0033] 1. In this invention, the normal force and shear force are self-decoupled through a radially magnetized flexible magnetic film, eliminating the need for complex calibration experiments to fit the decoupling model. The decoupling of the normal force and shear force is achieved by establishing a mathematical model relating the magnetic flux density on the magnetic film to the normal force and shear force.

[0034] 2. In this invention, the sensing device can realize three-dimensional force measurement. When a normal force is applied, the surface of the flexible magnetic film will undergo local deformation, and when a tangential force is applied, the flexible magnetic film will undergo tangential displacement.

[0035] 3. In this invention, different measurement ranges and sensitivities can also be achieved by changing the thickness of the microstructure layer with different elastic moduli and the flexible magnetic film.

[0036] 4. In this invention, the location of the applied force is detected based on the total internal reflection method. An infrared camera is used to measure the infrared distance to the human hand.

[0037] 5. In this invention, multimodal measurement of three-dimensional force, force position, and infrared distance can be realized.

[0038] 6. In this invention, different measurement units are used for multimodal measurement, thus eliminating the need for complex multimodal decoupling algorithms, resulting in a simple structure and wide range of applications. Attached Figure Description

[0039] Figure 1 is a schematic diagram of the structure of the self-decoupling three-dimensional force multimodal sensing device based on radial magnetization provided in an embodiment of the present invention;

[0040] Figure 2 is a top view of the radially magnetized flexible magnetic film of the present invention;

[0041] Figure 3 is a schematic diagram of the magnetic field of the flexible magnetic film of the present invention;

[0042] Figure 4 is a schematic diagram of the normal force applied to the self-decoupled three-dimensional force multimodal sensing device in this invention;

[0043] Figure 5 is a schematic diagram of the application of shear force to the self-decoupled three-dimensional force multimodal sensing device in this invention;

[0044] Figure 6 is a schematic diagram of total internal reflection within the optical waveguide layer of the present invention;

[0045] Figure 7 is a schematic diagram of the force position detection of the present invention;

[0046] Figure 8 is a schematic diagram of the coordinate transformation of the present invention;

[0047] Figure 9 is a schematic diagram of infrared distance measurement according to the present invention.

[0048] Figure reference numerals: 1-Flexible magnetic film, 2-Microstructure layer, 3-Optical waveguide layer, 4-Shell, 5-LED light, 6-Ordinary camera, 7-PCB, 8-Infrared camera, 9-Signal processing device, 1-1: S pole, 1-2: N pole, 1-3: Magnetic field lines, 1-4: Upper surface of flexible magnetic film, 3-1: Lower surface of optical waveguide layer; 5-1: Light ray, 5-2: Leaked light ray, 6-1: Leaked light spot, 6-2: Imaging surface, 8-1: Infrared light spot. Detailed Implementation

[0049] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0050] Example 1

[0051] This invention provides a self-decoupled three-dimensional force multimodal sensing device based on radial magnetization, as shown in Figure 1. It includes a radially magnetized flexible magnetic film 1, a microstructure layer 2, an optical waveguide layer 3, a housing 4, a conventional camera 6, a Hall element (not shown), a PCB board 7, and an infrared camera 8. The flexible magnetic film 1, microstructure layer 2, optical waveguide layer 3, and housing 4 are stacked sequentially from top to bottom. The PCB board 7, conventional camera 6, and infrared camera 8 are all located on the bottom surface of the housing 4. The Hall element is mounted on the PCB board 7. An LED lamp 5 serves as the light source, and the light emitted by the LED lamp 5 undergoes total internal reflection within the optical waveguide layer 3. The centers of the flexible magnetic film 1, microstructure layer 2, and optical waveguide layer 3 are aligned with the center of the Hall element on the same vertical line. The conventional camera 6 and infrared camera 8 are symmetrically placed on either side of the Hall element.

[0052] In practical implementation, the flexible magnetic film 1 can be made from a mixture of PDMS (polydimethylsiloxane) and NdFeB (neodymium iron boron) magnetic powder, with a thickness of 1 mm. After curing, the film is radially magnetized using a pulse magnetizer. The radially magnetized flexible magnetic film 1 is shown in Figure 2, with S pole 1-1 and N pole 1-2 spaced out from the inside to the outside. The magnetic field lines 1-3 of the flexible magnetic film 1 are distributed as shown in Figure 3. The microstructure layer 2 is made of silicone rubber with a Shore A70 hardness, and is shaped like an inverted pyramid with a height of 1 mm. The optical waveguide layer 3 is made of PMMA (polymethyl methacrylate) with a thickness of 1 mm. The flexible film 1 and the microstructure layer 2 are assembled using an adhesive. The Hall element is a commercially available MLX90393 with adjustable sensitivity. To ensure consistent measurement ranges in the x and y directions, the center of the magnetic poles of the flexible magnetic film 1 needs to be aligned with the center of the Hall element. The outer casing 4 is made of stainless steel and is 5mm high. The ordinary camera 6 is used to detect the position of the force, and the infrared camera 8 is used to realize infrared distance measurement.

[0053] The working principle of this invention embodiment is as follows:

[0054] 1. The flexible magnetic film 1 is magnetized by radial magnetization. When an external force is applied to the flexible magnetic film 1, the flexible magnetic film 1 will undergo local deformation, and the magnetic flux density below it will change. The Hall element is used to measure the change, and the change is converted into an electrical signal through the PCB board 7. The magnitude and direction of the three-dimensional force can be obtained by analyzing and processing the electrical signal.

[0055] 2. After radial magnetization, the change in the total magnetic flux density B(z) of the flexible magnetic film 1 is only related to the force in the z direction, and the ratio R of the magnetic flux density in the x direction to that in the z direction is... xzThe change is only related to the force in the x-direction, and the ratio R of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction is... yz The change is only related to the force in the y direction;

[0056] Thirdly, when subjected to a force in the z-direction, the flexible magnetic membrane 1 generates a pit in the contact area. The magnetic flux density at the corresponding location changes due to local deformation, as shown in Figure 4. A mathematical model is established between the z-direction force, z-direction displacement, and total magnetic flux density B(z). When subjected to shear forces in the x or y direction, as shown in Figure 5, the flexible magnetic membrane 1 will generate corresponding x-direction or y-direction displacements, and the center position of the flexible magnetic membrane 1 will shift. The ratios R of the x-direction displacement and magnetic flux density are established respectively. xz The mathematical model between them and the ratio R of displacement and magnetic flux density in the y-direction. yz A mathematical model between them is used to achieve three-dimensional force decoupling;

[0057] 4. When the flexible magnetic film 1 is subjected to force, the force value is transmitted to the microstructure layer 2, thereby causing the microstructure layer 3 to come into contact with the optical waveguide layer 3. A light spot is generated in the contact area. The light spot is captured by a regular camera 6, and the correspondence between the center position of the light spot and the position of the force is established, which can realize the detection of the position of the force.

[0058] Fifth: When the infrared camera 8 is located at different positions in the z-direction of the flexible magnetic film 1, the images captured by the infrared camera 8 are different. By establishing the correspondence between the size of the infrared image and the different positions in the z-direction, infrared distance measurement can be realized.

[0059] Example 2

[0060] This invention provides a self-decoupled three-dimensional force multimodal sensing system based on radial magnetization, as shown in Figure 1. In addition to the self-decoupled three-dimensional force multimodal sensing device based on radial magnetization described in Embodiment 1, it also includes a signal processing device 9. The signal processing device 9 is connected to a conventional camera 6, a PCB board 7, and an infrared camera 8. Specifically, the signal processing device includes:

[0061] The data acquisition module is used to acquire leaked spot photos taken by ordinary cameras and infrared spot photos taken by infrared cameras;

[0062] The force position acquisition module is used to calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera. The current position of the center point of the leaked spot on the imaging plane of the ordinary camera is substituted into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position.

[0063] The infrared distance sensing module is used to calculate the size of the infrared spot in the infrared spot photo, and substitute the current size of the infrared spot into the pre-stored relationship curve between the size of the infrared spot and the infrared sensing distance to obtain the current infrared sensing distance.

[0064] The three-dimensional force decoupling module is used to decouple three-dimensional forces according to the following formula:

[0065] In the formula, F x F y F z Let S represent the force in the x, y, and z directions of the three-dimensional force, respectively; S represent the contact area; G represent the shear modulus; E represent the elastic modulus; h represent the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board; k = 2π / T represents the wave number; T is the period; a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients; R xz (x i ), R xz (x0) represent the deformation of the flexible magnetic membrane under the action of force in the x direction to x. i R is the ratio of the magnetic flux density in the x-direction to the magnetic flux density in the z-direction when the object is in a certain position and is not under force. yz (y i ), R yz (y0) represents the deformation of the flexible magnetic film under the action of force in the y direction to the y direction. i The ratio of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction when the object is in its current position and is not under force, B(z) i B(z0) and B(z0) represent the deformation of the flexible magnetic film to z0 under the action of force in the z direction, respectively. i The total magnetic flux density at position and without force, where (x0, y0, z0) represents the original coordinates of the center point of the flexible magnetic membrane when it is not under force. i ,y i ,z i () represents the coordinates of the center point of the flexible magnetic membrane after it has been deformed by a three-dimensional force.

[0066] The derivation process of the decoupling formula for the three-dimensional force decoupling module is as follows:

[0067] In the xy plane, the thickness of the flexible magnetic membrane 1 is d. Let the upper and lower surfaces of the flexible magnetic membrane 1 be z = 0 and z = d, respectively. The ideal flexible magnetic membrane 1 is radially magnetized by the superposition of two orthogonal sine curves in the xy plane. The magnetic field components in the x, y, and z directions are respectively: M x =M0sin(kx) M y =M0sin(ky) (1) M z =0

[0068] Where k = 2π / T is the wave number, T is the period, and M0 is the maximum amplitude of each component. x M y M z This represents the magnetic field components at coordinates (x, y, z).

[0069] The field in the plane, from the Poisson equation and Laplace equation above and below, is as follows:

[0070] in, The scalar potential representing the magnetic field above the flexible magnetic film. The scalar potential representing the internal magnetic field of the flexible magnetic film. This represents the scalar potential of the magnetic field beneath the flexible magnetic membrane.

[0071] The magnetic flux density B is proportional to the sum of the magnetic field H and the magnetization M. The formula for calculating the magnetic flux density B is: B=μ0(H+M) (3)

[0072] Where μ0 is the permeability of free space, with a value of 4π × 10⁻⁶. -7 H·m -1 Based on the fact that the magnetization outside the magnetic material is 0, the magnetic flux density below the flexible magnetic film 1 can be obtained as follows:

[0073] Among them, B x B y B z Let x, y, and z represent the magnetic flux density at coordinates (x, y, z) below the flexible magnetic membrane 1.

[0074] According to formula (4), calculate the magnetic flux density B(x,y,z) below the Halbach plane and the magnetic field ratio R. xz and R yz The calculation formula is:

[0075] When the displacement caused by the applied force is <1mm, equation (5) can be approximated as: B(x,y,z)=a0μ0M0(1-e kd )e kz +b0 R xz =a1tan(kx)+b1 (6) R yz = a2tan(ky) + b2

[0076] Where a0, a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients, obtained by fitting multiple data pairs. Then, according to equation (6), the expressions for x, y, and z can be obtained as follows:

[0077] When the flexible magnetic membrane 1 is subjected to an external force, the coordinates of its center point deform from (x0, y0, z0) to (x... i ,y i ,z i Based on the position, the forces F in the x, y, and z directions can be obtained according to Hooke's Law. x F y F z The calculation formula is:

[0078] Where S is the contact area, γ x γ y Let ε, Δx, Δy, Δz, G, E, and h represent the strain in the x-direction, strain in the y-direction, strain in the z-direction, deformation in the x-direction, deformation in the y-direction, deformation in the z-direction, shear modulus, elastic modulus, and the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, respectively. The expressions for shear modulus and elastic modulus are:

[0079] If υ is Poisson's ratio, then we can obtain:

[0080] Therefore, F x Only related to the deformation in the x-direction and R xz Related to F y Only the deformation in the y direction and R yz Related to F z It depends only on the deformation in the z-direction and the total magnetic flux density B(z). Given S, G, k, and h, this invention can achieve F. x F y F z Three-dimensional force decoupling.

[0081] As shown in Figure 6, when no external force is applied, the LED acts as the light source, and total internal reflection occurs inside the optical waveguide layer 3. The refractive index n1 of the optical waveguide layer is 1.49, and the refractive index n2 of air is 1. At point B, according to the law of refraction, we can obtain: n1sinθ i =n2sinθ t

[0082] Where, θ i Let θ be the angle of incidence. t When the angle of refraction is θ, t When the angle is 90°, the critical angle of incidence can be obtained as:

[0083] As shown in Figure 7, when a force is applied to the flexible magnetic film 1, the microstructure layer 2 deforms and comes into contact with the optical waveguide layer 3. Since the total internal reflection condition is not met in the contact area, leakage light will be generated at point D. A photo of the leakage light spot is obtained through a regular camera 6. The location of the applied force can be deduced from the center position of the leakage light spot 6-1.

[0084] As shown in Figure 8, assuming the center position of the upper surface 1-4 of the flexible magnetic film 1 is taken as the origin, a coordinate system x′y′ is established. Assuming the coordinates of the force loading point on the upper surface 1-4 of the flexible magnetic film 1 are (x′1, y′1), since the optical waveguide layer 3 has the same length and width as the flexible magnetic film 1 and the center is located on the same straight line, similarly, assuming the center position of the lower surface 3-1 of the optical waveguide layer 3 is taken as the origin, a coordinate system x″y″ is established. Then, leakage light will be generated at the corresponding position (x″1, y″1) of the optical waveguide layer 3, forming a leakage light spot on the ordinary camera 6. assuming the center position of the imaging surface 6-2 of the ordinary camera 6 is taken as the origin, a coordinate system x″′y″′ is established. By processing the image of the leakage light spot, the center point of the leakage light spot can be located, and the coordinates of the center point in the coordinate system x″′y″′ (x″′1, y″′1) can be obtained. Calculate the coordinate transformation curve between coordinate system x″′y″′ and coordinate system x′y′, and use it as the relationship curve f between the center point of the light spot and the position of the force loading point. Based on the curve, when the coordinates (x″′1, y″′1) are known, the coordinates (x′1, y′1) of the position of the force loading point can be obtained.

[0085] All objects with a temperature above absolute zero (-273.15℃) emit electromagnetic radiation; this phenomenon is called thermal radiation. The normal human body temperature is approximately 36-37℃ (about 300 Kelvin). According to the blackbody radiation law, at this temperature, the electromagnetic waves mainly radiated by the human body are in the infrared band (wavelength approximately 700nm to 1mm). When a human hand or other warm object approaches the upper surface of the flexible magnetic membrane 1, as shown in Figure 9(a), the radiated infrared light is captured by the infrared camera 8, as shown in Figure 9(b), which shows the infrared spot images of the hand at two different locations. By establishing a mathematical model between the size of the infrared spot and the distance, infrared distance measurement can be achieved. Specifically, the size of the infrared spot in the infrared spot images at different distances can be calculated; by fitting the infrared spot size in the infrared spot images at different distances, the relationship curve between the infrared spot size and the infrared sensing distance can be obtained. For example, if the distance between the human hand and the top of the flexible magnetic membrane is d1, and the diameter of the infrared spot is L, then the relationship curve G is: d1 = G(L).

[0086] It is worth noting that in the embodiments of the above system, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of the present invention.

[0087] The embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art will clearly understand that each implementation can be achieved using software plus necessary general-purpose hardware platforms, or it can be implemented solely through hardware, as long as the function or purpose can be achieved.

[0088] Example 3

[0089] This invention provides a self-decoupled three-dimensional force multimodal sensing method based on radial magnetization. This method is implemented based on the self-decoupled three-dimensional force multimodal sensing device described in Embodiment 1 above. The method includes:

[0090] Obtain leaked light spot photos taken by ordinary cameras and infrared light spot photos taken by infrared cameras;

[0091] Calculate the position of the center point of the leaked spot in the leaked spot photo on the imaging plane of the ordinary camera, and substitute the current position of the center point of the leaked spot on the imaging plane of the ordinary camera into the pre-stored relationship curve between the center point position of the spot and the force loading position to obtain the current force loading position.

[0092] Calculate the size of the infrared spot in the infrared spot image, substitute the current size of the infrared spot into the pre-stored relationship curve between the size of the infrared spot and the infrared sensing distance, and obtain the current infrared sensing distance;

[0093] The three-dimensional forces are decoupled according to the following formula:

[0094] In the formula, F x F y F z Let S represent the force in the x, y, and z directions of the three-dimensional force, respectively; S represent the contact area; G represent the shear modulus; E represent the elastic modulus; h represent the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board; k = 2π / T represents the wave number; T is the period; a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients; R xz (x i ), R xz(x0) represent the deformation of the flexible magnetic membrane under the action of force in the x direction to x. i R is the ratio of the magnetic flux density in the x-direction to the magnetic flux density in the z-direction when the object is in a certain position and is not under force. yz (y i ), R yz (y0) represents the deformation of the flexible magnetic film under the action of force in the y direction to the y direction. i The ratio of the magnetic flux density in the y-direction to the magnetic flux density in the z-direction when the object is in its current position and is not under force, B(z) i B(z0) and B(z0) represent the deformation of the flexible magnetic film to z0 under the action of force in the z direction, respectively. i The total magnetic flux density at position and without force, where (x0, y0, z0) represents the original coordinates of the center point of the flexible magnetic membrane when it is not under force. i ,y i ,z i () represents the coordinates of the center point of the flexible magnetic membrane after it has been deformed by a three-dimensional force.

[0095] This method corresponds one-to-one with the system in Embodiment 2. For details not covered, please refer to Embodiment 1.

[0096] It should be understood that the embodiments and descriptions above are only the principles, main features and advantages of the present invention. Various changes and modifications can be made to the present invention without departing from the spirit and scope of the invention, and all such changes and modifications fall within the protection scope of the present invention.

Claims

1. A self-decoupled three-dimensional force multi-modal sensing device based on radial magnetization, characterized by: The application relates to a self-decoupling three-dimensional force multi-modal sensing device, which comprises a flexible magnetic film with radial magnetization, a microstructure layer, an optical waveguide layer, a shell, a Hall element, a PCB board, a common camera and an infrared camera, wherein the flexible magnetic film, the microstructure layer and the optical waveguide layer are sequentially stacked from top to bottom, the PCB board, the common camera and the infrared camera are located on the bottom surface of the shell, and the Hall element is mounted on the PCB board.

2. The self-decoupled three-dimensional force multi-modal sensing device based on radial magnetization of claim 1, wherein: The center of the flexible magnetic film, the microstructure layer and the optical waveguide layer is located on the same vertical line with the center of the Hall element.

3. The self-decoupled three-dimensional force multi-modal sensing device based on radial magnetization of claim 1, wherein: The common camera and the infrared camera are symmetrically arranged on the two sides of the Hall element.

4. The self-decoupled three-dimensional force multi-modal sensing device based on radial magnetization of claim 1, wherein: The material of the flexible magnetic film is PDMS and NdFeB magnetic powder.

5. The self-decoupled three-dimensional force multi-modal sensing device based on radial magnetization of claim 1, wherein: The material of the microstructure layer is silicone rubber, and the shape is inverted pyramid type.

6. The self-decoupled three-dimensional force multi-modal sensing device based on radial magnetization of claim 1, wherein: The material of the optical waveguide layer is PMMA.

7. A self-decoupled three-dimensional force multi-modal sensing system based on radial magnetization, characterized by: The application further discloses a signal processing device of the self-decoupling three-dimensional force multi-modal sensing device. A data acquisition module is used for acquiring a leakage light spot photo taken by the common camera and an infrared light spot photo taken by the infrared camera. A force position acquisition module is used for calculating the position of a leakage light spot center point in the common camera imaging plane, and substituting the position of the current leakage light spot center point in the common camera imaging plane into a pre-stored relationship curve between the light spot center point position and the force loading position to obtain the current force loading position. An infrared distance sensing module is used for calculating the size of the infrared light spot in the infrared light spot photo, and substituting the size of the current infrared light spot into a pre-stored relationship curve between the infrared light spot size and the infrared sensing distance to obtain the current infrared sensing distance. A three-dimensional force decoupling module for decoupling three-dimensional forces according to the following formula: where F x , F y , F z represent the forces in the x, y, z directions of the three-dimensional force respectively, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB board, k = 2π / T represents the wave number, T is the period, a1, a2 and b0, b1, b2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x0) represent the ratios of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the flexible magnetic film is deformed to the x i position under the action of the force in the x direction and when it is not subjected to force respectively, R yz (y i ), R yz (y0) represent the ratios of the magnetic flux density in the y direction to the magnetic flux density in the z direction when the flexible magnetic film is deformed to the y i position under the action of the force in the y direction and when it is not subjected to force respectively, B(z i ), B(z0) represent the total magnetic flux density when the flexible magnetic film is deformed to the z i position under the action of the force in the z direction and when it is not subjected to force respectively, (x0, y0, z0) represents the original coordinate values of the center point of the flexible magnetic film when it is not subjected to force, (x i , y i , z i ) represents the coordinate values of the center point of the flexible magnetic film after it is deformed under the action of the three-dimensional force.

8. The self-decoupled three-dimensional force multi-modal sensing system based on radial magnetization of claim 7, wherein: The relationship curve between the light spot center point position and the force loading position is obtained by the following method: A coordinate system x'y' is established with the center position of the upper surface of the flexible magnetic film as the coordinate origin; A coordinate system x'''y''' is established with the center position of the common camera imaging plane as the coordinate origin; A coordinate conversion curve of the coordinate system x'''y''' and the coordinate system x'y' is calculated as the relationship curve between the light spot center point position and the force loading position.

9. The self-decoupled three-dimensional force multi-modal sensing system based on radial magnetization of claim 7, wherein: The relationship curve between the infrared light spot size and the infrared sensing distance is obtained by the following method: An infrared sensing object is placed at different distances from the upper surface of the flexible magnetic film to obtain infrared light spot photos when the infrared sensing object is placed; The sizes of the infrared light spots in the infrared light spot photos at different distances are calculated; The relationship curve between the infrared light spot size and the infrared sensing distance is obtained by fitting the sizes of the infrared light spots in the infrared light spot photos at different distances.

10. A self-decoupled three-dimensional force multi-modal sensing method based on radial magnetization, characterized in that: The method is realized based on the self-decoupling three-dimensional force multi-modal sensing device, and the method comprises the following steps: Acquiring a leakage light spot photo taken by the common camera and an infrared light spot photo taken by the infrared camera; Calculating the position of a leakage light spot center point in the common camera imaging plane, and substituting the position of the current leakage light spot center point in the common camera imaging plane into a pre-stored relationship curve between the light spot center point position and the force loading position to obtain the current force loading position; The size of the infrared light spot in the infrared light spot photo is calculated, and the size of the current infrared light spot is substituted into a relationship curve between the size of the infrared light spot and the infrared sensing distance to obtain the current infrared sensing distance; Decouple the three-dimensional force according to the following formula: where F x , F y , and F z represent forces in the x, y, and z directions of the three-dimensional force, S represents the contact area, G represents the shear modulus, E represents the elastic modulus, h represents the distance between the upper surface of the flexible magnetic film and the upper surface of the PCB, k = 2p / T represents the wave number, T is the period, a1, a2, and b0, b1, b2 are pre-calibrated fitting coefficients, R xz (x i ), R xz (x0) represent the ratio of the magnetic flux density in the x direction to the magnetic flux density in the z direction when the flexible magnetic film is deformed to the x i position under the action of the force in the x direction and when not subjected to force, respectively, R yz (y i ), R yz (y0) represent the ratio of the magnetic flux density in the y direction to the magnetic flux density in the z direction when the flexible magnetic film is deformed to the y i position under the action of the force in the y direction and when not subjected to force, respectively, B(z i ), B(z0) represent the total magnetic flux density when the flexible magnetic film is deformed to the z i position under the action of the force in the z direction and when not subjected to force, respectively, (x0, y0, z0) represents the original coordinate value of the center point of the flexible magnetic film when not subjected to force, (x i , y i , z i ) represents the coordinate value of the center point of the flexible magnetic film after deformation under the action of the three-dimensional force.

Images