Non-contact force tactile reproduction system and method based on three-dimensional orthogonal background electromagnet

Abstract

The application discloses a non-contact force haptic reproduction system and method based on three-dimensional orthogonal background electromagnet, which realizes force haptic reproduction in a real operation space in a three-dimensional orthogonal background electromagnet through a three-axis fingertip electromagnet worn on a fingertip, wherein the fingertip electromagnet is composed of sub-modules, and the background electromagnet is composed of sub-modules; simulation obtains influence of force of the background electromagnet on the fingertip electromagnet, and obtains a mapping relationship of the force of the fingertip electromagnet to the background electromagnet excitation current in the whole three-dimensional operation space; three-dimensional position coordinates and rotation angles of a human finger are obtained by an optical method, and are sent into a force haptic interaction model to calculate target three-dimensional feedback force; based on offline simulation data and given three-dimensional position coordinates, rotation angles and target three-dimensional feedback force of the human finger, excitation currents of the background electromagnet and the fingertip electromagnet are finally determined.

Description

Technical Field

[0001] This invention belongs to the field of force and tactile reproduction technology in human-computer interaction, and particularly relates to a non-contact force and tactile reproduction system and method based on a three-dimensional orthogonal background electromagnet. Background Technology

[0002] Touch is essential for maintaining and enriching most of our daily activities, and it is the combined resources of the nervous system, musculoskeletal system, and skin (our largest sensory organ) that enable human touch perception. The sensory organ for touch is the skin, which, as the largest organ in the human body, accounts for approximately 15% of body weight.

[0003] In recent years, with the rapid development of force-tactile human-computer interaction technology, various force-tactile reproduction methods have been researched and developed in fields such as on-board education, medicine, and video games, leading to the invention of a wide variety of force-tactile reproduction devices. Early traditional wearable and force feedback lever-type force-tactile reproduction devices, due to their reliance on torque motors and mechanical structures, significantly affected the realism of the operator's force-tactile perception. Furthermore, they suffered from drawbacks such as limited effective operating space due to the mechanical linkage structure.

[0004] To overcome the shortcomings of traditional force-haptic reproduction devices and achieve more natural force-haptic reproduction, Yeongyu et al. designed WeHAPTIC, a wearable haptic device with finger position tracking. This device utilizes three Hall sensors and a motor encoder to measure fingertip position through forward kinematics and employs a PI controller combined with a robust control algorithm based on a disturbance observer to precisely control the motor current, thereby obtaining the desired force. Wang et al. developed a force feedback glove based on a soft bending actuator. The glove is made of silicone elastomer, and air is injected into the internal cavity of the elastomer to cause it to expand uniformly. Therefore, by adjusting the injected air pressure and through a linkage mechanism, the device allows the fingertips to feel a controllable feedback force.

[0005] The patent "Non-contact Force-Tactile Reproduction System and Method Based on Electromagnetic Field Combination Excitation Control" (application publication number: CN109145513A) mentions a force-tactile reproduction system whose stacked flat solenoid coils can generate a specific electromagnetic field within them. This system includes a permanent magnet module in the palm and multiple electromagnet modules at the fingertips to achieve force-tactile reproduction. However, its force-tactile reproduction dimension is limited, only achieving one-dimensional force-tactile reproduction.

[0006] Berkelman et al. designed a tactile interaction system based on rectangular flat coils. The core components of this system are a set of overlapping rectangular flat coils and a set of magnetometer sensors. During tactile interaction, the position of a permanent magnet worn on the finger is obtained through a Hall effect sensor array. However, the force it can generate is limited, and its accuracy needs further improvement.

[0007] Pedram et al. achieved accurate control of feedback force based on an electromagnet array system. They first designed a planar electromagnet array consisting of 27 closely arranged cylindrical electromagnets to generate a spatial magnetic field. Using force / torque sensors and a translation stage, they measured the axial force, radial force, and torque experienced by the permanent magnet at different positions within a 50mm range above the array. The measured data reflected the mapping relationship between the excitation current of the electromagnet array and the force on the permanent magnet. Combined with PD control, the system achieved accurate feedback of fingertip force / torque and was successfully applied to virtual texture reproduction. However, its effective operating space still needs further improvement. Summary of the Invention

[0008] To address the aforementioned technical problems, the present invention aims to propose a magnetic multi-point force tactile reproduction device and control method based on a three-dimensional orthogonal background electromagnet, thereby achieving force tactile feedback with a wider range of perceived force in three-dimensional space.

[0009] The present invention proposes a non-contact force tactile reproduction method based on a three-dimensional orthogonal background electromagnet, the method comprising:

[0010] Step 1: Establish a virtual scene including virtual hand and virtual object models, and establish the spatial mapping relationship between the real operating space inside the background electromagnet module and the virtual scene, as well as the three-dimensional positional mapping relationship between the human hand and the virtual hand.

[0011] Step 2: Using the finite element method, obtain the background magnetic field distribution of one of the background electromagnet submodules, and simulate the three-dimensional position coordinates, rotation angle, driving current of the fingertip electromagnet module and the mapping relationship of the driving current of the background electromagnet submodule on the force of each magnetic moment component of the fingertip electromagnet module, and obtain its offline simulation data.

[0012] The obtained offline simulation data is extended to other background electromagnet submodules, thereby obtaining offline simulation data for all background electromagnet submodules and fingertip electromagnet modules;

[0013] Step 3: Obtain finger position images to obtain real-time three-dimensional position and rotation angle information of the fingertip electromagnet worn by the person, determine the number of fingers, and determine whether it is single-point tactile reproduction or multi-point tactile reproduction.

[0014] Step four: The central control module calculates the target three-dimensional feedback force of the fingertip electromagnet module at the current moment based on the force-tactile interaction model of human fingers and virtual objects in a virtual scene.

[0015] Step 5: Based on the offline simulation data and the given three-dimensional position coordinates, rotation angle, and target three-dimensional feedback force of the fingertip electromagnet module, the force generation control method based on the principle of magnetic field superposition is used to obtain the driving current of the fingertip electromagnet module and the background electromagnet module.

[0016] Furthermore, in step one, the geometric center of the background electromagnetic field module is taken as the origin, the central axis of the two background electromagnet sub-modules connected end to end is taken as the X-axis, the central axis of any other background electromagnet sub-module in other directions is selected as the Y-axis, and the Z-axis is determined according to the right-hand rule to establish a spatial coordinate system.

[0017] The background electromagnet submodule is a hollow cylindrical coil formed by multiple layers of uniformly and densely wound circular coils. The six background electromagnet submodules EMx1, EMx2, EMy1, EMy2, EMz1, and EMz2 are independent units, each independently controlled, and each corresponding to one of the six drive currents I. x1 I x2 I y1 I y2 I z1 I z2 ;

[0018] The fingertip electromagnet module adopts a spherical structure in which three fingertip electromagnet sub-modules are symmetrically placed in three orthogonal directions. The fingertip electromagnet sub-modules in the x, y, and z directions of the fingertip electromagnet module numbered i are named EM respectively. FTix EM FTiy FM FTiz These correspond to three drive currents I, respectively. FTix I FTiy I FTiz The maximum safe current of the submodule is I. FTmax The fingertip electromagnet submodule consists of two circular coils connected in series in the same direction, which overlap in space.

[0019] Furthermore, step two includes:

[0020] Step 2.1: Simulate the background electromagnet submodule EMx1, ensuring its current is within -I... max To I max The values ​​increase sequentially, ranging from 0 to L+2R1 on the x-axis, from -R1 to R1 on the y-axis, where R1 is the inner diameter of the background electromagnet, L is the height of the 3D background electromagnet module, and from -R1 to R1 on the z-axis to represent the simulation space, where R1 is the inner diameter length of the sub-module of the back electromagnet module.

[0021] Using the finite element method, the background magnetic field distribution within this space is obtained, and the force situation of the fingertip electromagnet submodule EMft1x is calculated. The specific relationship is expressed as follows:

[0022] F = f(P) 1x P 1y P 1z , α1, β1, γ1, I x1 I FT1x I FT1y I FT1z )

[0023] Where F represents the force acting on EMft1x, (P 1x P 1y P 1z (α1, β1, γ1) represent the three-dimensional position coordinates and rotation angle of EMft1x in space, respectively;

[0024] Step 2.2: Extend the obtained offline simulation data of EMx1 on EMft1x to EMft1y and EMft1z to obtain the offline simulation data of EMx1 on the entire fingertip electromagnet EMft1.

[0025] Step 2.3: Extend the obtained offline simulation data of EMx1 to the fingertip electromagnet EMft1 to the other background electromagnets EMx2, Emy1...EMz2, so as to obtain the offline simulation data of the entire three-dimensional background electromagnet to EMft1 and all other background electromagnet sub-modules, thus obtaining the offline simulation data of the entire background electromagnet module.

[0026] Furthermore, step three includes:

[0027] The camera module captures the current moment of the fingertip electromagnet EM. FT Pose information P(P) x P y P z , α, β, γ), where P x P y and P z α represents the three-dimensional coordinates of the fingertip electromagnet, and α, β, and γ represent the rotation angles of the fingertip electromagnet.

[0028] Furthermore, the specific method for single-point tactile reproduction control is as follows:

[0029] The control methods for the background electromagnet submodules EMx1 and EMx2 are as follows: Based on the fingertip electromagnet EM... FT1 The coordinate P on the X-axis x1In EMx1 and EMx2, select the distance |P| between the center point of the fingertip electromagnet and the center point of the background electromagnet. x1 -P EMxi |, i = 1, 2, the smaller one applies the excitation current. When the distances are equal, EMx1 is selected to apply the excitation current. The selected background electromagnet is denoted as EMx_DR1, and its driving current is denoted as Ix_DR1. The other background electromagnet in the same direction is denoted as EMx_DR2, and its driving current is denoted as Ix_DR2. The force on the fingertip electromagnet in the x-direction is denoted as F1x, and the target force in the x-direction is denoted as F1x'. The maximum safe current I is applied to all three sub-modules of the fingertip electromagnet. FTmax Choose a suitable excitation current Ix_DR1 for EMx_DR1, with the current direction so that its magnetic moment points in the positive X-axis direction, so that the force F1x on the fingertip electromagnet in the X-direction is equal to the component F1x' of the target force in the X-direction; when Ix_DR1 is the maximum safe current Imax, if the amplitude of the force component |F1x| of the fingertip electromagnet is still less than the amplitude of the target force component |F1x'|, then apply a suitable current Ix_DR2 to EMx_DR2, with the current direction so that its magnetic moment points in the negative X-axis direction, so that the fingertip electromagnet meets the force requirements; when the maximum safe current Imax is applied to both EMx_DR1 and EMx_DR2, and the force requirements of the fingertip electromagnet are still not met, then the output F1x at this time is the saturation output value;

[0030] The driving currents Iy1, Iy2 and Iz1, Iz2 of the background electromagnets EMy1, EMy2 and EMz1, EMz2 are determined using the same method.

[0031] Furthermore, the multi-point tactile reproduction control method is as follows:

[0032] Multiple fingertip electromagnets (EMs) were captured using a camera module. FTi Let i = 1, 2, ..., n, where n is the number of reproduced points, and P be the pose information. i (Px i Py i Pz i α i ,β i γ i ), where Px i Py i and Pz i Let α be the three-dimensional coordinate information of the i-th fingertip electromagnet. i β i and γ i The rotation angle information of the i-th fingertip electromagnet is denoted as Isum_DRi;

[0033] The control methods for background electromagnets EMx1 and EMx2 are as follows: For each fingertip electromagnet, the control is sequentially based on its X coordinate Px... i In EMx1 and EMx2, select the value such that the distance between the center point of the fingertip electromagnet and the center point of the background electromagnet is |P xi -P EMxi |, i = 1, 2, apply the excitation current to the smaller one. If the distances are equal, select EMx1 to apply the excitation current. The selected background electromagnet is denoted as EMx_DR1, and its driving current is denoted as Ix_DR1. The other background electromagnet in the same direction is denoted as EMx_DR2, and its driving current is denoted as Ix_DR2. The force on the fingertip electromagnet in the x-direction is denoted as Fix, and the target force in the x-direction is denoted as Fix'. Apply the maximum safe current I to all three sub-modules of the fingertip electromagnet. FTmax Select an appropriate excitation current Ix_DR1 for EMx_DR1 such that the force Fix on the fingertip electromagnet in the X direction is equal to the component Fix' of the target force in the X direction. When Ix_DR1 is the maximum safe current Imax, if the amplitude of the force component |Fix| on the fingertip electromagnet is still less than the amplitude of the target force component |Fi'|, then apply an appropriate current Ix_DR2 to EMx_DR2 so that the fingertip electromagnet meets the force requirements. If the maximum safe current Imax is applied to both EMx_DR1 and EMx_DR2, and the force requirements of the fingertip electromagnet are still not met, then the current current configuration is set to saturation current output.

[0034] The absolute value of the total required current |Isum_DR| for each fingertip electromagnet is obtained in sequence. The required current for each fingertip electromagnet is sorted by size. The background electromagnet current configuration of the fingertip electromagnet with the largest absolute value of the total required current is output. The direction of the current of the fingertip electromagnet is adjusted so that its magnetic moment in the x direction is consistent with the direction of the magnetic moment of the background electromagnet.

[0035] The driving currents Iydri1, Iydri2, Izdri1, and Izdri2 of EMy1, EMy2, and EMz1 and EMz2 are obtained using the same method, and the currents of all fingertip electromagnets are adjusted until the forces (F) in all three directions of all fingertip electromagnets are equal. xn F yn F zn ) and the target force (F′) xn F′ yn F′ zn The error is less than the allowable error value.

[0036] This application also provides a non-contact force tactile reproduction system based on a three-dimensional orthogonal background electromagnet, the tactile reproduction system comprising a background electromagnet module, a fingertip electromagnet module, an electromagnet drive control module, a camera module, a power supply module and a central control module;

[0037] The background electromagnet module comprises 3 pairs of spatially orthogonal 6 background electromagnet sub-modules, with two electromagnet sub-modules in each direction;

[0038] The fingertip electromagnet module adopts a spherical structure in which three fingertip electromagnet sub-modules are symmetrically placed in three orthogonal directions;

[0039] The electromagnet drive control module includes an electromagnet control submodule and an electromagnet drive submodule. The electromagnet control submodule is used to receive control information from the central control module and generate PWM signals to control the electromagnet drive submodule to output the required electrical signals.

[0040] The central control module obtains three-dimensional feedback force based on the interaction between the fingertip position information and the virtual environment, and realizes the mapping from force to electromagnet excitation current based on the multi-point interactive haptic reproduction control algorithm.

[0041] Furthermore, a specific excitation current is applied to the background electromagnet module to generate the required background electromagnetic field. A spatial coordinate system is established by taking the geometric center of the background electromagnetic field module as the origin, the central axis of two connected background electromagnet sub-modules as the X-axis, the central axis of any other background electromagnet sub-module as the Y-axis, and determining the Z-axis according to the right-hand rule. Each background electromagnet sub-module is a hollow cylindrical coil formed by multiple layers of uniformly wound circular coils. The physical parameters of the six sub-modules are identical. The six background electromagnet sub-modules EMx1, EMx2, Emy1, Emy2, EMz1, and EMz2 are independent units, each independently controlled, corresponding to six driving currents I. x1 I x2 I y1 I y2 I z1 I z2 ;

[0042] The fingertip electromagnet module adopts a spherical structure in which three fingertip electromagnet sub-modules are symmetrically placed in three orthogonal directions. The fingertip electromagnet sub-modules in the x, y, and z directions of the fingertip electromagnet module numbered i are named EM respectively. FTix EM FTiy EM FTiz These correspond to three drive currents I, respectively. FTix I FTiy I FTiz The maximum safe current of the submodule is I.FTmax The fingertip electromagnet submodule consists of two circular coils connected in series in the same direction, which overlap in space.

[0043] The electromagnet drive control module consists of an electromagnet control submodule and an electromagnet drive submodule. The electromagnet control submodule is used to receive control information from the central control module and generate PWM signals to control the electromagnet drive submodule to output the required electrical signals.

[0044] The central control module obtains three-dimensional feedback force based on the interaction between the fingertip position information and the virtual environment, and realizes the mapping from force to electromagnet excitation current based on the multi-point interactive haptic reproduction control algorithm.

[0045] Furthermore, each sub-module of the aforementioned three-dimensional background electromagnet module has inner and outer diameters of R1 and R2, respectively, a height of L, and a maximum safe current of I. max The two background electromagnets in the x-direction are composed of sub-modules EMx1 and EMx2 connected end-to-end, with their axes coinciding with the x-axis. The two background electromagnet sub-modules EMy1 and EMy2 in the y-direction are symmetrically mounted on either side of the background electromagnet sub-module group in the x-direction, with their axes coinciding with the y-axis. The two background electromagnet sub-modules EMz1 and EMz2 in the z-direction are symmetrically mounted on either side of the background electromagnet sub-module group in the x-direction, with their axes coinciding with the z-axis. The center coordinates of the axes of the two background electromagnets in the x-direction are (P...). EMx1 ,0,0),(P EMx1 ,0,0).

[0046] Furthermore, the fingertip electromagnet module consists of three orthogonally placed cylindrical hollow coil sub-modules with their centers at the same point. Each sub-module is composed of two coils connected in series, and they overlap in space.

[0047] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0048] (1) The three-dimensional orthogonal background electromagnet module designed based on the principle of magnetic field superposition can generate the required background magnetic field in the operating space. Combined with the excitation current of multiple fingertip electromagnets, it can realize force tactile feedback with a larger range of force perception in three-dimensional space.

[0049] (2) The system equipment designed with this control method can achieve more refined, multi-point force and tactile perception, enhancing the realism and immersion of the user. Attached Figure Description

[0050] Figure 1 These are schematic diagrams of the background electromagnet and the fingertip electromagnet.

[0051] Figure 2 This is a schematic diagram of the structure of a magnetic multi-point force tactile reproduction system based on a three-dimensional orthogonal background electromagnet. Detailed Implementation

[0052] To make the control method and advantages of the present invention clearer, the control method of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0053] Example 1

[0054] This application provides a non-contact force tactile reproduction system based on a three-dimensional orthogonal background electromagnet. The tactile reproduction system includes a background electromagnet module, a fingertip electromagnet module, an electromagnet drive control module, a camera module, a power supply module, and a central control module.

[0055] The background electromagnet module comprises 3 pairs of spatially orthogonal 6 background electromagnet sub-modules, with two electromagnet sub-modules in each direction;

[0056] The fingertip electromagnet module adopts a spherical structure in which three fingertip electromagnet sub-modules are symmetrically placed in three orthogonal directions;

[0057] The electromagnet drive control module includes an electromagnet control submodule and an electromagnet drive submodule. The electromagnet control submodule is used to receive control information from the central control module and generate PWM signals to control the electromagnet drive submodule to output the required electrical signals.

[0058] The central control module obtains three-dimensional feedback force based on the interaction between the fingertip position information and the virtual environment, and realizes the mapping from force to electromagnet excitation current based on the multi-point interactive haptic reproduction control algorithm.

[0059] Furthermore, a specific excitation current is applied to the background electromagnet module to generate the required background electromagnetic field. A spatial coordinate system is established by taking the geometric center of the background electromagnetic field module as the origin, the central axis of two connected background electromagnet sub-modules as the X-axis, the central axis of any other background electromagnet sub-module as the Y-axis, and determining the Z-axis according to the right-hand rule. Each background electromagnet sub-module is a hollow cylindrical coil formed by multiple layers of uniformly wound circular coils. The physical parameters of the six sub-modules are identical. The six background electromagnet sub-modules EMx1, EMx2, Emy1, Emy2, EMz1, and EMz2 are independent units, each independently controlled, corresponding to six driving currents I. x1 I x2 I y1 I y2 Iz1 I z2 ;

[0060] The fingertip electromagnet module adopts a spherical structure in which three fingertip electromagnet sub-modules are symmetrically placed in three orthogonal directions. The fingertip electromagnet sub-modules in the x, y, and z directions of the fingertip electromagnet module numbered i are named EM respectively. FTix EM FTiy EM FTiz These correspond to three drive currents I, respectively. FTix I FTiy I FTiz The maximum safe current of the submodule is I. FTmax The fingertip electromagnet submodule consists of two circular coils connected in series in the same direction, which overlap in space.

[0061] The electromagnet drive control module consists of an electromagnet control submodule and an electromagnet drive submodule. The electromagnet control submodule is used to receive control information from the central control module and generate PWM signals to control the electromagnet drive submodule to output the required electrical signals.

[0062] The central control module obtains three-dimensional feedback force based on the interaction between the fingertip position information and the virtual environment, and realizes the mapping from force to electromagnet excitation current based on the multi-point interactive haptic reproduction control algorithm.

[0063] Furthermore, each sub-module of the aforementioned three-dimensional background electromagnet module has inner and outer diameters of R1 and R2, respectively, a height of L, and a maximum safe current of I. max The two background electromagnets in the x-direction are composed of sub-modules EMx1 and EMx2 connected end-to-end, with their axes coinciding with the x-axis. The two background electromagnet sub-modules EMy1 and EMy2 in the y-direction are symmetrically mounted on either side of the background electromagnet sub-module group in the x-direction, with their axes coinciding with the y-axis. The two background electromagnet sub-modules EMz1 and EMz2 in the z-direction are symmetrically mounted on either side of the background electromagnet sub-module group in the x-direction, with their axes coinciding with the z-axis. The center coordinates of the axes of the two background electromagnets in the x-direction are (P...). EMx1 ,0,0),(P EMx1 ,0,0).

[0064] Furthermore, the fingertip electromagnet module consists of three orthogonally placed cylindrical hollow coil sub-modules with their centers at the same point. Each sub-module is composed of two coils connected in series, and they overlap in space.

[0065] like Figure 1As shown, the magnetic multi-point force tactile reproduction system based on three-dimensional orthogonal background electromagnets includes a background electromagnet module 1, a fingertip electromagnet module 2, an electromagnet drive control module 3, a fingertip position tracking module 4, and a central control module 5.

[0066] The system consists of a background electromagnet module, a fingertip electromagnet module, an electromagnet drive control module, a fingertip position tracking module, a power supply module, and a central control module.

[0067] The three-dimensional background electromagnet module consists of three pairs of spatially orthogonal six background electromagnet sub-modules, with two sub-modules in each direction. A specific excitation current is passed through the background electromagnet module to generate the required background electromagnetic field.

[0068] The background electromagnet submodule is a hollow cylindrical coil composed of multiple layers of uniformly and densely wound circular coils. All six submodules have identical physical parameters. Each of the six background electromagnet submodules is an independent unit, controlled independently, and corresponds to one of the six drive currents.

[0069] The fingertip electromagnet module employs a spherical structure with three fingertip electromagnet sub-modules symmetrically placed in three orthogonal directions. When current is applied to the fingertip electromagnet, it experiences a force in the background magnetic field; the magnitude and direction of the applied current can alter the magnitude and direction of the force. The fingertip electromagnet sub-modules in the x-direction, y-direction, and z-direction are named as follows:

[0070] The fingertip electromagnet submodule consists of two identical circular coils located in the same direction.

[0071] The electromagnet drive control module consists of an electromagnet control submodule and an electromagnet drive submodule. The electromagnet control submodule is used to receive control information from the central control module and generate PWM signals to control the electromagnet drive submodule to output the required electrical signals.

[0072] The fingertip position tracking module is based on the LeapMotion motion sensor. Its main function is to accurately obtain the position information of the fingertip and transmit it to the central control module.

[0073] The central control module obtains three-dimensional feedback force based on the interaction between the fingertip position information and the virtual environment, and realizes the mapping from force to electromagnet excitation current based on the multi-point interactive haptic reproduction control algorithm.

[0074] Example 2

[0075] Based on the aforementioned non-contact force tactile reproduction system using a three-dimensional orthogonal background electromagnet, this application also provides a non-contact force tactile reproduction method using a three-dimensional orthogonal background electromagnet, the method comprising:

[0076] Step 1: Establish a virtual scene including virtual hand and virtual object models, and establish the spatial mapping relationship between the real operating space inside the background electromagnet module and the virtual scene, as well as the three-dimensional positional mapping relationship between the human hand and the virtual hand.

[0077] Step 2: Using the finite element method, obtain the background magnetic field distribution of one of the background electromagnet submodules, and simulate the three-dimensional position coordinates, rotation angle, driving current of the fingertip electromagnet module and the mapping relationship of the driving current of the background electromagnet submodule on the force of each magnetic moment component of the fingertip electromagnet module, and obtain its offline simulation data.

[0078] The obtained offline simulation data is extended to other background electromagnet submodules, thereby obtaining offline simulation data for all background electromagnet submodules and fingertip electromagnet modules;

[0079] Step 3: Obtain finger position images to obtain real-time three-dimensional position and rotation angle information of the fingertip electromagnet worn by the person, determine the number of fingers, and determine whether it is single-point tactile reproduction or multi-point tactile reproduction.

[0080] Step four: The central control module calculates the target three-dimensional feedback force of the fingertip electromagnet module at the current moment based on the force-tactile interaction model of human fingers and virtual objects in a virtual scene.

[0081] Step 5: Based on the offline simulation data and the given three-dimensional position coordinates, rotation angle, and target three-dimensional feedback force of the fingertip electromagnet module, the force generation control method based on the principle of magnetic field superposition is used to obtain the driving current of the fingertip electromagnet module and the background electromagnet module.

[0082] Furthermore, in step one, the geometric center of the background electromagnetic field module is taken as the origin, the central axis of the two background electromagnet sub-modules connected end to end is taken as the X-axis, the central axis of any other background electromagnet sub-module in other directions is selected as the Y-axis, and the Z-axis is determined according to the right-hand rule to establish a spatial coordinate system.

[0083] The background electromagnet submodule is a hollow cylindrical coil formed by multiple layers of uniformly and densely wound circular coils. The six background electromagnet submodules EMx1, EMx2, EMy1, EMy2, EMz1, and EMz2 are independent units, each independently controlled, and each corresponding to one of the six drive currents I. x1 I x2 I y1 I y2 I z1 Iz2 ;

[0084] The fingertip electromagnet module adopts a spherical structure in which three fingertip electromagnet sub-modules are symmetrically placed in three orthogonal directions. The fingertip electromagnet sub-modules in the x, y, and z directions of the fingertip electromagnet module numbered i are named EM respectively. FTix EM FTiy EM FTiz These correspond to three drive currents I, respectively. FTix I FTiy I FTiz The maximum safe current of the submodule is I. FTmax The fingertip electromagnet submodule consists of two circular coils connected in series in the same direction, which overlap in space.

[0085] Furthermore, step two includes:

[0086] Step 2.1: Simulate the background electromagnet submodule EMx1, ensuring its current is within -I... max To I max The values ​​increase sequentially, ranging from 0 to L+2R1 on the x-axis, from -R1 to R1 on the y-axis, where R1 is the inner diameter of the background electromagnet, L is the height of the 3D background electromagnet module, and from -R1 to R1 on the z-axis to represent the simulation space, where R1 is the inner diameter length of the sub-module of the back electromagnet module.

[0087] Using the finite element method, the background magnetic field distribution within this space is obtained, and the force situation of the fingertip electromagnet submodule EMft1x is calculated. The specific relationship is expressed as follows:

[0088] F = f(P) 1x P 1y P 1z , α1, β1, γ1, I x1 I FT1x I FT1y I FT1z )

[0089] Where F represents the force acting on EMft1x, (P 1x P 1y P 1z (α1, β1, γ1) represent the three-dimensional position coordinates and rotation angle of EMft1x in space, respectively;

[0090] Step 2.2: Extend the obtained offline simulation data of EMx1 on EMft1x to EMft1y and EMft1z to obtain the offline simulation data of EMx1 on the entire fingertip electromagnet EMft1.

[0091] Step 2.3: Extend the obtained offline simulation data of EMx1 to the fingertip electromagnet EMft1 to the other background electromagnets EMx2, Emy1...EMz2, so as to obtain the offline simulation data of the entire three-dimensional background electromagnet to EMft1 and all other background electromagnet sub-modules, thus obtaining the offline simulation data of the entire background electromagnet module.

[0092] Furthermore, step three includes:

[0093] The camera module captures the current moment of the fingertip electromagnet EM. FT Pose information P(P) x P y P z , α, β, γ), where P x P y and P z α represents the three-dimensional coordinates of the fingertip electromagnet, and α, β, and γ represent the rotation angles of the fingertip electromagnet.

[0094] Furthermore, the specific method for single-point tactile reproduction control is as follows:

[0095] The control methods for the background electromagnet submodules EMx1 and EMx2 are as follows: Based on the fingertip electromagnet EM... FT1 The coordinate P on the X-axis x1 In EMx1 and EMx2, select the distance |P| between the center point of the fingertip electromagnet and the center point of the background electromagnet. x1 -P EMxi |, i = 1, 2, the smaller one applies the excitation current. When the distances are equal, EMx1 is selected to apply the excitation current. The selected background electromagnet is denoted as EMx_DR1, and its driving current is denoted as Ix_DR1. The other background electromagnet in the same direction is denoted as EMx_DR2, and its driving current is denoted as Ix_DR2. The force on the fingertip electromagnet in the x-direction is denoted as F1x, and the target force in the x-direction is denoted as F1x'. The maximum safe current I is applied to all three sub-modules of the fingertip electromagnet. FTmax Choose a suitable excitation current Ix_DR1 for EMx_DR1, with the current direction so that its magnetic moment points in the positive X-axis direction, so that the force F1x on the fingertip electromagnet in the X-direction is equal to the component F1x' of the target force in the X-direction; when Ix_DR1 is the maximum safe current Imax, if the amplitude of the force component |F1x| of the fingertip electromagnet is still less than the amplitude of the target force component |F1x'|, then apply a suitable current Ix_DR2 to EMx_DR2, with the current direction so that its magnetic moment points in the negative X-axis direction, so that the fingertip electromagnet meets the force requirements; when the maximum safe current Imax is applied to both EMx_DR1 and EMx_DR2, and the force requirements of the fingertip electromagnet are still not met, then the output F1x at this time is the saturation output value;

[0096] The driving currents Iy1, Iy2 and Iz1, Iz2 of the background electromagnets EMy1, EMy2 and EMz1, EMz2 are determined using the same method.

[0097] Furthermore, the multi-point tactile reproduction control method is as follows:

[0098] Multiple fingertip electromagnets (EMs) were captured using a camera module. FTi Let i = 1, 2, ..., n, where n is the number of reproduced points, and P be the pose information. i (Px i Py i Pz i α i ,β i γ i ), where Px i Py i and Pz i Let α be the three-dimensional coordinate information of the i-th fingertip electromagnet. i β i and γ i The rotation angle information of the i-th fingertip electromagnet is denoted as Isum_DRi;

[0099] The control methods for background electromagnets EMx1 and EMx2 are as follows: For each fingertip electromagnet, the control is sequentially based on its X coordinate Px... i In EMx1 and EMx2, select the value such that the distance between the center point of the fingertip electromagnet and the center point of the background electromagnet is |P xi -P EMxi |, i = 1, 2, apply the excitation current to the smaller one. If the distances are equal, select EMx1 to apply the excitation current. The selected background electromagnet is denoted as EMx_DR1, and its driving current is denoted as Ix_DR1. The other background electromagnet in the same direction is denoted as EMx_DR2, and its driving current is denoted as Ix_DR2. The force on the fingertip electromagnet in the x-direction is denoted as Fix, and the target force in the x-direction is denoted as Fix'. Apply the maximum safe current I to all three sub-modules of the fingertip electromagnet. FTmaxSelect an appropriate excitation current Ix_DR1 for EMx_DR1 such that the force Fix on the fingertip electromagnet in the X direction is equal to the component Fix' of the target force in the X direction. When Ix_DR1 is the maximum safe current Imax, if the amplitude of the force component |Fix| on the fingertip electromagnet is still less than the amplitude of the target force component |Fi'|, then apply an appropriate current Ix_DR2 to EMx_DR2 so that the fingertip electromagnet meets the force requirements. If the maximum safe current Imax is applied to both EMx_DR1 and EMx_DR2, and the force requirements of the fingertip electromagnet are still not met, then the current current configuration is set to saturation current output.

[0100] The absolute value of the total required current |Isum_DR| for each fingertip electromagnet is obtained in sequence. The required current for each fingertip electromagnet is sorted by size. The background electromagnet current configuration of the fingertip electromagnet with the largest absolute value of the total required current is output. The direction of the current of the fingertip electromagnet is adjusted so that its magnetic moment in the x direction is consistent with the direction of the magnetic moment of the background electromagnet.

[0101] The driving currents Iydri1, Iydri2, Izdri1, and Izdri2 of EMy1, EMy2, and EMz1 and EMz2 are obtained using the same method, and the currents of all fingertip electromagnets are adjusted until the forces (F) in all three directions of all fingertip electromagnets are equal. xn F yn F zn ) and the target force (F′) xn F′ yn F′ zn The error is less than the allowable error value.

[0102] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0103] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A non-contact force tactile reproduction method based on three-dimensional orthogonal background electromagnet, characterized by, The method comprises: Step one, establishing a virtual scene including a virtual hand and a virtual object model, and establishing a spatial mapping relationship between a real operating space inside a background electromagnet module and the virtual scene, and a three-dimensional position mapping relationship between a human hand and the virtual hand; Step two, obtaining the background magnetic field distribution of one of the background electromagnet sub-modules using the finite element method, and simulating the mapping relationship between the three-dimensional position coordinates, rotation angle, driving current of the fingertip electromagnet module and the force of the driving current of the background electromagnet sub-module on each magnetic moment component of the fingertip electromagnet module, and obtaining the offline simulation data; The obtained offline simulation data is extended to other background electromagnet sub-modules, so as to obtain the offline simulation data of all background electromagnet sub-modules and fingertip electromagnet modules; Step three, obtaining a finger position image to obtain real-time three-dimensional position information and rotation angle information of the fingertip electromagnet worn by the human finger, determining the number of fingers, and determining whether single-point or multi-point haptic reproduction is performed; Step four, the central control module calculates the target three-dimensional feedback force of the fingertip electromagnet module based on the force haptic interaction model between the human finger and the virtual object in the virtual scene; Step five, according to the offline simulation data and the given three-dimensional position coordinates, rotation angle and target three-dimensional feedback force of the fingertip electromagnet module, a force generation control method is adopted based on the principle of magnetic field superposition to obtain the driving current of the fingertip electromagnet module and the background electromagnet module.

2. The non-contact force tactile reproduction method based on three-dimensional orthogonal background electromagnet according to claim 1, characterized by, In step one, the geometric center of the background electromagnet module is taken as the origin, the central axis of the two background electromagnet sub-modules connected head to tail is taken as the X axis, the central axis of an arbitrary other direction background electromagnet sub-module is taken as the Y axis, and the Z axis is determined according to the right-hand rule to establish a spatial coordinate system; The background electromagnet sub-module is a hollow cylindrical coil tightly wound by multiple layers of uniformly tightly wound circular coils. Six background electromagnet sub-modules EMx1, EMx2, EMy1, EMy2, EMz1, and EMz2 are independent units, each of which is independently controlled and corresponds to six driving currents I x1 , x2 , y1 , y2 , z1 , z2 ; The fingertip electromagnet module adopts a spherical structure with three fingertip electromagnet sub-modules symmetrically placed in three orthogonal directions, and the fingertip electromagnet sub-modules in x, y and z directions of the fingertip electromagnet module numbered i are respectively named as EM FTix , EM FTiy , EM FTiz , respectively corresponding to three driving currents I FTix , I FTiy , I FTiz , and the maximum safe current of the sub-module is I FTmax ; the fingertip electromagnet sub-module is composed of two circular coils in the same direction in series, and the two circular coils are overlapped in space.

3. The non-contact force tactile reproduction method based on three-dimensional orthogonal background electromagnet according to claim 2, characterized by, Step two comprises: Step 2.1: Simulate the background electromagnet submodule EMx1, ensuring its current is within -I... max To I max The values ​​increase sequentially, ranging from 0 to L+2R1 on the x-axis, from -R1 to R1 on the y-axis, where R1 is the inner diameter of the background electromagnet, L is the height of the 3D background electromagnet module, and -R1 to R1 on the z-axis represents the simulation space. Using finite element method, the distribution of background magnetic field in the simulation space is obtained, and the force of the electromagnetic finger sub-module EM FTix is calculated, which is specifically expressed as: F = f(P 1x , P 1y , P 1z , a1, b1, g1, Ix1, I FT1x , I FT1y , I FT1z ) where F represents the EM FT1x force (P 1x , P 1y , P 1z ) and (α1, β1, γ1) represent the EM FT1x three-dimensional position coordinates and rotation angles in space; Step 2.

2. Extend the offline simulation data of EMx1 pair EM FT1x to EM FT1y and EM FT1z , so as to obtain the offline simulation data of EMx1 pair entire fingertip electromagnet EM FT1 ; Step 2.

3. The offline simulation data of the pair of EMx1 finger electromagnets EM FT1 is extrapolated to the rest of the background electromagnets EMx2, EMy1...EMz2, thus obtaining the offline simulation data of the pair of EM FT1 The offline simulation data of the rest of the background electromagnet sub-modules is extrapolated, thus obtaining the offline simulation data of the entire background electromagnet module.

4. The non-contact force tactile reproduction method based on three-dimensional orthogonal background electromagnet according to claim 3, characterized by, Step three comprises: The camera module captures the current moment of the fingertip electromagnet EM. FT Pose information P(P) x ,P y ,P z ,α,β,γ), where P x P y and P z α represents the three-dimensional coordinates of the fingertip electromagnet, and α, β, and γ represent the rotation angles of the fingertip electromagnet.

5. The non-contact force tactile reproduction method based on three-dimensional orthogonal background electromagnet according to claim 4, characterized by, The single-point haptic reproduction control method specifically comprises: The control methods for the background electromagnet submodules EMx1 and EMx2 are as follows: Based on the fingertip electromagnet EM... FT1 The coordinate P on the X-axis x1 In EMx1 and EMx2, select the distance |P| between the center point of the fingertip electromagnet and the center point of the background electromagnet. x1 -P EMxi |, i = 1, 2, the smaller one applies the excitation current. When the distances are equal, EMx1 is selected to apply the excitation current. The selected background electromagnet is denoted as EMx_DR1, and its driving current is denoted as I. x_DR1 The other background electromagnet in the same direction is denoted as EMx_DR2, and its driving current is denoted as I. x_DR2 Let F1x be the force acting on the fingertip electromagnet in the x-direction, and F1x' be the target force acting on it in the x-direction. Apply the maximum safe current I to all three sub-modules of the fingertip electromagnet. FTmax Select a suitable EMx_DR1 excitation current I x_DR1 The direction of the current is such that its magnetic moment points in the positive X-axis direction, so that the force F1x on the fingertip electromagnet in the X direction is equal to the component F1x' of the target force in the X direction; when I x_DR1 For the maximum safe current I max If the amplitude of the force component |F1x| of the fingertip electromagnet is still less than the amplitude of the target force component |F1x'|, then a suitable current I is applied to EMx_DR2. x_DR2 The current direction is such that its magnetic moment points in the negative X-axis direction, ensuring the fingertip electromagnet meets the force requirements; when the maximum safe current I is applied to both EMx_DR1 and EMx_DR2... max If the force requirement of the fingertip electromagnet is still not met, then the output F1x at this time is the saturation output value. The driving currents I of the background electromagnets EMyi, EMy2 and EMzi, EMz2 are determined in the same way y1 , y2 and z1 , z2 .

6. The non-contact force tactile reproduction method based on three-dimensional orthogonal background electromagnet according to claim 5, characterized by, The multi-point haptic reproduction control method specifically comprises: Obtain the pose information P of the plurality of fingertip electromagnets EM through the camera module FTi i (P xi ,P yi ,P zi ,α i ,β i ,γ i ), i = 1, 2…n, n is the number of the reproduction points, wherein P xi , P yi and P zi are the three-dimensional coordinate information of the i-th fingertip electromagnet, α i , β i and γ i are the rotation angle information of the i-th fingertip electromagnet, and the total background electromagnet driving current required by the i-th fingertip electromagnet is denoted as I sun_DRi ;​ The control method of background electromagnet EMx1 and EMx2 is as follows: for each fingertip electromagnet, according to its X coordinate P xi , the distance between the fingertip electromagnet and the center point of the background electromagnet |P xi -P EMxi |, i=1, 2, the smaller one applies the exciting current, if the distance is equal, select EMx1 to apply the exciting current, the selected background electromagnet is recorded as EMx_DR1, and its driving current is recorded as I x_DR1 , while the other background electromagnet in the same direction is recorded as EMx_DR2, and its driving current is recorded as I x_DR2 ; the force on the fingertip electromagnet in the x direction is recorded as Fix, and the target force in the x direction is recorded as Fix'; the maximum safe current I FTmax is applied to the three sub-modules of the fingertip electromagnet, and the appropriate EMx_DR1 exciting current I x_DR1 is selected so that the force on the fingertip electromagnet in the x direction Fix is equal to the component of the target force in the x direction Fix'; when I x_DR1 is the maximum safe current I max , the amplitude of the fingertip electromagnet force component |Fix| is still less than the amplitude of the target force component |Fix'|, and the appropriate current I x_DR2 is applied to EMx_DR2 so that the fingertip electromagnet meets the force requirement; when the maximum safe current I max is applied to EMx_DR1 and EMx_DR2, the fingertip electromagnet still cannot meet the force requirement, and the current configuration at the moment is taken as the saturation current output. The absolute value I of the required current sum of each fingertip electromagnet is obtained in sequence sum_DR The required current of each fingertip electromagnet is sorted by size, the background electromagnet current configuration of the fingertip electromagnet with the maximum absolute value of the required current sum is output, and the current direction of the fingertip electromagnet is adjusted so that the magnetic moment of the fingertip electromagnet in the x direction is consistent with the direction of the magnetic moment of the background electromagnet. The driving currents I of EMy1, EMy2 and EMz1, EMz2 are obtained in the same way y_DR1 , y_DR2 , z_DR1 , z_DR2 , and the currents of all the finger electromagnets are adjusted until the errors of the forces (F xn , F yn , F zn ) in the three directions of all the finger electromagnets with respect to the target forces (F' xn , F' yn , F' zn ) are less than the allowed error value.

7. A tactile rendering system characterized by, The haptic reproduction system is used to implement the non-contact force haptic reproduction method based on three-dimensional orthogonal background electromagnets according to claim 1, and the haptic reproduction system comprises a background electromagnet module, a fingertip electromagnet module, an electromagnet driving control module, a camera module, a power module and a central control module; The background electromagnet module comprises 3 pairs of 6 background electromagnet sub-modules in space orthogonal to each other, and two electromagnet sub-modules in each direction; The fingertip electromagnet module adopts a spherical structure with 3 fingertip electromagnet sub-modules symmetrically placed in 3 orthogonal directions; The electromagnet driving control module comprises an electromagnet control submodule and an electromagnet driving submodule, the electromagnet control submodule is used to receive control information from the central control module, and generate a PWM signal to control the electromagnet driving submodule to output the required electrical signal; The central control module obtains three-dimensional feedback force according to the interaction between the fingertip position information and the virtual environment, and realizes the mapping from force to electromagnet excitation current according to the multi-point interaction haptic reproduction control algorithm. 8.The tactile rendering system according to claim 7, wherein, The background electromagnet module is connected to a specific excitation current to generate a required background electromagnetic field; a space coordinate system is established with the geometric center of the background electromagnet module as the origin, the central axis of the two background electromagnet sub-modules connected head to tail as the X axis, the central axis of an arbitrary other direction background electromagnet sub-module as the Y axis, and the Z axis determined according to the right-hand rule; the background electromagnet sub-module is a hollow cylindrical coil formed by densely winding a plurality of layers of uniformly dense circular coils, and six sub-modules have completely identical physical parameters; the six background electromagnet sub-modules EMx1, EMx2, EMy1, EMy2, EMz1, and EMz2 are independent units and are independently controlled, and correspond to six driving currents I x1 , I x2 , I y1 , I y2 , I z1 , I z2 ; The fingertip electromagnet module adopts a spherical structure with three fingertip electromagnet sub-modules symmetrically placed in three orthogonal directions, and the fingertip electromagnet sub-modules in x, y and z directions of the fingertip electromagnet module numbered i are respectively named as EM FTix , EM FTiy , EM FTiz , respectively corresponding to three driving currents I FTix , I FTiy , I FTiz , and the maximum safe current of the sub-module is I FTmax ; the fingertip electromagnet sub-module is composed of two circular coils in the same direction in an overlapped manner in space; The electromagnet driving control module is composed of an electromagnet control submodule and an electromagnet driving submodule; the electromagnet control submodule is used for receiving control information from the central control module and generating a PWM signal to control the electromagnet driving submodule to output a required electric signal; The central control module obtains three-dimensional feedback force according to the interaction between the fingertip position information and the virtual environment, and realizes the mapping from the force to the electromagnet excitation current according to a multi-point interactive tactile reproduction control algorithm. 9.The tactile rendering system according to claim 8, wherein, The inner and outer diameter length of each sub-module of the three-dimensional background electromagnet module is R1 and R2 respectively, the height is L, and the maximum safe current is I max ; two background electromagnets in x direction are combined by sub-modules EMx1 and EMx2 head to tail, the axis of which coincides with x axis; two background electromagnet sub-modules EMy1 and EMy2 are symmetrically installed on both sides of the background electromagnet sub-module group in x direction, the axis of which coincides with y axis; two background electromagnet sub-modules EMz1 and EMz2 are symmetrically installed on both sides of the background electromagnet sub-module group in x direction, the axis of which coincides with z axis; the axis center coordinates of the two background electromagnets in x direction are (P EMx1 ,0,0) and (P EMx1 ,0,0) respectively.

10. The tactile rendering system of claim 7, wherein, The fingertip electromagnet module is composed of three orthogonal cylindrical hollow coil submodules with the same center point, each of which is composed of two coils in series and overlaps in space.

Images