On-Chip Permanent Magnet Positioning and Tracking System and Method

Abstract

On-chip permanent magnet positioning and tracking system, including an SOPC hardware system and a parameter calibration platform. The SOPC hardware system is integrated on an FPGA. The parameter calibration platform includes a magnetic sensor array flat plate and several magnet platforms. The magnetic sensor array flat plate is arranged with a sensor array, and the sensor array is composed of several three-axis magnetic sensors. The magnet platform is used to arrange permanent magnets, and the position of the magnet platform is such that the magnetic induction intensity of the permanent magnets arranged thereon can be detected by the sensor array. The present invention also provides an on-chip permanent magnet positioning and tracking method, which is realized based on this on-chip permanent magnet positioning and tracking system, including a parameter calibration link and a positioning and tracking link. The present invention can solve the problem that the signal is easily affected by occlusion during the optical tracking process, resulting in inaccurate positioning, and realize the positioning and tracking of in-vivo magnetic devices in medical operations.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical auxiliary instruments, and particularly relates to a chip-integrated permanent magnet positioning and tracking system. Background Art

[0002] In the operation of catheter-type medical devices, problems such as the catheter tip being easily stuck, coiled into a mass, or misinserted into other lumens often occur. At this time, the physician needs to continuously rotate the catheter, retract the catheter, or reinsert the catheter, and these operations are likely to cause lumen injury or perforation. To help physicians perform catheter operations more safely and conveniently in the lumen, researchers have developed magnetic navigation drive system devices for various aspects such as digestive endoscopy, bronchoscopy, heart diseases, and neurovascular diseases. By implanting a permanent magnet at the catheter tip and using a robotic arm with a magnet, remote teleoperation can achieve the delivery, withdrawal, and turning of the catheter, and only preoperative CT, ultrasound, etc. are required, without exposure to X-rays during the operation. Among them, to further achieve the automation and intelligence of the magnetic navigation drive system device, it is necessary to realize the pose feedback of the catheter tip.

[0003] In an existing technology, during surgery, the robot tracks the position of the surgical instrument through an optical tracker to obtain the position information of the surgical site. The biggest problem with using optical tracking is that it is easily affected by occlusion, and the signal often disappears due to being blocked by medical staff during the operation. Summary of the Invention

[0004] In order to overcome the above-mentioned disadvantages of the prior art, the purpose of the present invention is to provide a chip-integrated permanent magnet positioning and tracking system and method to solve the problem that the signal is easily affected by occlusion during optical tracking, resulting in inaccurate positioning, and to achieve the positioning and tracking of in-vivo magnetic devices during medical surgery.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] The chip-integrated permanent magnet positioning and tracking system includes an SOPC hardware system and a parameter calibration platform;

[0007] The SOPC hardware system is integrated on an FPGA and includes a program processing IP core, a system reset IP core, an AXI bus IP core, a UART IP core, and an LCD IP core;

[0008] The parameter calibration platform includes a magnetic sensor array flat plate and several magnet platforms; a sensor array is arranged on the magnetic sensor array flat plate, and the sensor array is composed of several three-axis magnetic sensors; the magnet platforms are used to arrange permanent magnets, and the positions of the magnet platforms are such that the magnetic induction intensity of the permanent magnets arranged thereon can be detected by the sensor array;

[0009] The IP core of the UART receives the magnetic induction intensity data detected by the sensor array, sends the magnetic induction intensity data to the program processing IP core through the AXI bus IP core. The program processing IP core calculates the permanent magnet pose data by using the differential evolution algorithm, sends it to the IP core of the UART through the AXI bus IP core, and sends the permanent magnet pose data to the LCD touch screen through the IP core of the LCD.

[0010] The present invention also provides a method for on-chip permanent magnet positioning and tracking, which is implemented based on the on-chip permanent magnet positioning and tracking system, and includes a parameter correction link and a positioning and tracking link;

[0011] The steps of the parameter correction link are as follows:

[0012] 11), establish the communication connections between the various parts in the SOPC hardware system and the communication connection between the SOPC hardware system and the sensor array;

[0013] 12), install the magnetic sensor array plate and the magnet platform;

[0014] 13), use a coordinate measuring machine to measure the three-dimensional coordinates of each three-axis magnetic sensor and the three-dimensional coordinates of each magnet platform;

[0015] 14), arrange the permanent magnets on each magnet platform in turn. When at the jth magnet platform, the magnetic field intensity measured by the ith three-axis magnetic sensor is B ij ;

[0016] 15), import the three-dimensional coordinates of each three-axis magnetic sensor, the three-dimensional coordinates of each magnet platform, and the magnetic field intensity measured by each three-axis magnetic sensor into the magnetic dipole mathematical model, and calculate the angle deviation θ i and the sensitivity coefficient K i ;

[0017] The steps of the positioning and tracking link are as follows:

[0018] 21), import the angle deviation and sensitivity coefficient of each three-axis magnetic sensor into the magnetic dipole mathematical model;

[0019] 22), calculate the position and attitude unit vector of the permanent magnet within the detection range by using the differential evolution algorithm.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] 1. By using the designed parameter correction platform, the coordinate measuring instrument directly measures the sensor position and calibrates the magnet pose, reducing the error of the calibration magnet.

[0022] 2. The differential evolution algorithm has a simple structure. Using this algorithm to solve the magnet pose is easy to implement, converges quickly, and has strong robustness.

[0023] 3. The system - on - a - programmable - chip has a flexible design method, can be trimmed, expanded, and upgraded, and has the function of in - system programmability for both software and hardware.

[0024] 4. It improves the electromagnetic interference and signal integrity of the system, and ensures the accuracy and real - time performance of the permanent magnet positioning and tracking. Brief Description of the Drawings

[0025] Figure 1 It is a schematic diagram of the structural function of the on - chip permanent magnet positioning and tracking system based on FPGA of the present invention.

[0026] Figure 2 It is a schematic diagram of the parameter calibration platform of the present invention.

[0027] Figure 3 It is a schematic diagram of the magnetic sensor array of the present invention.

[0028] Figure 4 It is a control flow chart of the on - chip controller based on FPGA of the present invention.

[0029] Figure 5 It is a control flow chart of the on - chip controller based on FPGA of the present invention entering the parameter calibration area.

[0030] Figure 6 It is a control flow chart of the on - chip controller based on FPGA of the present invention entering the positioning and tracking area. Detailed Embodiment

[0031] The following combines the drawings and embodiments to detail the implementation manner of the present invention.

[0032] Since magnetic positioning and tracking do not have the problem of line - of - sight occlusion, it is expected to replace the optical tracking described in the background technology and be applied in medical surgeries.

[0033] The on - chip permanent magnet positioning and tracking system of the present invention mainly includes an SOPC hardware system and a parameter calibration platform.

[0034] Among them, the SOPC hardware system is integrated on an FPGA. It is a programmable system - on - a - chip based on FPGA, and uses programmable logic technology to complete the main logic functions of magnetic positioning and tracking with a single FPGA chip.

[0035] Such as Figure 1As shown in the figure, the SOPC hardware system mainly includes a program processing IP core (Processing System), a system reset IP core (System_Reset), an AXI bus IP core (AXI Interconect Block), a UART IP core (AXI_UART), and an LCD IP core (AXI_LCD). The program processing IP core provides clock signals for the system reset IP core, the AXI bus IP core, and the UART IP core. The program processing IP core provides a reset signal for the system reset IP core, and the system reset IP core provides reset signals for the AXI bus IP core and the UART IP core.

[0036] As Figure 2 shown in the figure, the parameter calibration platform provides parameter calibration for the sensor, and mainly includes a magnetic sensor array flat plate 1 and several magnet platforms 2. The magnetic sensor array flat plate 1 is used to arrange the sensor array, and the sensor array of the present invention is composed of several three-axis magnetic sensors 6. The magnet platform 2 is used to arrange permanent magnets. The position of the magnet platform 2 is such that the magnetic induction intensity of the permanent magnets arranged thereon can be detected by the sensor array.

[0037] ]>According to this structure, the UART IP core receives the magnetic induction intensity data detected by the sensor array, sends the magnetic induction intensity data to the program processing IP core through the AXI bus IP core, the program processing IP core calculates the permanent magnet pose data by using the differential evolution algorithm, sends it to the UART IP core through the AXI bus IP core, and sends the permanent magnet pose data to the LCD touch screen through the LCD IP core.

[0038] In an embodiment of the present invention, the parameter calibration platform further includes an optical platform 3. A flat plate support clamp 5 and several support rods 4 are fixed on the optical platform 3. The flat plate support clamp 5 is used to clamp and fix the magnetic sensor array flat plate 1, and each magnet platform 2 is respectively installed on a support rod 4. Among them, the magnetic sensor array flat plate 1 is installed vertically, and its magnetic sensor installation surface faces the magnet platform 2. To ensure the positioning accuracy, in the present invention, the number of both the magnet platform 2 and the three-axis magnetic sensors 6 should be greater than 5. For the convenience of installation, the optical platform 3 is a horizontal flat plate with a large number of vertical installation holes, and the support rods 4 are used to be installed in different installation holes to facilitate the adjustment of their positions. The permanent magnet of the present invention is a magnet, and it can be installed and fixed on the magnet platform 2 through structures or components such as clamping parts and grooves. Correspondingly, the flat plate support clamp 5 of the present invention is an upward clamping structure, and the clamping structure is symmetric left and right, clamping and fixing the magnetic sensor array flat plate 1, so that the magnetic sensor array flat plate 1 is in a vertical state, and the surface where its sensors are located faces the permanent magnets installed on the magnet platform 2.

[0039] In an embodiment of the present invention, as Figure 3As shown, each three-axis magnetic sensor 6 is evenly distributed and pasted (or glued, etc.) on the magnetic sensor array flat plate 1, where the magnetic sensor array flat plate 1 can be a PCB board, an acrylic board, or the like. Figure 3 Numbers 1 to 16 in it show 16 three-axis magnetic sensors 6, and their installation method is surface mounting.

[0040] As Figure 4 shown, using the on-chip permanent magnet positioning and tracking system of the present invention, the process of positioning and tracking includes a parameter calibration link and a positioning and tracking link.

[0041] Among them, the parameter calibration link calculates the angle deviation θ i and the sensitivity coefficient K i of the i-th three-axis magnetic sensor. This link is executed in advance. After the parameter calibration is completed, generally, it is not necessary to perform parameter calibration again with the change of the usage scenario during subsequent positioning and tracking.

[0042] The process of the parameter calibration link is as follows:

[0043] 1), Establish the communication connections of each part in the SOPC hardware system and the communication connection between the SOPC hardware system and the sensor array.

[0044] 11), Connect the SOPC hardware system based on FPGA to the sensor array and the LCD touch screen respectively.

[0045] 12), Power on and start the FPGA hardware circuit configuration program on the SD card. Through the human-computer interaction interface on the LCD, configure the initialization state of the on-chip control system of the magnetic positioning and tracker to ensure that the SOPC hardware system establishes communication with the sensor array.

[0046] 2), Click the enter button in the parameter calibration area of the LCD touch screen, then enter the parameter calibration area, as Figure 5 :

[0047] 3), Install the magnetic sensor array flat plate 1 and the magnet platform 2.

[0048] 3.1), Use the optical platform 3 to clamp the magnetic sensor array flat plate 1 in the flat plate support clamp 5.

[0049] 3.2), Measure the three-dimensional coordinates of each three-axis magnetic sensor 6 and the three-dimensional coordinates of each magnet platform 2 on the three-coordinate measuring machine, and save the data.

[0050] The three-axis magnetic sensors 6 are patch-mounted (or glued, etc.) on the magnetic sensor array flat plate 1. Due to manufacturing and installation errors, position, angle, and sensitivity errors of the three-axis magnetic sensors 6 will occur. Therefore, parameter calibration is required first. Taking the geometric center of the sensor array composed of the three-axis magnetic sensors 6 as the coordinate origin, the X-axis is parallel to the horizontal lower edge of the magnetic sensor array flat plate 1, the Y-axis is parallel to the vertical lower edge of the magnetic sensor array flat plate 1, and the positive direction of the Z-axis is to the right, forming a system three-dimensional coordinate system.

[0051] 3.3) Remove environmental noise.

[0052] 4) Arrange permanent magnets on each magnet platform 2 in sequence. When at the j-th magnet platform 2, the magnetic field intensity measured by the i-th three-axis magnetic sensor 6 is B ij , and save the data. Among them, B ij is a vector, expressed as (B ijx , B ijy , B ijz ), B ix , B iy , B iz are the magnetic components of the X, Y, and Z axes respectively. For N magnet platforms 2, there are a total of 3 * N * M magnetic components of the magnetic field intensity measured by M three-axis magnetic sensors 6.

[0053] 5) Import the three-dimensional coordinates of each three-dimensional magnetic sensor 6 in the measured sensor array, the three-dimensional coordinates of each magnet platform 2, and the magnetic field intensity data of each three-axis magnetic sensor 6 into the magnetic dipole mathematical model, and calculate the angle deviation θ i and the sensitivity coefficient K i of the i-th three-axis magnetic sensor (6). θ i and K i are also three-dimensional vectors, which are (θ ix , θ iy , θ iz ) and (K ix , K iy , K iz ) respectively.

[0054] The expression of the magnetic dipole mathematical model is:

[0055]

[0056] Among them, H i represents the unit vector of the magnetic field direction of the permanent magnet, and H i = [rotx(θ ix ) · roty(θ iy ) · rotz(θ iz )] -1·(0, 1, 0), rotx, roty, and rotz are the rotation matrices around the X, Y, and Z axes of the local coordinate system of the three-axis magnetic sensor 6 itself. The axes of the local coordinate system of the three-axis magnetic sensor 6 are parallel to the axes of the global coordinate system. P ij is the vector difference between the spatial position coordinates (x i , y i , z i ) of the i-th three-axis magnetic sensor 6 and the position coordinates (a j , b j , c j ) of the j-th permanent magnet, and R ij is the modulus of the spatial position of the i-th three-axis magnetic sensor 6 relative to the position vector of the j-th permanent magnet; B V is a constant characterizing the magnetic field strength of the magnetic source, μ r is the relative permeability (in air), μ0 is the permeability of free space, and M T is the magnetization intensity.

[0057] 6) Solve the above model by the least squares method to calculate the angular deviation θ i of the three-axis magnetic sensor and the sensitivity coefficient K i , and save the data. The magnetic sensor array plate 1 can be removed from the plate support clamp 5. Place it at a suitable position within the magnetic field range of the permanent magnet to be tracked as needed.

[0058] 7) Click the exit button in the parameter correction area of the LCD touch screen to exit the parameter correction area and enter the main interface, completing the parameter correction process.

[0059] Click the positioning and tracking area of the LCD touch screen, then enter the positioning and tracking process, and start positioning and tracking the magnet. For example, Figure 6 , the process of the positioning and tracking process is as follows:

[0060] 1) Import the angular deviation and sensitivity coefficient of the three-axis magnetic sensor obtained in the parameter correction process.

[0061] 2) Import the angular deviation θ i and the sensitivity coefficient K i of the three-axis magnetic sensor into the magnetic dipole mathematical model. The model formula is as follows:

[0062]

[0063] where H i ’ = [rotx(θ ix ) · roty(θ iy ) · rotz(θ iz )] -1 · (m, n, p), and (m, n, p) is the unit vector of the permanent magnet attitude.

[0064] 3) Click the start positioning and tracking button. Solve the mathematical model of the magnetic dipole by the differential evolution algorithm to obtain the position (a', b', c') of the permanent magnet and the attitude unit vector (m, n, p). If the previous steps have been executed before, this step can be directly executed. If not, a message window will pop up, showing "Model calibration parameters not input".

[0065] 4.4) Click the button in the positioning and tracking area of the LCD touch screen to exit the positioning and tracking area and enter the main interface.

[0066] Based on the principles of the above system and method, the present invention can be used in medical or non-medical surgeries. Specifically, when the permanent magnet is inside the body and there is an iron device outside the body, the iron device can cooperate with the permanent magnet to play a role in positioning and guiding. In order to position and track the permanent magnet inside the body, the calibrated magnetic sensor array plate 1 is fixed at an appropriate position within the magnetic field range of the permanent magnet and is communicatively connected to the SOPC hardware system, then the permanent magnet inside the body can be positioned and tracked. Among them, the algorithm runs in the program processing IP core, and the processing result is output to the LCD touch screen for display through the LCD IP core. Among them, through high-resolution CT, three-dimensional reconstruction and registration of the human body are carried out, and the position of the magnet and the three-dimensional reconstruction model of the human body can be displayed together on the LCD screen.

[0067] For example, in magnetic navigation endotracheal intubation, three-dimensional reconstruction of the respiratory tract can be carried out through high-resolution CT of the head and neck. Utilizing the anatomical feature that the position of the thyroid cartilage and the glottis is relatively fixed, a positioning magnet is placed at the thyroid cartilage, and the position of the positioning magnet is sensed by the magnetic induction sensor array. Through the positional relationship among the positioning magnet, the thyroid cartilage, and the glottis, the position of the glottis can be obtained, and then automatic intubation can be performed.

Claims

1. A method for on-chip permanent magnet positioning and tracking, which is implemented based on an on-chip permanent magnet positioning and tracking system. The on-chip permanent magnet positioning and tracking system includes an SOPC hardware system and a parameter calibration platform; The SOPC hardware system is integrated on an FPGA and includes a program processing IP core, a system reset IP core, an AXI bus IP core, a UART IP core, and an LCD IP core; The parameter calibration platform includes a magnetic sensor array flat plate (1) and several magnet platforms (2); a sensor array is arranged on the magnetic sensor array flat plate (1), and the sensor array is composed of several three-axis magnetic sensors (6); the magnet platform (2) is used to arrange permanent magnets, and the position of the magnet platform (2) is such that the magnetic induction intensity of the permanent magnets arranged thereon can be detected by the sensor array; The UART IP core receives the magnetic induction intensity data detected by the sensor array, sends the magnetic induction intensity data to the program processing IP core through the AXI bus IP core, the program processing IP core uses the differential evolution algorithm to calculate the pose data of the permanent magnet, sends it to the UART IP core through the AXI bus IP core, and sends the permanent magnet pose data to the LCD touch screen through the LCD IP core; It is characterized in that The positioning and tracking method includes a parameter calibration link and a positioning and tracking link; The steps of the parameter calibration link are as follows: 11), Establish communication connections between various parts of the SOPC hardware system and the communication connection between the SOPC hardware system and the sensor array; 12), Install the magnetic sensor array flat plate (1) and the magnet platform (2); 13), Use a three-coordinate measuring machine to measure the three-dimensional coordinates of each three-axis magnetic sensor (6) and the three-dimensional coordinates of each magnet platform (2); 14), arrange the permanent magnets on each magnet platform (2) in sequence. When at the j-th magnet platform (2), the magnetic field intensity measured by the i-th three-axis magnetic sensor (6) is ; 15), import the three-dimensional coordinates of each of the three-axis magnetic sensors (6), the three-dimensional coordinates of each magnet platform (2), and the magnetic field strength measured by each three-axis magnetic sensor (6) into the magnetic dipole mathematical model, and solve for the angular deviation of the ith three-axis magnetic sensor (6) and the sensitivity coefficient ; the magnetic dipole mathematical model is expressed as follows: min Among them, is the unit vector indicating the magnetic field direction of the permanent magnet, , , , are the rotation matrices about the x, y, and z axes of the local coordinate system of the three-axis magnetic sensor (6) itself, and the axes of the local coordinate system of the three-axis magnetic sensor (6) are parallel to the axes; is the spatial position coordinate of the i-th three-axis magnetic sensor (6) and the position coordinate of the j-th permanent magnet is the vector difference, is the modulus of the vector of the spatial position of the i-th three-axis magnetic sensor (6) relative to the position vector of the j-th permanent magnet; is a constant characterizing the magnetic field strength of the magnetic source; Solving the mathematical model of the magnetic dipole by the least square method to obtain the angular deviation of the i-th three-axis magnetic sensor (6) and the sensitivity coefficient ; The steps of the positioning and tracking link are as follows: 21), Import the angle deviation and sensitivity coefficient of each three-axis magnetic sensor (6) into the magnetic dipole mathematical model; the magnetic dipole mathematical model is expressed as follows: min Among them , is the unit vector of the permanent magnet body attitude; 22), Use the differential evolution algorithm to calculate the position and attitude unit vector of the permanent magnet within the detection range.

2. The on-chip permanent magnet positioning and tracking method according to claim 1, wherein Taking the geometric center of the sensor array as the coordinate origin, the X-axis is parallel to the horizontal lower edge of the magnetic sensor array flat plate (1), the Y-axis is parallel to the vertical lower edge of the magnetic sensor array flat plate (1), and the positive direction of the Z-axis is to the right, forming a system three-dimensional coordinate system, and using this system three-dimensional coordinate system as the global coordinate system.

3. The on-chip permanent magnet positioning and tracking method according to claim 1, wherein The program processing IP core provides clock signals for the system reset IP core, the AXI bus IP core, and the UART IP core. The program processing IP core provides a reset signal for the system reset IP core, and the system reset IP core provides reset signals for the AXI bus IP core and the UART IP core.

4. The on-chip permanent magnet positioning and tracking method according to claim 1, characterized in that The parameter calibration platform further includes an optical platform (3). A flat plate support clamp (5) and several support rods (4) are fixed on the optical platform (3). The flat plate support clamp (5) is used to clamp and fix the magnetic sensor array flat plate (1), and each magnet platform (2) is respectively installed on a support rod (4).

5. The on-chip permanent magnet positioning and tracking method according to claim 1, characterized in that The magnetic sensor array flat plate (1) is installed vertically, and its magnetic sensor installation surface faces the magnet platform (2).

6. The on-chip permanent magnet positioning and tracking method according to claim 1, characterized in that, The number of the magnet platforms (2) and the number of the three-axis magnetic sensors (6) are both greater than 5.

Images