Precise positioning system based on multi-modal image fusion

Abstract

The invention discloses an accurate positioning system based on multi-modal image fusion, and the system comprises a multi-modal image collection module which synchronously collects an MRI structure image, an fMRI function image and a DTI white matter fiber bundle image of the brain of a patient; the image preprocessing module is connected with the acquisition module and is used for de-noising, registering and standardizing the multi-modal image data to generate fused image data under a unified coordinate system; the target analysis module is connected with the preprocessing module and calculates initial coordinates of the target based on the fused data; the dynamic correction module is connected with the analysis module, monitors the head movement in real time and dynamically corrects the target position; and the target spot guiding terminal is connected with the correction module and controls the mechanical arm to accurately position according to the corrected coordinates. According to the system, the transcranial magnetic stimulation target spot positioning accuracy and the personalized adaptive capacity can be remarkably improved, the problems that a traditional method depends on experience and the single-mode image accuracy is insufficient are solved, and meanwhile the positioning stability and the effect evaluability in treatment are ensured.

Description

Technical Field

[0001] This invention relates to the field of biomedical image pattern recognition technology, specifically to a precise positioning system based on multimodal image fusion. Background Technology

[0002] Transcranial magnetic stimulation (TMS), as a non-invasive neuromodulation technique, plays an increasingly important role in the treatment of neurological diseases and neuroscience research. However, in practical applications, the precise localization of stimulation targets remains a critical problem that urgently needs to be solved. Existing localization techniques based on single-modality imaging also face their own limitations. For example, relying solely on structural magnetic resonance imaging (MRI) to visualize TMS can identify the brain's functional activity areas, but each stimulation requires MRI imaging for localization, resulting in high costs and poor timeliness. While localization using electrophysiological signals such as electroencephalography (EEG) offers high temporal resolution, its spatial resolution is low, making it difficult to accurately determine the location of the stimulation target.

[0003] With the continuous development of neuroscience and medical imaging technology, multimodal imaging technology has gradually emerged, bringing new hope for solving the problem of transcranial magnetic stimulation (TMS) target localization. Multimodal image fusion technology can integrate the advantages of different imaging modalities. For example, structural magnetic resonance imaging (MRI) can clearly present the anatomical structure of the brain, functional magnetic resonance imaging (fMRI) can reflect the functional activity of the brain, and diffusion tensor imaging (DTI) can reveal the direction of white matter fiber tracts in the brain. Fusion analysis of this multimodal image information can provide more comprehensive and accurate information for TMS target localization, contributing to personalized and precise neuromodulation, and possessing significant clinical application value and research significance. However, how to efficiently and accurately apply multimodal imaging technology to TMS localization systems still faces many technical challenges. Summary of the Invention

[0004] The purpose of this invention is to provide a precise localization system based on multimodal image fusion. This invention can significantly improve the accuracy and personalized adaptability of transcranial magnetic stimulation target localization, solve the problems of reliance on experience and insufficient accuracy of single-modal images in traditional methods, and ensure the stability of localization and the assessability of effects during treatment.

[0005] The technical solution of this invention: A precise positioning system based on multimodal image fusion, comprising: The multimodal imaging acquisition module is used to simultaneously acquire MRI structural images, fMRI functional images, and DTI white matter fiber tract images of the patient's brain. The image preprocessing module, connected to the multimodal image acquisition module, is used to perform denoising, registration, and standardization processing on the acquired multimodal image data to generate fused image data in a unified coordinate system. The target analysis module, connected to the image preprocessing module, is used to calculate the initial coordinates of the target location based on the fused image data; The dynamic correction module, connected to the target analysis module, is used to monitor the patient's head movement in real time and dynamically correct the target position. The target guidance terminal is connected to the dynamic correction module and is used to control the robotic arm to perform precise positioning based on the corrected target coordinates.

[0006] The aforementioned precise positioning system based on multimodal image fusion includes a multimodal image acquisition module comprising: MRI equipment is used to obtain information about the anatomical structure of the brain; fMRI equipment is used to acquire information about areas of functional activity in the brain; DTI equipment is used to acquire information about the orientation of white matter fiber bundles.

[0007] The aforementioned precise positioning system based on multimodal image fusion includes an image preprocessing module comprising a denoising unit, a registration unit, and a normalization unit. The denoising unit uses a wavelet transform algorithm to remove high-frequency noise interference, the registration unit uses a mutual information-based registration algorithm to align image data from different modalities to the same coordinate system, and the normalization unit converts the registered image data into a representation in a standard brain template space.

[0008] In the aforementioned precise localization system based on multimodal image fusion, the target analysis module uses the following fusion algorithm to calculate the target location: ; in, , and These are the weighting coefficients, Indicates the anatomical structure of the brain. Indicates areas of brain activity. Indicates the orientation of white matter fiber bundles. This is the final target location.

[0009] The aforementioned precise positioning system based on multimodal image fusion, wherein the weighting coefficients , and Adjustments are made dynamically based on disease type, including: For depression, set =0.4, =0.5, =0.1; For Parkinson's disease, set =0.3, =0.4, =0.3; For Alzheimer's disease, set =0.5, =0.3, =0.2.

[0010] The aforementioned precision positioning system based on multimodal image fusion includes a dynamic correction module comprising a motion sensor and a correction unit. The motion sensor is used to monitor the head movement state in real time and generate a displacement vector; the correction unit is used to dynamically adjust the target coordinates according to the displacement vector.

[0011] The aforementioned precision positioning system based on multimodal image fusion includes a target guidance terminal comprising a robotic arm and a control system. The robotic arm moves to the target position according to the target positioning parameters, and the control system solves the angle parameters of each joint of the robotic arm through an inverse kinematics algorithm and verifies the actual position of the robotic arm through a built-in laser rangefinder and vision sensor.

[0012] The aforementioned precision positioning system based on multimodal image fusion also includes an effect evaluation and feedback module, which is connected to the target analysis module and is used to evaluate the stimulus effect and optimize the positioning strategy.

[0013] The aforementioned precision positioning system based on multimodal image fusion includes an effect evaluation and feedback module comprising an EEG signal acquisition unit and a behavioral testing unit. The EEG signal acquisition unit is used to record changes in the patient's electroencephalogram (EEG) signals after receiving stimulation, while the behavioral testing unit is used to design an evaluation plan and assess the effect based on the patient's specific condition.

[0014] In the aforementioned precision positioning system based on multimodal image fusion, the effect evaluation feedback module generates a quantitative effect evaluation report by statistically analyzing EEG signals and behavioral test data, and feeds the evaluation results back to the target analysis module to adjust the weighting coefficients or recalculate the target position.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By integrating multimodal image data, the problem of insufficient spatial or temporal resolution in traditional single-modal image localization methods is overcome, significantly improving the accuracy and personalized adaptability of target localization.

[0016] 2. During the target analysis process, a fusion algorithm based on multimodal image data is used, which can comprehensively reflect the brain structure and function information, thereby achieving more accurate target localization, and can be dynamically adjusted according to different disease types and clinical needs.

[0017] 3. By introducing a dynamic correction module, this invention solves the problem of target point displacement caused by patient head movement, ensuring the accuracy and stability of positioning during the stimulation process.

[0018] 4. This invention achieves quantitative evaluation and closed-loop optimization of stimulation effect through the effect evaluation feedback module, providing a scientific basis for subsequent treatment, and facilitating the accumulation of clinical data to further improve the positioning algorithm. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the structure of the multimodal image acquisition module of the present invention; Figure 2 This is a schematic diagram of image preprocessing; Figure 3 This is a schematic diagram of the target analysis process; Figure 4 This is a schematic diagram of the dynamic calibration process. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0021] Example: A precision localization system based on multimodal image fusion, the core of which lies in achieving high-precision localization of transcranial magnetic stimulation targets by integrating multiple medical image data and combining dynamic correction and feedback mechanisms. The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] First, we will introduce the implementation method of the multimodal image acquisition module. For example... Figure 1 As shown, this module consists of an MRI device, an fMRI device, and a DTI device, used to simultaneously acquire structural and functional data of the patient's brain. In practice, the patient is placed on an integrated imaging platform, and the MRI, fMRI, and DTI scanning devices are activated simultaneously via the control center. The MRI device acquires information about the patient's brain anatomy, the fMRI device records changes in brain functional activity areas, and the DTI device captures the direction of white matter fiber tracts. The data acquired by each device is transmitted in real time to the central processing unit via a high-speed data transmission interface and stored as raw data files in a unified format for subsequent processing.

[0023] The image preprocessing module performs a series of processing steps on the received raw data to generate the final fused image data. First, denoising is performed on the MRI, fMRI, and DTI data separately, using wavelet transform algorithms to remove high-frequency noise interference while preserving key anatomical and functional information. Next, registration is performed, using a mutual information-based registration algorithm to align the image data from different modalities to the same coordinate system, ensuring spatial consistency across modalities. Finally, standardization is performed, converting the registered image data into a representation in a standard brain template space, thereby eliminating the influence of individual differences in brain morphology. After completing these steps, the system generates fused image data in a unified coordinate system, serving as the basis for subsequent target analysis.

[0024] The core of the target analysis module lies in calculating the initial coordinates of the target points based on fused image data and performing dynamic correction. The target analysis process employs the fusion algorithm proposed in this invention, the core formula of which is: ; in, , and These are the weighting coefficients, Indicates the anatomical structure of the brain. Indicates areas of brain activity. Indicates the orientation of white matter fiber bundles. This is the final target location.

[0025] The weighting coefficients , and Adjustments are made dynamically based on disease type, including: For depression, set =0.4, =0.5, =0.1; For Parkinson's disease, set =0.3, =0.4, =0.3; For Alzheimer's disease, set =0.5, =0.3, =0.2.

[0026] The initial coordinates of the target point are calculated using this formula. Simultaneously, the dynamic correction module monitors head movement in real time using motion sensors mounted on the patient's head. When head displacement is detected, the system adjusts the target point coordinates accordingly based on the displacement vector provided by the sensors, ensuring the accuracy of the target point's position during stimulation.

[0027] The target guidance terminal is responsible for converting the corrected target coordinates into control commands for the robotic arm, enabling precise positioning of the transcranial magnetic stimulation coil. Specifically, the system first calculates the target position and posture required by the robotic arm's end effector based on the target coordinates, and then uses inverse kinematics algorithms to solve for the angle parameters of each joint of the robotic arm. The control system generates corresponding motor drive signals based on the calculation results, controlling the robotic arm to move to the target position along a predetermined trajectory. Upon reaching the target position, the system verifies the actual position of the robotic arm using a built-in laser rangefinder and vision sensors, ensuring sub-millimeter level positioning accuracy. Furthermore, the robotic arm is equipped with a force feedback device that automatically adjusts the movement speed and force when approaching the patient's head, avoiding discomfort or injury.

[0028] The efficacy evaluation and feedback module quantifies the treatment effect after stimulation and feeds the results back to the target analysis module. This module includes an EEG signal acquisition unit and a behavioral testing unit. The EEG signal acquisition unit records changes in the patient's electroencephalogram (EEG) signals after stimulation, while the behavioral testing unit designs assessment plans based on the patient's specific condition and evaluates the effect. The evaluation process consists of two main stages: EEG signal acquisition and behavioral testing. In the EEG signal acquisition stage, the system records changes in the patient's EEG signals after stimulation using high-density electrode caps, focusing on changes in the power spectral density of the functional brain regions corresponding to the stimulation site. The behavioral testing designs corresponding assessment plans based on the patient's specific condition; for example, the Hamilton Depression Rating Scale can be used for scoring patients with depression, and the Motor Function Rating Scale can be used for assessing patients with movement disorders. After statistical analysis, the collected evaluation data generates a quantitative efficacy evaluation report. If the evaluation results show that the stimulation effect does not meet expectations, the system feeds the relevant data back to the target analysis module, optimizing subsequent localization strategies by adjusting weighting coefficients or recalculating target locations.

[0029] In practical applications, taking the treatment of patients with depression as an example, the system first acquires structural and functional data of the patient's brain through a multimodal image acquisition module, focusing on brain regions closely related to emotion regulation, such as the prefrontal cortex and cingulate gyrus. After the image preprocessing module processes the acquired data, the target analysis module calculates the initial target coordinates based on a fusion algorithm. Considering the characteristics of depression treatment, [the system is then configured with appropriate parameters]. =0.4, =0.5, =0.1. During treatment, the dynamic correction module continuously monitors the patient's head movement and adjusts the target position in real time. The target guidance terminal controls the robotic arm to precisely move the transcranial magnetic stimulation coil to the target position and begin applying stimulation. After each stimulation, the effect evaluation feedback module, through EEG signal analysis, found a significant increase in the alpha wave power of the patient's left prefrontal cortex, while behavioral tests showed some relief of depressive symptoms. These positive results were recorded and used to optimize subsequent treatment plans.

[0030] In terms of system hardware configuration, the MRI equipment uses a 3T superconducting magnet, the fMRI equipment is equipped with a 32-channel phased array receiving coil, and the DTI equipment supports scanning with a maximum isotropic resolution of 2mm. The motion sensor uses a six-axis inertial measurement unit with MEMS technology and a sampling frequency of 100Hz. The robotic arm uses a seven-DOF collaborative robot with a repeatability accuracy better than 0.1mm. The EEG acquisition system uses a 256-lead amplifier with a sampling rate of up to 20kHz. All hardware devices are connected to the central processing unit via Gigabit Ethernet to ensure real-time and reliable data transmission.

[0031] In terms of software implementation, the system adopts a modular design, with each functional module developed independently and communicating through standardized interfaces. The image processing algorithm is implemented on the MATLAB platform, the robotic arm control program is written in C++, and the user interface is developed using Python. The system database uses a MySQL relational database management system to store patient information, image data, and treatment records. To ensure system security, all sensitive data is protected using the AES-256 encryption algorithm, and a comprehensive data backup and recovery mechanism is established.

[0032] Regarding system maintenance and upgrades, hardware devices are regularly calibrated and maintained to ensure that all performance indicators meet requirements. The software system releases new versions regularly through an online update mechanism, continuously optimizing algorithm performance and user interface experience. Simultaneously, a comprehensive logging mechanism has been established to facilitate the tracking and analysis of various events during the treatment process, providing data support for subsequent improvements.

[0033] Furthermore, this embodiment uses moderate depression (Hamilton Depression Rating Scale HAMD-17 score of 24) to illustrate the use of the system of the present invention: 1. Multimodal image acquisition module; Equipment configuration: MRI equipment: 3T superconducting magnet (Siemens Prisma), acquiring brain anatomical images (T1-weighted sequence, slice thickness 1mm, no gaps, matrix 256×256). fMRI equipment: 32-channel phased array coil, acquiring resting-state functional images (BOLD sequence, TR=2000ms, TE=30ms, slice thickness 3mm, total 40 slices); DTI equipment: employs a single-excitation spin echo sequence (b-value = 1000 s / mm², 30 diffusion directions, 2 mm layer thickness, isotropic resolution).

[0034] Data Acquisition Process: The patient lies supine with their head fixed to the integrated imaging platform. Three devices are simultaneously activated via the control center and data is continuously acquired for 15 minutes. The raw data is transmitted to the central processing unit via high-speed Ethernet and stored as a DICOM format file.

[0035] 2. Image preprocessing module, such as Figure 2 As shown; Denoising: The denoising unit of the preprocessing module uses a wavelet transform algorithm (the mother wavelet is db4, and the decomposition layer is 3) to remove high-frequency physiological noise (such as breathing and heartbeat artifacts) in the fMRI signal and equipment noise in MRI / DTI, while retaining the gray value of the brain region and the direction of the fiber bundle.

[0036] Registration Alignment: The registration unit, based on the mutual information algorithm (threshold set to 0.85), aligns fMRI, DTI images and MRI structural images to the MNI152 standard coordinate system (error ≤ 0.5 mm) to ensure the consistency of the three in spatial position.

[0037] Standardization processing: The standardized unit converts the registered images into a standard brain template space (voxel size 1×1×1mm³), eliminating the influence of individual brain morphology differences on subsequent analysis, and finally generating fused image data in a unified coordinate system.

[0038] 3. Target analysis module; Fusion algorithm application: The target analysis module calculates the initial target coordinates based on the fusion formula. ; For patients with depression, the weighting coefficient is dynamically adjusted as follows: =0.4 (emphasizing MRI anatomy) =0.5 (emphasis on fMRI functional activity), =0.1 (focusing on DTI fiber bundle orientation).

[0039] Key brain region localization: such as Figure 3As shown, MRI revealed that the gray matter volume in the left dorsolateral prefrontal cortex (DLPFC) was slightly smaller than the normal mean; fMRI showed a decreased resting low-frequency amplitude (ALFF) in this region (reduced functional activity); DTI showed that the integrity (FA value) of the white matter fiber bundles in this region and the cingulate gyrus was normal. The initial target coordinates were calculated as follows: MNI coordinates (-45, 26, 30), corresponding to the left DLPFC region.

[0040] 4. Dynamic correction module, such as Figure 4 As shown; Motion monitoring: The MEMS six-axis inertial sensor (sampling frequency 100Hz) of the dynamic correction module is fixed to the frontal pole of the patient's head to monitor the translation (x, y, z axes) and rotation (pitch, yaw, roll) motion of the head in real time and generate displacement vectors (unit: mm / °).

[0041] Real-time adjustment: When the sensor detects a head translation > 0.3 mm or rotation > 0.2°, the correction unit immediately performs dynamic correction of the target point coordinates based on the displacement vector. For example, when the patient's head translates 1.2 mm to the right, the system automatically adjusts the target point's x-axis coordinate from -45 to -46.2 to ensure positioning accuracy. Figure 4 As shown.

[0042] 5. Target guidance terminal Robotic arm control: The seven-degree-of-freedom collaborative robot (with a repeatability of 0.08 mm) receives the corrected target coordinates, and the control system solves the angles of each joint (35° for the shoulder joint, 90° for the elbow joint, etc.) through inverse kinematics algorithm, driving the robotic arm to move to the target position with the TMS coil.

[0043] Position verification: The robotic arm's built-in laser rangefinder (error ±0.05mm) and vision sensor (resolution 1920×1080) verify the actual position. After confirming that the deviation from the target coordinates is ≤0.1mm, the stimulation device is triggered.

[0044] Stimulation parameters: High-frequency rTMS stimulation was used (frequency 10Hz, intensity 110% of the motor threshold, single stimulation 20s, interval 40s, total 30 stimulations).

[0045] 6. Effectiveness evaluation and feedback module; Assessment metrics acquisition: EEG signal acquisition unit (256 leads, sampling rate 20kHz) recorded the electroencephalographic activity of the left DLPFC within 30 minutes after stimulation, revealing... The wave (8-13Hz) power increased by 23% compared to before stimulation (indicating enhanced functional activity); the behavioral testing unit was assessed using the HAMD-17 scale, and the patient's score decreased to 15 points (effective), while the Hamilton Anxiety Rating Scale (HAMA) score decreased from 18 points to 10 points.

[0046] Feedback Optimization: The effect evaluation feedback module generates a quantitative report showing that the stimulation effect has met the target. The system feeds the data back to the target analysis module, retaining the current weighting coefficients for subsequent treatment.

[0047] Results: After 6 weeks of treatment (5 times a week), the patient's HAMD score dropped to 8 (clinical cure), and the fMRI functional connectivity strength of the left DLPFC (with the cingulate gyrus) increased by 1.8 times compared to before treatment. Furthermore, target displacement due to head movement during treatment was consistently controlled within 0.3 mm. Compared to traditional single MRI localization, this system improved the treatment effectiveness by 40% with no adverse reactions.

[0048] In summary, this invention achieves precise target localization of transcranial magnetic stimulation (TMS) by integrating multimodal imaging data, introducing a dynamic correction mechanism, and establishing a closed-loop feedback system. The various modules of the system work collaboratively, forming a complete treatment process from data acquisition to final effect evaluation. This not only improves the accuracy of target localization but also significantly enhances the predictability and consistency of treatment outcomes.

Claims

1. A precise positioning system based on multimodal image fusion, characterized in that: include: The multimodal imaging acquisition module is used to simultaneously acquire MRI structural images, fMRI functional images, and DTI white matter fiber tract images of the patient's brain. The image preprocessing module, connected to the multimodal image acquisition module, is used to perform denoising, registration, and standardization processing on the acquired multimodal image data to generate fused image data in a unified coordinate system. The target analysis module, connected to the image preprocessing module, is used to calculate the initial coordinates of the target location based on the fused image data; The dynamic correction module, connected to the target analysis module, is used to monitor the patient's head movement in real time and dynamically correct the target position. The target guidance terminal is connected to the dynamic correction module and is used to control the robotic arm to perform precise positioning based on the corrected target coordinates.

2. The precise positioning system based on multimodal image fusion according to claim 1, characterized in that: The multimodal image acquisition module includes: MRI equipment is used to obtain information about the anatomical structure of the brain; fMRI equipment is used to acquire information about areas of functional activity in the brain; DTI equipment is used to acquire information about the orientation of white matter fiber bundles.

3. The precise positioning system based on multimodal image fusion according to claim 1, characterized in that: The image preprocessing module includes a denoising unit, a registration unit, and a normalization unit. The denoising unit uses a wavelet transform algorithm to remove high-frequency noise interference. The registration unit uses a mutual information-based registration algorithm to align image data of different modalities to the same coordinate system. The normalization unit converts the registered image data into a representation in the standard brain template space.

4. The precise positioning system based on multimodal image fusion according to claim 1, characterized in that: The target analysis module uses the following fusion algorithm to calculate the target location: ； in, , and These are the weighting coefficients, Indicates the anatomical structure of the brain. Indicates areas of brain activity. Indicates the orientation of white matter fiber bundles. This is the final target location.

5. The precise positioning system based on multimodal image fusion according to claim 1, characterized in that: The weighting coefficients , and The adjustments are made dynamically based on the disease type, including: For depression, set =0.4, =0.5, =0.1; For Parkinson's disease, set =0.3, =0.4, =0.3; For Alzheimer's disease, set =0.5, =0.3, =0.

2.

6. The precise positioning system based on multimodal image fusion according to claim 1, characterized in that: The dynamic correction module includes a motion sensor and a correction unit. The motion sensor is used to monitor the head movement state in real time and generate a displacement vector; the correction unit is used to dynamically adjust the target coordinates according to the displacement vector.

7. The precise positioning system based on multimodal image fusion according to claim 6, characterized in that: The target guidance terminal includes a robotic arm and a control system. The robotic arm moves to the target position according to the target positioning parameters. The control system solves the angle parameters of each joint of the robotic arm through an inverse kinematics algorithm and verifies the actual position of the robotic arm through a built-in laser rangefinder and vision sensor.

8. The precise positioning system based on multimodal image fusion according to claim 1, characterized in that: It also includes an effect evaluation feedback module, which is connected to the target analysis module and is used to evaluate the stimulus effect and optimize the positioning strategy.

9. The precise positioning system based on multimodal image fusion according to claim 8, characterized in that: The effect evaluation and feedback module includes an EEG signal acquisition unit and a behavioral testing unit. The EEG signal acquisition unit is used to record the changes in the patient's electroencephalogram (EEG) signals after receiving stimulation, and the behavioral testing unit is used to design an evaluation plan based on the patient's specific condition and evaluate the effect.

10. The precise positioning system based on multimodal image fusion according to claim 9, characterized in that: The effect evaluation feedback module generates a quantitative effect evaluation report by statistically analyzing EEG signals and behavioral test data, and feeds the evaluation results back to the target analysis module to adjust the weighting coefficients or recalculate the target location.

Images