Parkinson's dyskinesia individualized SCAN network positioning method based on multi-modal image and deep learning

Abstract

The invention discloses a Parkinson's dyskinesia individualized SCAN network positioning method based on a multi-modal image and deep learning. The method comprises the steps of obtaining multi-modal medical image data, preprocessing the multi-modal medical image data, obtaining a multi-modal structure image and functional connection data, and calculating a spontaneous neural activity index of a whole-brain voxel level; taking a priori brain region related to the spontaneous neural activity index and dyskinesia as a seed point, constructing a seed point voxel function connection graph representing individual brain function connection, and performing nonlinear feature fusion and extraction through the deep learning network model; the bilinear attention network is adopted to capture the interaction information of the feature data and the individual dyskinesia symptom which is significantly related, an individualized SCAN network positioning result is obtained, the structure-function coupling characteristics of the individual brain are comprehensively described, the cross-modal pathological features related to the dyskinesia can be more sensitively recognized, and the accuracy and accuracy of the diagnosis and treatment of the dyskinesia can be improved. And the accuracy and robustness of abnormal brain region detection are obviously improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of Parkinson's dyskinesia individualized SCAN network positioning, and particularly relates to a Parkinson's dyskinesia individualized SCAN network positioning method based on multi-modal images and deep learning. BACKGROUND

[0002] Parkinson's disease (PD) is a common neurodegenerative disease, its main symptoms include tremor, stiffness, bradykinesia, etc., and drug-induced dyskinesia (Levodopa-Induced Dyskinesia, LID) may occur during the course of disease progression and treatment, the mechanism of LID is complex, involving abnormal changes in functional connectivity of multiple brain regions, therefore, accurate positioning of the key brain function network causing LID is crucial for understanding its pathophysiological mechanism and developing individualized treatment strategies, at present, the methods for studying brain function network of PD patients mainly include traditional neuroimaging analysis techniques such as resting-state functional magnetic resonance imaging (rs-fMRI) and diffusion tensor imaging (DTI), as well as some emerging machine learning algorithms, however, these methods have certain limitations when applied to Parkinson's dyskinesia individualized SCAN (Symptom-Centric Abnormal Connectivity Network) network positioning, and the following problems exist: Traditional research often relies on single-modal image data (such as using only rs-fMRI or DTI), which cannot comprehensively capture the interaction between multiple biomarkers causing LID; Most existing research tends to use population-level data analysis to identify generally existing brain function network patterns, ignoring the importance of individual differences. The disease development trajectory, drug response and LID performance caused by each PD patient are highly individualized, and the population average result cannot effectively guide the design and optimization of individualized treatment plan; Although machine learning, especially deep learning models, has shown great potential in handling complex medical image data, most current applications still remain in preliminary classification or regression tasks, failing to deeply mine the underlying functional connectivity patterns. In addition, many deep learning models are considered as "black boxes", and the prediction results they provide are difficult for clinicians to understand and accept, thereby limiting the promotion and application of these technologies in actual diagnosis and treatment process. SUMMARY

[0003] The purpose of this invention is to provide a personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning, so as to solve the technical problems of single-modal imaging data sources, lack of personalized consideration and high computational model complexity in the existing technology for Parkinson's disease symptoms.

[0004] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution: This invention provides a personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning, comprising the following steps: Multimodal medical imaging data of individual Parkinson's disease patients are acquired, the multimodal imaging data are preprocessed to obtain multimodal structural images and functional connectivity data, and spontaneous neural activity indices at the whole brain voxel level are calculated based on the multimodal structural images and functional connectivity data. Using the spontaneous neural activity index and the prior brain regions associated with dyskinesia as seed points, the functional connectivity between the seed points and other voxels in the whole brain is calculated to construct a seed point voxel functional connectivity map that represents the functional connectivity of an individual brain. A deep learning network model is constructed, and the seed voxel functional connectivity map of the individual and the standardized structural image are used as inputs to the deep learning network model. Nonlinear feature fusion and extraction are performed through the deep learning network model to identify and output feature data. A bilinear attention network is used to capture the interaction information that is significantly related to the feature data and individual dyskinesia symptoms. The interaction information is integrated through a convolutional network to generate a spatial distribution map of abnormal brain function networks and obtain individualized SCAN network localization results. The individualized SCAN network localization results are visualized and output to guide the clinical individualized treatment target localization.

[0005] As a preferred embodiment of the present invention, multimodal medical imaging data of an individual Parkinson's disease patient is acquired, and the multimodal imaging data is preprocessed to obtain multimodal structural images and functional connectivity data, including: Acquire individual multimodal medical imaging data of Parkinson's disease patients, wherein the multimodal medical imaging data includes at least high-resolution structural magnetic resonance imaging (sMRI) data and resting-state functional magnetic resonance imaging (rs-fMRI) data; The sMRI data is preprocessed, the format of the sMRI data is converted and corrected and then normalized, and the normalized sMRI data is segmented into brain tissue to obtain multimodal structural images in standard space. The rs-fMRI data were preprocessed to remove unprocessed data. The rs-fMRI data within the time interval are spatially standardized by slice time correction and head motion correction, and the standardized rs-fMRI data are then smoothed and filtered. The preprocessed rs-fMRI data were then subjected to nuisance signal regression statistics to remove interference signals, specifically as follows: For the time series of each voxel in the preprocessed rs-fMRI data, a generalized linear model is used for fitting, and the expression is: in, This represents the time-series signal of the original fMRI. , Represents the nuisance signal regression values ​​at different time points. This represents the estimated coefficients of each signal regressor. The residual of the remaining signal after regression; From the preprocessed rs-fMRI data, time series of the whole brain or regions of interest (ROIs) are extracted, and the functional connectivity matrix between time series of different brain regions is calculated to obtain functional connectivity data.

[0006] As a preferred embodiment of the present invention, based on the multimodal structural images and functional connectivity data, spontaneous neural activity indices at the whole-brain voxel level are calculated, including: Voxel-based morphological analysis (VBM) was performed on the multimodal structural images to calculate the gray matter volume of each voxel in the whole brain, which serves as an indicator reflecting the local macroscopic structure of the brain. Based on the preprocessed functional magnetic resonance time series, the fractional low-frequency amplitude and local consistency value of each voxel in the whole brain were calculated as indicators reflecting the intensity of local spontaneous neural activity in the brain. The gray matter volume, fractional low-frequency amplitude, and local consistency value calculated for each voxel are standardized by z-score, and the standardized indices are weighted and combined at the voxel level to generate a multidimensional voxel-level feature vector. The feature vector serves as an indicator of spontaneous neural activity for training and prediction of the SCAN network model.

[0007] As a preferred embodiment of the present invention, the spontaneous neural activity index and the prior brain region associated with dyskinesia are used as seed points. The functional connectivity between the seed points and other voxels in the whole brain is calculated to construct a seed point voxel functional connectivity map representing the functional connectivity of an individual brain, including: Based on the multimodal structural image, a region of interest (ROI) associated with dyskinesia symptoms is defined, and the voxels contained within the ROI are used to determine the seed point coordinates in coordinate form. The average time series of all voxels within the seed point region is extracted from the preprocessed resting-state functional magnetic resonance imaging (rs-fMRI) data to obtain the average time series of the seed point. Calculate the correlation coefficient between the average time series of each seed point and the time series of every other voxel in the whole brain to obtain the functional connectivity strength map between the seed point and the voxels of the whole brain. By combining all seed points with the functional connectivity strength map calculated from whole-brain voxels, a multidimensional seed point voxel functional connectivity map representing the functional connectivity patterns of an individual brain is constructed.

[0008] As a preferred embodiment of the present invention, the seed point voxel functional connectivity graph employs an attention mechanism to construct a graph neural network architecture, including: The seed point voxel functional connectivity graph is constructed as a graph structure G=(V, E, A), where node V represents each node in the graph. The set, Let E represent a voxel in the brain, and let E represent two nodes. and The functional connection lines between them, where A represents the node feature, and each node... Include a feature vector ; An attention-based graph convolutional network adaptively learns the weight coefficients of different neighbor nodes with respect to the center node in the graph structure G. The graph neural network consists of multiple stacked graph attention layers to obtain the similarity between seed points. For each seed point, a Softmax normalization operation is performed on its neighboring points. The features of neighboring seed points are aggregated according to weight coefficients. The transformed features of neighboring nodes are then weighted and summed. A non-linear activation function is used to obtain the node. Feature representation; The feature vector of the graph structure is obtained by averaging all the features after iterating through multiple graph attention layers.

[0009] As a preferred embodiment of the present invention, a deep learning network model is constructed, and the seed point voxel functional connectivity map and the normalized structure image of the individual are used together as input to the deep learning network model. The deep learning network model performs nonlinear feature fusion and extraction, identifies and outputs feature data, including: Using the seed point voxel functional connectivity map as input, a three-dimensional convolutional graph neural network is used to extract the whole brain functional connectivity pattern features and output the first high-level feature vector. Using the standardized structural image as input, a three-dimensional convolutional neural network is used to extract the morphological features of the brain's anatomical structure and output a second high-level feature vector. The first high-level feature vector and the second high-level feature vector are input into the multimodal feature fusion module, and nonlinear fusion is performed through the multimodal feature fusion module to generate a fused feature vector; The fused feature vector is input to the output layer of the three-dimensional convolutional graph neural network. The output layer consists of one or more fully connected layers and outputs feature data that characterizes the brain function abnormality patterns related to individual dyskinesia.

[0010] In a preferred embodiment of the present invention, the multimodal feature fusion module employs a fully connected layer for feature fusion, specifically as follows: A fully connected layer is used to map the first high-level feature vector and the second high-level feature vector to the same feature dimension space; The first high-level feature vector and the second high-level feature vector are concatenated, and the nonlinear interaction relationship between the vectors is learned through a multilayer perceptron. Based on modal attention weights, the contribution of brain functional and structural modalities to the final decision is adaptively calculated and weighted summed. A bimodal feature interaction tensor is constructed from the weighted summed decision information through a convolution module to capture the relationship between features and obtain fused feature data.

[0011] As a preferred embodiment of the present invention, a bilinear attention network is used to capture interaction information that is significantly related to the feature data and individual dyskinesia symptoms, including: The fused feature data is mapped to two feature spaces to obtain structure-related feature vectors and functional connectivity-related feature vectors. Construct a bilinear pooling layer and calculate the low-rank approximate bilinear pooling result between the structure-related feature vector and the function-connection-related feature vector to capture the second-order interaction features of the structure-function cross-modality. Using an attention mechanism, the patient's clinical score for dyskinesia was used as a guiding signal to apply symptom-oriented attention weights to bilinear second-order interaction features, thereby obtaining brain network connectivity patterns that are highly correlated with symptom severity. The deep learning network model outputs a weighted interaction feature representation, which is then used as input for spatial decoding and anomaly localization by the convolutional network.

[0012] As a preferred embodiment of the present invention, a spatial distribution map of abnormal brain function networks is generated by integrating interactive information through a convolutional network to obtain individualized SCAN network localization results, including: The interaction feature representation output by the bilinear attention network is spatially reshaped and converted into three-dimensional feature volume data aligned with a standard brain template. Each voxel contains a high-dimensional interaction feature vector, forming an initial interaction feature map. The interaction feature map is encoded using a cascaded three-dimensional convolutional neural network. Local and global spatial patterns are extracted using 3×3×3 convolutional kernels with different receptive fields to capture the aggregation and topological structure of abnormal functional connections in brain space. A spatial attention module is introduced into the middle layer of the network to dynamically adjust the weights of features in each brain region based on the prior symptom-related brain network template, thereby strengthening the response of key pathways related to the pathological mechanism of Parkinson's disease dyskinesia. Spatial resolution is gradually restored by transposing convolution, and a three-dimensional scalar map aligned with the whole brain voxels is output. The pixel value represents the probability value of the corresponding brain region participating in the abnormal function network, which is denoted as the spatial distribution map of the abnormal brain function network. The anomaly probability map is binarized using an adaptive thresholding method to extract significantly high-activation regions. Adjacent clusters are then merged using connectivity analysis to obtain individualized symmetric or asymmetric anomaly function network nodes and their connecting edges. The core node coordinates of the SCAN network are mapped to a standardized stereotactic brain atlas, and the Brodmann partition, MNI coordinates and anatomical names are output to generate a personalized list of treatment target recommendations, which can be used to guide the implantation of deep brain stimulation (DBS) electrodes or the localization of transcranial magnetic stimulation (TMS) coils.

[0013] As a preferred embodiment of the present invention, the SCAN network defines the extracted abnormal brain functional regions as a symptom-centered abnormal connectivity network, and its core nodes are functional hub areas that are highly correlated with the intensity of the patient's dyskinesia symptoms.

[0014] Compared with the prior art, the present invention has the following advantages: This invention employs a bilinear attention network to model high-order fusion features extracted by deep learning, explicitly capturing the second-order interaction between structural and functional features. It also combines the patient's clinical symptom scores as guiding signals, assigning higher weights to brain connectivity patterns highly correlated with the severity of dyskinesia. This mechanism not only enhances the model's understanding of pathophysiological processes but also improves the interpretability of feature representations, making the output more biologically meaningful.

[0015] This invention uses the regions of spontaneous neural activity abnormalities in individual patients, combined with known key prior brain regions associated with dyskinesia, as dynamic seed points to construct individualized seed point-voxel functional connectivity maps. This strategy overcomes the limitations of traditional population template-driven analysis, fully considering the heterogeneity among different patients in terms of lesion distribution, symptom severity, and drug response, achieving true "one person, one map" functional network modeling. This provides a reliable basis for subsequent precision treatment. Furthermore, it comprehensively utilizes multiple modal imaging data, such as structural MRI and functional fMRI, to obtain standardized structural images and whole-brain voxel-level functional connectivity maps through preprocessing. Based on this, spontaneous neural activity indices are calculated to comprehensively characterize the structure-function coupling characteristics of the individual brain. Compared to research methods that rely solely on a single modality, this approach can more sensitively identify cross-modal pathological features associated with dyskinesia, significantly improving the accuracy and robustness of abnormal brain region detection. Attached Figure Description

[0016] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0017] Figure 1 The flowchart illustrates the personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning, as provided in this embodiment of the invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, this invention provides a personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning, including the following steps: Multimodal medical imaging data of individual Parkinson's disease patients are acquired, the multimodal imaging data are preprocessed to obtain multimodal structural images and functional connectivity data, and spontaneous neural activity indices at the whole brain voxel level are calculated based on the multimodal structural images and functional connectivity data. Using the spontaneous neural activity index and the prior brain regions associated with dyskinesia as seed points, the functional connectivity between the seed points and other voxels in the whole brain is calculated to construct a seed point voxel functional connectivity map that represents the functional connectivity of an individual brain. A deep learning network model is constructed, and the seed voxel functional connectivity map of the individual and the standardized structural image are used as inputs to the deep learning network model. Nonlinear feature fusion and extraction are performed through the deep learning network model to identify and output feature data. A bilinear attention network is used to capture the interaction information that is significantly related to the feature data and individual dyskinesia symptoms. The interaction information is integrated through a convolutional network to generate a spatial distribution map of abnormal brain function networks and obtain individualized SCAN network localization results. In this embodiment, the patient's spontaneous neural activity abnormality region combined with known key prior brain regions related to dyskinesia is used as a dynamic seed point to construct an individualized seed point-voxel functional connectivity map. This strategy breaks through the limitations of traditional population template-driven analysis, fully considers the heterogeneity among different patients in terms of lesion distribution, symptom severity, and drug response, and achieves true "one person, one map" functional network modeling, providing a reliable basis for subsequent precision treatment.

[0020] The individualized SCAN network localization results are visualized and output to guide the clinical individualized treatment target localization.

[0021] In this embodiment, multiple modal imaging data, such as structural MRI and functional fMRI, are comprehensively utilized. Standardized structural images and functional connectivity maps at the whole-brain voxel level are obtained through preprocessing. Spontaneous neural activity indices, such as ALFF / fALFF / ReHo, are calculated based on these data to comprehensively characterize the structure-function coupling characteristics of an individual's brain. Compared with research methods that rely solely on a single modality, this approach can more sensitively identify cross-modal pathological features associated with dyskinesia, significantly improving the accuracy and robustness of abnormal brain region detection.

[0022] Acquire multimodal medical imaging data of individual Parkinson's disease patients, preprocess the multimodal imaging data to obtain multimodal structural images and functional connectivity data, including: Acquire individual multimodal medical imaging data of Parkinson's disease patients, wherein the multimodal medical imaging data includes at least high-resolution structural magnetic resonance imaging (sMRI) data and resting-state functional magnetic resonance imaging (rs-fMRI) data; The sMRI data is preprocessed, the format of the sMRI data is converted and corrected and then normalized, and the normalized sMRI data is segmented into brain tissue to obtain multimodal structural images in standard space. In this embodiment, rigorous preprocessing steps are performed on the sMRI and rs-fMRI data, such as format conversion, correction, normalization, brain tissue segmentation, slice time correction, head motion correction, and smoothing filtering, to ensure that the final data used for analysis has high quality and consistency, thereby improving the reliability of subsequent analysis results.

[0023] The rs-fMRI data were preprocessed to remove unprocessed data. The rs-fMRI data within the time interval are spatially standardized by slice time correction and head motion correction, and the standardized rs-fMRI data are then smoothed and filtered. The preprocessed rs-fMRI data were then subjected to nuisance signal regression statistics to remove interference signals, specifically as follows: For the time series of each voxel in the preprocessed rs-fMRI data, a generalized linear model is used for fitting, and the expression is: in, This represents the time-series signal of the original fMRI. , Represents the nuisance signal regression values ​​at different time points. This represents the estimated coefficients of each signal regressor. The residual of the remaining signal after regression; In this embodiment, the nuisance signal regression method is used to process rs-fMRI data, which can effectively remove interference signals caused by physiological noise or head movements and retain true neural activity information. This method fits the time series of each voxel with a generalized linear model and calculates the residual as the denoised signal, further improving the accuracy of functional connectivity analysis.

[0024] In this embodiment, the sMRI data is normalized and then segmented into brain tissue, so that the brain images of different patients can be accurately mapped into a standard space, which facilitates cross-individual comparisons. The spatial standardization of rs-fMRI data helps to reduce the impact of individual differences when performing functional connectivity analysis in the whole brain, and improves the universality and reproducibility of the research results.

[0025] From the preprocessed rs-fMRI data, time series of the whole brain or regions of interest (ROIs) are extracted, and the functional connectivity matrix between time series of different brain regions is calculated to obtain functional connectivity data.

[0026] Based on the aforementioned multimodal structural images and functional connectivity data, spontaneous neural activity indices at the whole-brain voxel level are calculated, including: Voxel-based morphological analysis (VBM) was performed on the multimodal structural images to calculate the gray matter volume of each voxel in the whole brain, which serves as an indicator reflecting the local macroscopic structure of the brain. In this embodiment, gray matter volume obtained from voxel-based morphological analysis (VBM) is used as a structural indicator, and fractional low-frequency amplitude (fALFF) and local consistency ReHo derived from fMRI are used as functional indicators. Multidimensional information on structural atrophy and spontaneous neural activity abnormalities is extracted simultaneously at each voxel location in the whole brain. This strategy breaks through the limitations of traditional research that separates structure and function, and realizes refined modeling of structure-function coupling. It can detect the functional compensation or imbalance state of Parkinson's disease patients earlier and more sensitively before obvious structural damage occurs, and is especially suitable for early prediction and individualized assessment of dyskinesia.

[0027] Based on the preprocessed functional magnetic resonance time series, the fractional low-frequency amplitude and local consistency value of each voxel in the whole brain were calculated as indicators reflecting the intensity of local spontaneous neural activity in the brain. The gray matter volume, fractional low-frequency amplitude, and local consistency value calculated for each voxel are standardized by z-score, and the standardized indices are weighted and combined at the voxel level to generate a multidimensional voxel-level feature vector. In this embodiment, the gray matter volume, fALFF, and ReHo index are weighted and combined after z-score standardization to generate a multi-dimensional voxel feature vector at a uniform scale. This standardization process effectively eliminates the differences in data distribution between different modalities and individuals, avoids the problem of a certain modality dominating model training due to excessively large dimensions, significantly improves the stability of feature input and cross-subject comparability, and provides a high-quality and balanced training data foundation for subsequent deep learning models.

[0028] The feature vector serves as an indicator of spontaneous neural activity for training and prediction of the SCAN network model.

[0029] In this embodiment, all the indicators are calculated and fused at a voxel level of 3mm×3mm×3mm, which fully preserves the spatial details of the original image and avoids the information loss caused by coarse-grained analysis based on brain region templates, such as AAL and Harvard-Oxford. This high-resolution feature map can accurately locate functional abnormalities in small lesions or marginal regions, such as the posterior putamen and the supplementary motor area, which are closely related to dyskinesia, providing fine spatial support for the construction of personalized SCAN networks.

[0030] Using the spontaneous neural activity indices and prior brain regions associated with dyskinesia as seed points, the functional connectivity between these seed points and other voxels in the whole brain is calculated to construct a seed point voxel functional connectivity map representing the functional connectivity of the individual brain, including: Based on the multimodal structural image, a region of interest (ROI) associated with dyskinesia symptoms is defined, and the voxels contained within the ROI are used to determine the seed point coordinates in coordinate form. The average time series of all voxels within the seed point region is extracted from the preprocessed resting-state functional magnetic resonance imaging (rs-fMRI) data to obtain the average time series of the seed point. Calculate the correlation coefficient between the average time series of each seed point and the time series of every other voxel in the whole brain to obtain the functional connectivity strength map between the seed point and the voxels of the whole brain. In this embodiment, a voxel-by-voxel correlation analysis strategy of "seed point-whole brain voxel" is adopted to calculate the time series correlation coefficient between each seed point and every non-seed voxel in the whole brain, and generate a continuous functional connectivity strength map. Compared with the traditional coarse-grained connectivity matrix of "seed point-ROI", this method realizes the functional connectivity mapping at the sub-region level or even the voxel level in space, and can finely characterize the spatial distribution gradient of abnormal connections.

[0031] By combining all seed points with the functional connectivity strength map calculated from whole-brain voxels, a multidimensional seed point voxel functional connectivity map representing the functional connectivity patterns of an individual brain is constructed.

[0032] In this embodiment, instead of relying on traditional fixed regions of interest (ROIs) based on population templates, it combines individualized spontaneous neural activity indicators, such as gray matter atrophy and fALFF / ReHo abnormalities, with prior brain regions known to be closely related to Parkinson's dyskinesia, such as the posterior putamen, subthalamic nucleus, globus pallidus, and prefrontal cortex, to dynamically determine seed points for functional connectivity analysis. This strategy takes into account both individual heterogeneity and disease specificity, avoiding the "one-size-fits-all" problem caused by the use of uniform templates in traditional methods. It significantly improves the sensitivity and specificity of functional connectivity analysis for dyskinesia-related neural circuits, such as the basal ganglia-thalamus-cortex circuit.

[0033] The seed point voxel functional connectivity graph employs an attention mechanism to construct a graph neural network architecture, including: The seed point voxel functional connectivity graph is constructed as a graph structure G=(V, E, A), where node V represents each node in the graph. The set, Let E represent a voxel in the brain, and let E represent two nodes. and The functional connection lines between them, where A represents the node feature, and each node... Include a feature vector ; An attention-based graph convolutional network adaptively learns the weight coefficients of different neighbor nodes with respect to the center node in the graph structure G. The graph neural network consists of multiple stacked graph attention layers to obtain the similarity between seed points. For each seed point, a Softmax normalization operation is performed on its neighboring points. The features of neighboring seed points are aggregated according to weight coefficients. The transformed features of neighboring nodes are then weighted and summed. A non-linear activation function is used to obtain the node. Feature representation; In this embodiment, in each graph attention layer, by performing Softmax normalization on the neighboring points of each seed point and aggregating the features of the neighboring seed points based on the weight coefficients, efficient local information integration is achieved. This weighted summation method not only considers the influence of directly connected nodes, but also indirectly incorporates the information of distant nodes through multi-layer stacking, making the final node feature representation more comprehensive and accurate, which helps to achieve better performance in subsequent classification or regression tasks.

[0034] The feature vector of the graph structure is obtained by averaging all the features after iterating through multiple graph attention layers.

[0035] In this embodiment, the application of a nonlinear activation function can effectively increase the model's nonlinear fitting ability, enabling it to capture more complex brain network topologies. This step ensures that even with a very complex network structure, the model can still learn deep feature representations, further enhancing its ability to learn features of various types of brain diseases.

[0036] In this embodiment, the feature representations of all nodes after multiple graph attention layers are processed iteratively, and the feature vector of the entire graph structure is obtained by mean calculation. This approach not only preserves local detail information but also achieves effective fusion of cross-scale features, providing rich global context information.

[0037] A deep learning network model is constructed, using the seed voxel functional connectivity map and the normalized structure image of the individual as inputs. The deep learning network model performs nonlinear feature fusion and extraction to identify and output feature data, including: Using the seed point voxel functional connectivity map as input, a three-dimensional convolutional graph neural network is used to extract the whole brain functional connectivity pattern features and output the first high-level feature vector. Using the standardized structural image as input, a three-dimensional convolutional neural network is used to extract the morphological features of the brain's anatomical structure and output a second high-level feature vector. The first high-level feature vector and the second high-level feature vector are input into the multimodal feature fusion module, and nonlinear fusion is performed through the multimodal feature fusion module to generate a fused feature vector; In this embodiment, by combining whole-brain functional connectivity features and brain anatomical morphological features, the present invention can comprehensively capture brain changes related to dyskinesia from multiple dimensions. This multimodal information fusion approach not only enhances the depth of understanding of disease characteristics, but also helps to discover subtle but crucial changes that are difficult to detect with a single modality, thereby significantly improving the accuracy and sensitivity of disease diagnosis.

[0038] The fused feature vector is input to the output layer of the three-dimensional convolutional graph neural network. The output layer consists of one or more fully connected layers and outputs feature data that characterizes the brain function abnormality patterns related to individual dyskinesia.

[0039] In this embodiment, a three-dimensional convolutional graph neural network is used to process the seed point voxel functional connectivity graph and the normalized structure image respectively, which can effectively extract high-level feature vectors in their respective fields. This method not only preserves the spatial structure information of the original data, but also automatically learns complex nonlinear relationships, making the final generated feature representation richer and more discriminative.

[0040] The multimodal feature fusion module uses a fully connected layer for feature fusion, specifically: A fully connected layer is used to map the first high-level feature vector and the second high-level feature vector to the same feature dimension space; The first high-level feature vector and the second high-level feature vector are concatenated, and the nonlinear interaction relationship between the vectors is learned through a multilayer perceptron. Based on modal attention weights, the contribution of brain functional and structural modalities to the final decision is adaptively calculated and weighted summed. A bimodal feature interaction tensor is constructed from the weighted summed decision information through a convolution module to capture the relationship between features and obtain fused feature data.

[0041] In this embodiment, the first high-level feature vector and the second high-level feature vector are mapped to the same feature dimension space through a fully connected layer, ensuring that data from two different sources can be compared and fused on the same scale, which greatly improves the compatibility and comparability between features. This mapping process not only helps to remove redundant information in the original features, but also learns a more compact and discriminative feature representation.

[0042] In this embodiment, a multilayer perceptron is used to process the spliced ​​feature vectors, which can effectively learn and model the complex nonlinear interaction between the two modalities. Compared with the simple linear combination method, this method can more accurately capture the subtle but crucial connection between brain function and structure, thereby improving the model's expressiveness.

[0043] A bilinear attention network is used to capture interaction information that is significantly related to individual dyskinesia symptoms, including: The fused feature data is mapped to two feature spaces to obtain structure-related feature vectors and functional connectivity-related feature vectors. In this embodiment, the fused features are mapped to the structurally relevant feature space and the functional connectivity relevant feature space, respectively. The outer product or low-rank approximate bilinear response between the two is calculated through a bilinear pooling layer, thereby explicitly modeling the nonlinear coupling relationship between structural atrophy and functional abnormality. Compared with traditional splicing or weighted fusion methods, this method can effectively capture second-order interaction patterns with clear neurobiological significance, such as "whether structural degradation in a certain region is accompanied by abnormal enhancement of its functional connectivity", and significantly improve the model's ability to represent the complex pathological mechanisms of Parkinson's dyskinesia.

[0044] Construct a bilinear pooling layer and calculate the low-rank approximate bilinear pooling result between the structure-related feature vector and the function-connection-related feature vector to capture the second-order interaction features of the structure-function cross-modality. In this embodiment, a Compact-Bilinear-Pooling low-rank approximation strategy is adopted, which significantly reduces computational complexity and the number of parameters while retaining key cross-modal interaction information.

[0045] Using an attention mechanism, the patient's clinical score for dyskinesia was used as a guiding signal to apply symptom-oriented attention weights to bilinear second-order interaction features, thereby obtaining brain network connectivity patterns that are highly correlated with symptom severity. The deep learning network model outputs a weighted interaction feature representation, which is then used as input for spatial decoding and anomaly localization by the convolutional network.

[0046] In this embodiment, the patient's clinical dyskinesia score, such as UPDRS-IV, AIMS, or PINC score, is used as a guiding signal and applied to the bilinear interaction features. Weights are dynamically allocated through an attention mechanism to highlight brain network connectivity patterns that are highly correlated with symptom severity. This symptom-centered modeling paradigm aligns the model's learning objectives directly with clinical needs, avoids interference from irrelevant features that may occur in unsupervised or weakly supervised models, and improves the clinical relevance and interpretability of the output results.

[0047] By integrating interactive information through convolutional networks, a spatial distribution map of abnormal brain function networks is generated, obtaining individualized SCAN network localization results, including: The interaction feature representation output by the bilinear attention network is spatially reshaped and converted into three-dimensional feature volume data aligned with a standard brain template. Each voxel contains a high-dimensional interaction feature vector, forming an initial interaction feature map. The interaction feature map is encoded using a cascaded three-dimensional convolutional neural network. Local and global spatial patterns are extracted using 3×3×3 convolutional kernels with different receptive fields to capture the aggregation and topological structure of abnormal functional connections in brain space. In this embodiment, the high-dimensional interactive feature representation is reshaped into three-dimensional feature volume data aligned with the MNI152 standard brain template, and then processed layer by layer through a 3D-CNN with an encoder-decoder structure. Finally, a three-dimensional probability map consistent with the original image voxel space is output. This end-to-end architecture avoids the information loss caused by ROI division or downsampling in traditional methods, and fully preserves the millimeter-level spatial resolution. It can accurately locate small lesions or abnormally connected regions with blurred boundaries, such as the posterior putamen and the marginal area of ​​the subthalamic nucleus, significantly improving the localization accuracy.

[0048] In this embodiment, cascaded 3×3×3 convolutional kernels are used for multi-layer encoding to construct a multi-receptive field feature extraction mechanism. This enables the model to capture the aggregation of local functional connectivity clusters, such as abnormal synchronization within the basal ganglia, and to identify long-range abnormal pathways across brain regions, such as cortico-striatal-thalamic circuit abnormalities. This multi-scale analysis capability helps to comprehensively characterize the complex brain network reorganization patterns associated with Parkinson's dyskinesia and enhance the depth of understanding of network-level pathological mechanisms.

[0049] A spatial attention module is introduced into the middle layer of the network to dynamically adjust the weights of features in each brain region based on the prior symptom-related brain network template, thereby strengthening the response of key pathways related to the pathological mechanism of Parkinson's disease dyskinesia. In this embodiment, a spatial attention module is embedded in the middle layer of the network, and key brain network templates known to be related to dyskinesia, such as motor circuits and limbic systems, are used as prior knowledge to guide the dynamic enhancement of the characteristic responses of relevant brain regions, such as the globus pallidus, supplementary motor area, and anterior cingulate cortex. This design effectively suppresses non-specific noise interference, making the model more focused on neural pathways with clear pathological significance, and significantly improving the sensitivity and specificity of abnormal network detection.

[0050] Spatial resolution is gradually restored by transposing convolution, and a three-dimensional scalar map aligned with the whole brain voxels is output. The pixel value represents the probability value of the corresponding brain region participating in the abnormal function network, which is denoted as the spatial distribution map of the abnormal brain function network. In this embodiment, the three-dimensional scalar map output by transposed convolution upsampling has each voxel value representing the probability value of the region participating in the SCAN network, forming a continuous and quantifiable distribution of abnormality. This probabilistic output supports binarization using an adaptive threshold method, avoiding inter-individual bias caused by a fixed threshold, and is suitable for personalized modeling and longitudinal efficacy evaluation of patients with different degrees of severity.

[0051] The anomaly probability map is binarized using an adaptive thresholding method to extract significantly high-activation regions. Adjacent clusters are then merged using connectivity analysis to obtain individualized symmetric or asymmetric anomaly function network nodes and their connecting edges. The core node coordinates of the SCAN network are mapped to a standardized stereotactic brain atlas, and the Brodmann partition, MNI coordinates and anatomical names are output to generate a personalized list of treatment target recommendations, which can be used to guide the implantation of deep brain stimulation (DBS) electrodes or the localization of transcranial magnetic stimulation (TMS) coils.

[0052] In this embodiment, the core nodes of the SCAN network are mapped to a standardized stereotactic brain atlas, and their MNI coordinates, Brodmann partitions, and anatomical names are automatically output to generate a structured list of individualized treatment target recommendations. This result can be seamlessly integrated into a neuronavigation system to guide the planning of deep brain stimulation electrode implantation paths or the precise positioning of transcranial magnetic stimulation coils, greatly improving surgical safety and treatment effectiveness.

[0053] The SCAN network defines the extracted abnormal brain functional regions as a symptom-centered abnormal connectivity network, with its core nodes being functional hub areas that are highly correlated with the intensity of the patient's dyskinesia symptoms.

[0054] In this embodiment, SCAN network targets are automatically generated based on individual multimodal imaging and symptom data, realizing a paradigm shift from general targets to personalized abnormal network hubs. This method is particularly suitable for patients with drug-resistant or complex symptoms, providing a scientific and repeatable technical path for precise neuromodulation therapy.

[0055] This invention employs a bilinear attention network to model high-order fusion features extracted by deep learning, explicitly capturing the second-order interaction between structural and functional features. It also combines the patient's clinical symptom scores as guiding signals, assigning higher weights to brain connectivity patterns highly correlated with the severity of dyskinesia. This mechanism not only enhances the model's understanding of pathophysiological processes but also improves the interpretability of feature representations, making the output more biologically meaningful.

[0056] This invention uses the regions of spontaneous neural activity abnormalities in individual patients, combined with known key prior brain regions associated with dyskinesia, as dynamic seed points to construct individualized seed point-voxel functional connectivity maps. This strategy overcomes the limitations of traditional population template-driven analysis, fully considering the heterogeneity among different patients in terms of lesion distribution, symptom severity, and drug response, achieving true "one person, one map" functional network modeling. This provides a reliable basis for subsequent precision treatment. Furthermore, it comprehensively utilizes multiple modal imaging data, such as structural MRI and functional fMRI, to obtain standardized structural images and whole-brain voxel-level functional connectivity maps through preprocessing. Based on this, spontaneous neural activity indices are calculated to comprehensively characterize the structure-function coupling characteristics of the individual brain. Compared to research methods that rely solely on a single modality, this approach can more sensitively identify cross-modal pathological features associated with dyskinesia, significantly improving the accuracy and robustness of abnormal brain region detection.

[0057] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning, characterized in that, Includes the following steps: Multimodal medical imaging data of individual Parkinson's disease patients are acquired, the multimodal imaging data are preprocessed to obtain multimodal structural images and functional connectivity data, and spontaneous neural activity indices at the whole brain voxel level are calculated based on the multimodal structural images and functional connectivity data. Using the spontaneous neural activity index and the prior brain regions associated with dyskinesia as seed points, the functional connectivity between the seed points and other voxels in the whole brain is calculated to construct a seed point voxel functional connectivity map that represents the functional connectivity of an individual brain. A deep learning network model is constructed, and the seed voxel functional connectivity map of the individual and the standardized structural image are used as inputs to the deep learning network model. Nonlinear feature fusion and extraction are performed through the deep learning network model to identify and output feature data. A bilinear attention network is used to capture the interaction information that is significantly related to individual dyskinesia symptoms. The interaction information is integrated through a convolutional network to generate a spatial distribution map of abnormal brain function networks and obtain individualized SCAN network localization results. The individualized SCAN network localization results are visualized and output to guide individualized clinical treatment target localization.

2. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 1, characterized in that, Acquire multimodal medical imaging data of individual Parkinson's disease patients, preprocess the multimodal imaging data to obtain multimodal structural images and functional connectivity data, including: Acquire individual multimodal medical imaging data of Parkinson's disease patients, wherein the multimodal medical imaging data includes at least high-resolution structural magnetic resonance imaging (sMRI) data and resting-state functional magnetic resonance imaging (rs-fMRI) data; The sMRI data is preprocessed, the format of the sMRI data is converted and corrected and then normalized, and the normalized sMRI data is segmented into brain tissue to obtain multimodal structural images in standard space. The rs-fMRI data were preprocessed to remove unprocessed data. The rs-fMRI data within the time interval are spatially standardized by slice time correction and head motion correction, and the standardized rs-fMRI data are then smoothed and filtered. The preprocessed rs-fMRI data were then subjected to nuisance signal regression statistics to remove interference signals, specifically as follows: For the time series of each voxel in the preprocessed rs-fMRI data, a generalized linear model is used for fitting, and the expression is: in, This represents the time-series signal of the original fMRI. , Represents the nuisance signal regression values ​​at different time points. This represents the estimated coefficients of each signal regressor. The residual of the remaining signal after regression; From the preprocessed rs-fMRI data, time series of the whole brain or regions of interest (ROIs) are extracted, and the functional connectivity matrix between time series of different brain regions is calculated to obtain functional connectivity data.

3. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 2, characterized in that, Based on the aforementioned multimodal structural images and functional connectivity data, spontaneous neural activity indices at the whole-brain voxel level are calculated, including: Voxel-based morphological analysis (VBM) was performed on the multimodal structural images to calculate the gray matter volume of each voxel in the whole brain, which serves as an indicator reflecting the local macroscopic structure of the brain. Based on the preprocessed functional magnetic resonance time series, the fractional low-frequency amplitude and local consistency value of each voxel in the whole brain were calculated as indicators reflecting the intensity of local spontaneous neural activity in the brain. The gray matter volume, fractional low-frequency amplitude, and local consistency value calculated for each voxel are standardized by z-score, and the standardized indices are weighted and combined at the voxel level to generate a multidimensional voxel-level feature vector. The feature vector serves as an indicator of spontaneous neural activity for training and prediction of the SCAN network model.

4. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 3, characterized in that, Using the spontaneous neural activity indices and prior brain regions associated with dyskinesia as seed points, the functional connectivity between these seed points and other voxels in the whole brain is calculated to construct a seed point voxel functional connectivity map representing the functional connectivity of the individual brain, including: Based on the multimodal structural image, a region of interest (ROI) associated with dyskinesia symptoms is defined, and the voxels contained within the ROI are used to determine the seed point coordinates in coordinate form. The average time series of all voxels within the seed point region is extracted from the preprocessed resting-state functional magnetic resonance imaging (rs-fMRI) data to obtain the average time series of the seed point. Calculate the correlation coefficient between the average time series of each seed point and the time series of every other voxel in the whole brain to obtain the functional connectivity strength map between the seed point and the voxels of the whole brain. By combining all seed points with the functional connectivity strength map calculated from whole-brain voxels, a multidimensional seed point voxel functional connectivity map representing the functional connectivity patterns of an individual brain is constructed.

5. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 3, characterized in that, The seed point voxel functional connectivity graph employs an attention mechanism to construct a graph neural network architecture, including: The seed point voxel functional connectivity graph is constructed as a graph structure G=(V, E, A), where node V represents each node in the graph. The set, Let E represent a voxel in the brain, and let E represent two nodes. and The functional connection lines between them, where A represents the node feature, and each node... Include a feature vector ; An attention-based graph convolutional network adaptively learns the weight coefficients of different neighbor nodes with respect to the center node in the graph structure G. The graph neural network consists of multiple stacked graph attention layers to obtain the similarity between seed points. For each seed point, a Softmax normalization operation is performed on its neighboring points. The features of neighboring seed points are aggregated according to weight coefficients. The transformed features of neighboring nodes are then weighted and summed. A non-linear activation function is used to obtain the node. Feature representation; The feature vector of the graph structure is obtained by averaging all the features after iterating through multiple graph attention layers.

6. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 5, characterized in that, A deep learning network model is constructed, using the seed voxel functional connectivity map and the normalized structure image of the individual as inputs. The deep learning network model performs nonlinear feature fusion and extraction to identify and output feature data, including: Using the seed point voxel functional connectivity map as input, a three-dimensional convolutional graph neural network is used to extract the whole brain functional connectivity pattern features and output the first high-level feature vector. Using the standardized structural image as input, a three-dimensional convolutional neural network is used to extract the morphological features of the brain's anatomical structure and output a second high-level feature vector. The first high-level feature vector and the second high-level feature vector are input into the multimodal feature fusion module, and nonlinear fusion is performed through the multimodal feature fusion module to generate a fused feature vector; The fused feature vector is input to the output layer of the three-dimensional convolutional graph neural network. The output layer consists of one or more fully connected layers and outputs feature data that characterizes the brain function abnormality patterns related to individual dyskinesia.

7. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 6, characterized in that, The multimodal feature fusion module uses a fully connected layer for feature fusion, specifically: A fully connected layer is used to map the first high-level feature vector and the second high-level feature vector to the same feature dimension space; The first high-level feature vector and the second high-level feature vector are concatenated, and the nonlinear interaction relationship between the vectors is learned through a multilayer perceptron. Based on modal attention weights, the contribution of brain functional and structural modalities to the final decision is adaptively calculated and weighted summed. A bimodal feature interaction tensor is constructed from the weighted summed decision information through a convolution module to capture the relationship between features and obtain fused feature data.

8. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 7, characterized in that, A bilinear attention network is used to capture interaction information that is significantly related to individual dyskinesia symptoms, including: The fused feature data is mapped to two feature spaces to obtain structure-related feature vectors and function-connection-related feature vectors. Construct a bilinear pooling layer and calculate the low-rank approximate bilinear pooling result between the structure-related feature vector and the function-connection-related feature vector to capture the second-order interaction features of the structure-function cross-modality. Using an attention mechanism, the patient's clinical score for dyskinesia was used as a guiding signal to apply symptom-oriented attention weights to bilinear second-order interaction features, thereby obtaining brain network connectivity patterns that are highly correlated with symptom severity. The deep learning network model outputs a weighted interaction feature representation, which is then used as input for spatial decoding and anomaly localization by the convolutional network.

9. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 8, characterized in that, By integrating interactive information through convolutional networks, a spatial distribution map of abnormal brain function networks is generated, obtaining individualized SCAN network localization results, including: The interaction feature representation output by the bilinear attention network is spatially reshaped and converted into three-dimensional feature volume data aligned with a standard brain template. Each voxel contains a high-dimensional interaction feature vector, forming an initial interaction feature map. The interaction feature map is encoded using a cascaded three-dimensional convolutional neural network. Local and global spatial patterns are extracted using 3×3×3 convolutional kernels with different receptive fields to capture the aggregation and topological structure of abnormal functional connections in brain space. A spatial attention module is introduced into the middle layer of the network to dynamically adjust the weights of features in each brain region based on the prior symptom-related brain network template, thereby strengthening the response of key pathways related to the pathological mechanism of Parkinson's disease dyskinesia. Spatial resolution is gradually restored by transposing convolution, and a three-dimensional scalar map aligned with the whole brain voxels is output. The pixel value represents the probability value of the corresponding brain region participating in the abnormal function network, which is denoted as the spatial distribution map of the abnormal brain function network. The anomaly probability map is binarized using an adaptive thresholding method to extract significantly high-activation regions. Adjacent clusters are then merged using connectivity analysis to obtain individualized symmetric or asymmetric anomaly function network nodes and their connecting edges. The core node coordinates of the SCAN network are mapped to a standardized stereotactic brain atlas, and the Brodmann partition, MNI coordinates and anatomical names are output to generate a personalized list of treatment target recommendations, which can be used to guide the implantation of deep brain stimulation (DBS) electrodes or the localization of transcranial magnetic stimulation (TMS) coils.

10. The personalized SCAN network localization method for Parkinson's dyskinesia based on multimodal imaging and deep learning according to claim 9, characterized in that, The SCAN network defines the extracted abnormal brain functional regions as a symptom-centered abnormal connectivity network, with its core nodes being functional hub areas that are highly correlated with the intensity of the patient's dyskinesia symptoms.

Images