Neurological disease diagnosis system based on space-time attention and dynamic domain self-adaption

Abstract

The invention discloses a neural disease diagnosis system based on space-time attention and dynamic field self-adaption. The method belongs to the technical field of cross-modal medical data adaptive analysis. The technical problem that a brand new system capable of simultaneously fusing multi-modal information and modeling multi-scale spatial-temporal features and having dynamic field adaptive ability is urgently needed to improve the accuracy and generalization ability of intelligent diagnosis of multi-site nerve diseases is solved. The system comprises a data preprocessing module for extracting a standardized time sequence of a brain region from fMRI time sequence data; the two-channel feature coding network module obtains global features through an attention mechanism; according to the feature fusion and classification module, a main task classifier executes a main task and predicts whether a to-be-tested person suffers from nerve diseases or not, and a domain task classifier executes a domain task and predicts a site to which the to-be-tested person belongs; and the dynamic balance training module adjusts the dynamic balance of the main task and the domain task through a dynamic balance control strategy.

Description

Technical Field

[0001] This invention belongs to the field of cross-modal medical data adaptive analysis technology, specifically relating to a neurological disease diagnostic system based on spatiotemporal attention and dynamic domain adaptation. Background Technology

[0002] Resting-state functional magnetic resonance imaging (rs-fMRI) can non-invasively record spontaneous neural activity patterns throughout the brain and has been widely used in the study of the mechanisms and auxiliary diagnosis of neurodevelopmental disorders such as attention deficit hyperactivity disorder (ADHD). However, current techniques have the following limitations in multi-site fMRI data analysis: Significant differences in cross-site data distribution: Differences in scanning equipment, sequence parameters, and subject populations used by different medical institutions lead to inconsistent data distribution, resulting in a significant decrease in the model's generalization ability.

[0003] Insufficient information from single modal features: Traditional methods often rely on single image features and cannot make full use of non-imaging information such as clinical questionnaires and behavioral scales, resulting in limited diagnostic accuracy.

[0004] Insufficient utilization of spatiotemporal features: Most methods perform single modeling in the time or spatial dimension, failing to simultaneously capture the dynamic changes of brain activity over time and the spatial connectivity patterns of brain regions.

[0005] Lack of dynamic training strategies: Existing domain adaptation methods mostly use fixed weights, which cannot dynamically adjust the balance between the main task and the domain task based on the task performance during training, resulting in unstable model training.

[0006] Therefore, there is an urgent need for a novel system that can simultaneously integrate multimodal information, model multi-scale spatiotemporal features, and possess dynamic domain adaptive capabilities, in order to improve the accuracy and generalization ability of intelligent diagnosis of multi-site neurological diseases. Summary of the Invention

[0007] To address the urgent need for a novel system capable of simultaneously integrating multimodal information, modeling multi-scale spatiotemporal features, and possessing dynamic domain adaptation capabilities to improve the accuracy and generalization of intelligent diagnosis of multi-site neurological diseases, this application provides a neurological disease diagnosis system based on spatiotemporal attention and dynamic domain adaptation. The system includes: Data preprocessing module: Collects clinical characteristic data and fMRI time-series data of test subjects from different sites, preprocesses the fMRI time-series data, and extracts data from the fMRI time-series data. Standardized time series of brain regions ; Dual-channel feature coding network module: for clinical feature data and Feature encoding is performed separately, and the encoded information is fused together to obtain global features through an attention mechanism; Feature fusion and classification module: The global features are fused to obtain fused features; the fused features are then input into the main task classifier and the domain task classifier respectively. The main task classifier performs the main task to predict whether the test subject has a neurological disease, and the domain task classifier performs the domain task to predict the site to which the test subject belongs. Dynamic balancing training module: Optimizes the domain task classifier through backpropagation mechanism to reduce domain task loss; adjusts the dynamic balance between the main task and the domain task through dynamic balancing control strategy.

[0008] Furthermore, the clinical feature data is the identity information data of the person being tested, and is not imaging information data.

[0009] Furthermore, feature encoding of clinical feature data specifically involves encoding the clinical feature data into shallow features through a two-layer fully connected network. and deep features : = ; = ; in, For clinical feature vectors, , , and These are the learnable parameters. For activation function, Indicates batch normalization; right Feature encoding specifically involves using a one-dimensional convolutional layer to extract temporal features: ; in, This indicates the extracted time features. This represents one-dimensional convolution; The fusion of encoded information specifically involves projecting shallow features and combining them with... Fusion yields the characteristics of the fused product : ; in, , , Represents the projection weight matrix. This represents the sigmoid function. Represents the gate weight matrix. This indicates element-wise multiplication.

[0010] Furthermore, obtaining global features through the attention mechanism specifically involves learning through a temporal attention mechanism. The importance weights of each time point are assigned to obtain the fMRI temporal feature matrix after temporal attention weighting. By learning the importance weights between brain regions through spatial attention mechanisms, an fMRI spatial feature matrix weighted by spatial attention weights is obtained. By introducing a learnable brain region importance parameter R, global features are obtained through global average pooling. : , , in This indicates global tie pooling.

[0011] Furthermore, the global features are fused to obtain the fused features, specifically as follows: pass Obtain global features attention weight vector ,in, This represents a multilayer perceptron; pass Obtaining deep features attention weight vector ; pass Obtain the weighted global feature matrix ; pass Obtain the weighted deep feature matrix ; pass Obtain the difference feature matrix ; pass Obtain the fusion feature matrix .

[0012] Furthermore, the specific operations performed in the main task classifier are as follows: Feature transformation layer: through Perform feature transformation to obtain the output vector after feature transformation. ;in, For learnable weight matrix, For learnable bias terms; Probability calculation layer: through Calculate the probability that the test subject has a neurological disease, where express function, This indicates the probability that the test subject is not a patient with a neurological disease. This indicates the probability that the test subject is a patient with a neurological disease. and These are two elements in the output vector z after feature transformation, corresponding to the original predicted values ​​of the two diagnostic results: where The original predicted value corresponding to the "non-neurological disease patients" category, The original predicted value corresponding to the category "patient with neurological disease"; Diagnostic decision-making level: through Make diagnostic decisions. This indicates a predictive result regarding whether the test subject has a neurological disorder. This indicates that the test subject does not suffer from a neurological disease. This indicates that the person being tested has a neurological disorder. Indicates a non-neurological disease. It indicates a neurological disorder.

[0013] Furthermore, the specific operations performed in the domain task classifier are as follows: Gradient inversion layer: through Obtain the multimodal fusion feature matrix ,in This indicates a gradient inversion layer. Indicates the dynamically adjusted gradient reversal strength; During forward propagation, During back propagation ,in, This represents the total loss function for the main task classifier and the domain task classifier. This indicates the operation of calculating partial derivatives; Feature transformation layer: through Obtain the original prediction vector of the site affiliation of the test personnel. ;in, For learnable weight matrix, For learnable bias terms; Site prediction layer: via Obtain the probability distribution vector of the test subjects belonging to each site. ,in Indicates the number of sites; Site decision-makers: Through Obtain the site of the person to be tested ,in This refers to any one of all sites. Indicates taking The site with the largest probability distribution vector.

[0014] Furthermore, the backpropagation mechanism requires forward propagation to occur first, specifically: Forward propagation phase: Domain task classifier receives data. Output the site to which the test subject belongs, and combine it with the real site tag of the test subject to calculate the domain task loss through the cross-entropy loss function; Backpropagation phase: The obtained domain task loss is backpropagated in the domain task classifier, and the parameters of each layer in the domain task classifier are optimized by the optimizer.

[0015] Furthermore, the dynamic balancing control strategy is implemented through a dynamic balancing controller, which dynamically adjusts the performance based on the primary task and the domain tasks. Specifically: Calculate the accuracy of the main task domain task accuracy ; Calculate the gradient norm ratio : ,in, The gradient represents the domain task loss. The gradient represents the loss of the main task. This indicates the L2 norm operation. As an adjustment factor, when >70% ,otherwise ; Smooth Update : ,in, Indicates the state before the update , Indicates the updated , It is a smoothing factor; Boundary constraints: , , ,in, This is a clipping function, indicating that the updated... The value is limited to and Within the range.

[0016] The beneficial effects of the system described in this invention are as follows: To address the issue of significant differences in data distribution across different sites: Differences in scanning equipment, sequence parameters, and subject populations used by different medical institutions lead to inconsistent data distribution, resulting in a significant decrease in the model's generalization ability.

[0017] The system of this invention effectively solves the problem of inconsistent data distribution across sites through dynamic adversarial learning between the domain task (site prediction) and the main task (disease prediction).

[0018] Addressing the issue of insufficient information from single modal features: Traditional methods often rely on single image features, making it difficult to fully utilize non-imaging information such as clinical questionnaires and behavioral scales, thus limiting diagnostic accuracy.

[0019] The system of this invention fully utilizes imaging (fMRI) and non-imaging (clinical features) modal information through "dual-channel feature encoding + bidirectional attention fusion", breaking through the limitations of traditional single modality.

[0020] To address the problem of insufficient utilization of spatiotemporal features: most methods perform single modeling in the time or spatial dimension, failing to simultaneously capture the dynamic changes of brain activity in the time series and the spatial connectivity patterns of brain regions.

[0021] The system of this invention achieves complete capture of the spatiotemporal features of brain activity through "hierarchical spatiotemporal attention + brain region importance modeling", which solves the shortcomings of the traditional method of "single-dimensional modeling".

[0022] The lack of dynamic training strategies is a problem: existing domain adaptation methods mostly use fixed weights, which cannot dynamically adjust the balance between the main task and the domain task based on the task performance during training, resulting in unstable model training.

[0023] The system of this invention replaces traditional fixed-weight training with a "dynamic balance control strategy" to solve the problem of unstable model training. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the workflow of the neurological disease diagnosis system in an embodiment of the present invention. Detailed Implementation

[0025] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0026] Example 1 This embodiment provides a neurological disease diagnostic system based on spatiotemporal attention and dynamic domain adaptation, the system comprising: Data preprocessing module: Collects 3D clinical feature data and fMRI time series data of test subjects from different sites, preprocesses the fMRI time series data, and extracts data from the fMRI time series data. Standardized time series of brain regions ; Dual-channel feature encoding network module: for 3D clinical feature data and Feature encoding is performed separately, and the encoded information is fused together to obtain global features through an attention mechanism; Feature fusion and classification module: The global features are fused to obtain fused features; the fused features are then input into the main task classifier and the domain task classifier respectively. The main task classifier performs the main task to predict whether the test subject has a neurological disease, and the domain task classifier performs the domain task to predict the site to which the test subject belongs. Dynamic balancing training module: Optimizes the domain task classifier through backpropagation mechanism to reduce domain task loss; adjusts the dynamic balance between the main task and the domain task through dynamic balancing control strategy.

[0027] The system's workflow diagram is as follows: Figure 1 As shown, subsequent embodiments will be combined with Figure 1 The workflow of the system is described in detail.

[0028] Example 2 This embodiment further defines Embodiment 1 and provides a detailed description of the data preprocessing module.

[0029] Data preprocessing module: Collects 3D clinical feature data and fMRI time series data of test subjects from different sites, preprocesses the fMRI time series data, and extracts data from the fMRI time series data. Standardized time series of brain regions .

[0030] The data preprocessing module collects fMRI time-series data and clinical characteristic data of the subjects from multiple sites.

[0031] Clinical characteristic data consists of the identity information of the test subjects, such as age, gender, education, and other non-imaging information.

[0032] The sites are the data sources for the test subjects. For example, if the fMRI time series data and clinical characteristic data of the test subjects are obtained through 20 different medical departments, then the 20 different medical departments are 20 sites.

[0033] The preprocessing workflow for fMRI time-series data is as follows: First, the original images are converted to standard NIfTI format and reoriented, with slice temporal correction performed if necessary. Then, head motion correction is performed on the whole-brain EPI sequences using FSL's MCFLIRT, and skull dissection is performed on the average functional images to generate brain masks. Spatially, approximately 6 mm FWHM Gaussian smoothing is applied to improve the signal-to-noise ratio, while temporally, low-frequency drift and high-frequency noise are removed through bandpass filtering (0.01–0.1 Hz) and linear detrending. Simultaneously, a confounding variable matrix (including Friston-24 motion parameters, WM / CSF principal components, and optional global signals) is constructed and regression is performed to reduce physiological and motion artifact interference. In the quality control stage, frame shift (FD) and DVARS indices are calculated, and abnormal frames are removed. For registration, the average functional images are affine aligned to the MNI152 standard space (4×4×4 mm resolution), and the transformation is applied to the entire 4D functional data. Finally, standardized time sequences of 116 brain regions are extracted based on the AAL template. As input for subsequent feature extraction and analysis.

[0034] Example 3 This embodiment further defines Embodiment 1 and provides a detailed description of the dual-channel feature coding network module.

[0035] Dual-channel feature encoding network module: for 3D clinical feature data and Feature encoding is performed separately, and the encoded information is fused together to obtain global features through an attention mechanism.

[0036] Clinical feature encoder: Clinical feature data is encoded into shallow features through a two-layer fully connected network. (32-dimensional) and deep features (64 dimensions): = ; = ; in, For clinical feature vectors, The dimension of the clinical feature vector represents the number of input clinical features. , , and These are the learnable parameters. For activation function, Indicates batch normalization; fMRI encoder: 1. Extracting temporal features using one-dimensional convolutional layers: ; in, This indicates the extracted time features. This represents one-dimensional convolution; For batch size, For time step, The hidden layer dimension is 128; the hidden layer dimension H is the number of output channels of the one-dimensional convolutional layer, which is directly defined by the convolutional layer parameters.

[0037] 2. Early fusion: Projecting shallow features into the hidden space and merging them with... Fusion yields the characteristics of the fused product : ; in, , , Represents the projection weight matrix. This represents the sigmoid function. Represents the gate weight matrix. This represents element-wise multiplication; The meaning is: first, use the projection weight matrix. Mapping clinical superficial features to , making its dimensions and Align the 128 dimensions (derived from the 128-dimensional hidden layer of the one-dimensional convolutional layer in the fMRI encoder) with the data; then stitch them together. and through The sigmoid function generates gate weights in the 0-1 interval. ), determine the importance of the two; finally use Weighted 、(1- Weighted The fused features are obtained by multiplying them element by element and then adding them together. This enables dynamic selection and fusion of features from two modalities.

[0038] The hidden space in early fusion is adapted to the fMRI temporal feature matrix. The 128-dimensional feature space originates from the parameter setting of "128 hidden layer dimensions" in the one-dimensional convolutional layer of the fMRI encoder. Dimensions Its core function is to project the weight matrix. Mapping 32-dimensional superficial clinical features to this space achieves [the desired result]. The dimensions are unified to meet the requirements of element-by-element fusion of subsequent gating mechanisms. The design logic comes from the early fusion goal of cross-modal feature alignment.

[0039] 3. Time-based attention mechanism: study Importance weights for each time point: ; ; in, Represents the time attention weight matrix. Represents the normalized exponential function, express Neural networks This represents the fMRI temporal feature matrix after time attention weighting; 4. Spatial attention mechanism: Learning the importance weights between brain regions (ROIs): ; ; ; ; ; Where Q (Qurry, query vector) represents the current fMRI temporal feature (after time attention processing). In spatial attention computation, it serves as a vector representation of the "query," used in conjunction with the "key." Similarity calculations are performed to uncover the associations between features of different brain regions (ROIs), with dimensions similar to... Maintain consistency.

[0040] K (Key, key vector): Corresponding to Q, it is... The transformed "key" vector representation, through similarity calculation with Q, determines the importance contribution of different brain region features to the currently queried brain region features, with dimensions and... Same.

[0041] V (Value, value vector): is The transformed "value" vector representation is used to weight and aggregate feature information from different brain regions based on attention weights calculated from Q and K, ultimately generating features containing spatial correlation information with a dimension of [missing information]. ,in Indicates the number of brain regions.

[0042] (Query weight matrix): is used to... Transformed into a learnable parameter matrix of Q, through matrix multiplication ( To achieve feature transformation, the dimensions must be matched. The output dimension of Q and the target dimension of Q are dynamically optimized during model training based on the spatial attention modeling effect, so as to accurately capture the spatial dependencies between brain regions.

[0043] (Key weight matrix): is the matrix that... Transformed into a learnable parameter matrix of K, through matrix multiplication ( Complete feature mapping, dimension design, and Similarly, the goal is to enable K to effectively perform similarity calculation with Q, and its parameters will be continuously adjusted with training iterations to optimize the weight allocation logic of spatial attention.

[0044] (Value weight matrix): is the matrix that... Transformed into a learnable parameter matrix of V, through matrix multiplication ( To achieve feature transformation, its dimension must ensure that V is compatible with the spatial attention weight matrix.

[0045] Represents the spatial attention weight matrix. This represents the dimension of the key vector (K), which is consistent with the dimension of the query vector (Q), specifically equal to the hidden layer dimension of the fMRI features (taking a value of 128). Divide by This is to alleviate the problem when the dimensionality is large. To address the issue of extreme numerical values ​​in the calculation results, ensure the spatial attention weight matrix... Reasonable weight allocation improves the stability of the model in capturing spatial correlation information between brain regions.

[0046] Let represent the fMRI spatial feature matrix after spatial attention weighting.

[0047] 5. Explicit modeling of brain region importance: Introducing learnable brain region importance parameters Global features are obtained through global average pooling. : ,

[0048] in This indicates global tie pooling.

[0049] Example 4 This embodiment further defines Embodiment 1 and provides a detailed description of the feature fusion and classification module.

[0050] Feature fusion and classification module: The global features are fused to obtain fused features; the fused features are then input into the main task classifier and the domain task classifier respectively. The main task classifier performs the main task to predict whether the test subject has a neurological disease, and the domain task classifier performs the domain task to predict the site to which the test subject belongs.

[0051] Feature fusion module: fuses global fMRI features with deep features.

[0052] Bidirectional attention mechanism: pass Obtain global features attention weight vector ,in, This represents a multilayer perceptron; pass Obtaining deep features attention weight vector ; pass Obtain the weighted global feature matrix ; pass Obtain the weighted deep feature matrix .

[0053] Feature difference learning: pass Obtain the difference feature matrix ; Fusion Enhancement: pass Obtain the fusion feature matrix ; The main task classifier is based on the fused feature matrix. The specific procedures involved in predicting whether a test subject has a neurological disorder are as follows: Feature transformation layer: through Perform feature transformation to obtain the output vector after feature transformation. ;in, For learnable weight matrix, For learnable bias terms; Probability calculation layer: through Calculate the probability that the test subject has ADHD, where express function, This indicates the probability that the test subject is not a patient with a neurological disease. This indicates the probability that the test subject is a patient with a neurological disease. and These are two elements in the output vector z after feature transformation, corresponding to the original predicted values ​​of the two diagnostic results: where The original predicted value corresponding to the "non-neurological disease patients" category, The original predicted value corresponding to the category "patient with neurological disease"; Diagnostic decision-making level: through Make diagnostic decisions. This indicates a predictive result regarding whether the test subject has a neurological disorder. This indicates that the test subject does not suffer from a neurological disease. This indicates that the person being tested has a neurological disorder; Indicates a non-neurological disease. It indicates a neurological disorder.

[0054] The specific operations performed in the domain task classifier are as follows: Gradient inversion layer: through Obtain the multimodal fusion feature matrix ,in This indicates a gradient inversion layer. Indicates the dynamically adjusted gradient reversal strength; During forward propagation, During back propagation ,in, Represents the total loss function. This indicates the operation of calculating partial derivatives; Feature transformation layer: through Obtain the original prediction vector of the site affiliation of the test personnel. ;in, For learnable weight matrix, For learnable bias terms; Site prediction layer: via Obtain the probability distribution vector of the test subjects belonging to each site. ,in Indicates the number of sites; Site decision-makers: Through Obtain the site of the person to be tested ,in This refers to any one of all sites. Indicates taking The site with the largest probability distribution vector.

[0055] Example 5 This embodiment further defines Embodiment 1 and provides a detailed description of the dynamic balance training module.

[0056] Dynamic balancing training module: Optimizes the domain task classifier through backpropagation mechanism to reduce domain task loss; adjusts the dynamic balance between the main task and the domain task through dynamic balancing control strategy.

[0057] The backpropagation mechanism requires forward propagation to occur first, specifically: Forward propagation phase: Domain task classifier receives data. Output the site to which the test subject belongs, and combine it with the real site tag of the test subject to calculate the domain task loss through the cross-entropy loss function; Backpropagation phase: The obtained domain task loss is backpropagated in the domain task classifier, and the parameters of each layer in the domain task classifier are optimized by the optimizer.

[0058] The core logic of backpropagation is error propagation and parameter update.

[0059] Backpropagation is the core mechanism for iterative optimization of models in deep learning. Its basic logic is: First, the model prediction results are obtained through forward propagation (forward computation of data from the input layer to the output layer); Then, the loss function value (i.e. the error magnitude) is calculated based on the difference between the predicted result and the true label. Finally, the loss is propagated back from the output layer to each layer using the "chain rule," the "contribution" (gradient) of each layer's parameters to the total loss is calculated, and the parameters (such as weights and biases) are adjusted according to the gradient to minimize the loss.

[0060] The "chain rule" is a common term in this field (adaptive analysis of cross-modal medical data and deep learning-driven disease diagnosis models), often used in the backpropagation process of multi-layer neural networks. It is a core rule for differentiating multi-layer composite functions, and in this system, it is mainly used to calculate the total loss. For the gradients of learnable parameters (such as weights and biases) at each layer, it supports minimizing the loss by backpropagating the error and adjusting the parameters, which aligns with the core logic of "error backpropagation and parameter update" in the document.

[0061] like Figure 1 As shown, the specific role of backpropagation is to connect the domain task classifier with the domain loss gradient. Figure 1 In the domain adaptation process, backpropagation revolves around the optimization of the domain classifier. Its core function is to transmit the gradient information of the domain task loss and drive the update of model parameters. The specific process is as follows: Forward propagation phase: prediction and loss calculation of the domain task classifier.

[0062] The domain task classifier receives cross-site data after feature extraction and fusion, and outputs the probability distribution of site categories (e.g., predicting the probability that a sample belongs to site A, B, or C). Combining this with the sample's real site labels, it calculates the domain task loss using the cross-entropy loss function. (i.e., the prediction error of the domain classifier, corresponding to) Figure 1 (The "Computational Domain Task Loss" step).

[0063] Backpropagation phase: backpropagation of loss gradient and parameter update.

[0064] Loss gradient generation: The domain task loss (scalar) is differentiated with respect to the output layer parameters of the domain task classifier. These "output layer parameters" correspond to the learnable parameters (weight matrix) in the "feature transformation layer" of the domain task classifier. With bias term The gradient of the output layer parameters is obtained; then, the gradient is backpropagated layer by layer through the chain rule. The gradient here is not just the "gradient of the output layer parameters" obtained above, but the "gradient of all layer parameters of the domain task classifier" (i.e., the complete domain loss gradient) obtained after backpropagating layer by layer through the chain rule, starting from the gradient of the output layer parameters above.

[0065] Calculate the gradients of all layer parameters of the domain task classifier, i.e., the domain loss gradient.

[0066] Parameter update triggering: Backpropagation backpropagates the domain loss gradient to the parameters of each layer of the domain task classifier, and uses an optimizer to adjust these parameters to reduce the domain task loss.

[0067] In summary, Figure 1 The domain task loss in the calculation is calculated based on the site category probability (such as the probability distribution of the predicted site) output by the site prediction step and the true site label of the sample, using loss functions such as cross-entropy. It is used to measure the error of the domain classifier in judging the site affiliation and is one of the bases for subsequent parameter updates.

[0068] Figure 1 Domain performance evaluation, based on the calculated domain task loss, combined with metrics such as prediction accuracy and confusion matrix, assesses the effectiveness of the domain classifier in eliminating cross-site data distribution differences. This provides a basis for dynamically adjusting model training strategies (e.g., ...). (Value) is provided for reference.

[0069] Figure 1The calculation of the main task loss is as follows: The main task classifier takes the "fused and enhanced" features as input and outputs the probability distribution of the sample corresponding to the main task target category (such as "illness" or "health" in disease diagnosis). Then, this probability distribution is compared with the sample's true label (provided by clinical diagnosis annotation, etc.) and calculated using the cross-entropy loss function (a common loss function in classification tasks, which is common knowledge in this field and widely used in medical image diagnosis, machine learning classification tasks, etc.) to obtain the main task loss (scalar), which quantifies the difference between the main task prediction result and the actual situation.

[0070] Figure 1 The performance evaluation of the main task is based on the prediction results (class probability or final class determination) of the main task classifier and the true labels of the samples. It comprehensively evaluates performance using multiple existing technical indicators, including prediction accuracy (number of correctly predicted samples / total number of samples), precision (number of true positives / number of predicted positives), recall (number of true positives / actual number of positives), F1 score (harmonic average of precision and recall), and the generation of a confusion matrix (showing the distribution of true positives, false positives, true negatives, and false negatives). These indicators and calculation methods are widely used in medical image diagnosis and machine learning classification tasks.

[0071] The dynamic balancing control strategy is implemented through a dynamic balancing controller, which dynamically adjusts the performance based on the primary task and the domain tasks. Specifically: Calculate the accuracy of the main task domain task accuracy ; Calculate the gradient norm ratio : ,in, The gradient represents the domain task loss. The gradient represents the loss of the main task. This indicates the L2 norm operation. As an adjustment factor, when >70% ,otherwise ; Smooth Update : ,in, Indicates the state before the update , Indicates the updated , It is a smoothing factor; Boundary constraints: , , ,in, This is a clipping function, indicating that the updated... The value is limited to and Within the range.

[0072] This embodiment also provides a method for training the system, using a cross-entropy loss function with class weights to address the sample imbalance problem: ,in, Indicates loss of the main task. Indicates the sample class weights. Indicates sample Real neurological disease labels This indicates that the main task classifier classifies the samples. Predictive probability of having a neurological disease; Total loss function: ,in, =1.2 is the primary task priority coefficient. This indicates the loss of domain tasks.

[0073] Gradient clipping is used during training to prevent gradient explosion. , These are constraints used to limit the magnitude of parameter gradient updates. The meanings of the variables are as follows: This represents the set of learnable parameters in the model, encompassing the weights of the main task classifier mentioned earlier. Domain classifier weights The weights of fully connected layers in feature encoders (such as clinical feature encoders and fMRI feature encoders) are core parameters that the model continuously optimizes through training.

[0074] Represents learnable parameters The gradient vector of the learnable parameters The set of partial derivatives reflects the learnable parameters. The trend of how changes affect the loss value is a key basis for parameter updates.

[0075] This refers to the L2 norm (Euclidean norm) operation on a vector, used to calculate the gradient vector. The magnitude of the gradient, i.e. the overall size of the gradient, can quantify the absolute magnitude of the gradient.

[0076] : This represents the threshold for the gradient magnitude, which is set to 1.0 in the file. Its function is to limit the maximum magnitude of gradient updates, so as to avoid excessive parameter updates due to excessively large gradient magnitudes, which could lead to unstable model training (such as training oscillations, gradient explosion) and other problems, and ensure a smooth parameter update process.

[0077] Training strategy: Use AdamW optimizer, learning rate Weight decay Batch size 4, training rounds 300.

[0078] Example 6 This embodiment further defines Embodiment 1, providing specific parameter configurations for each module in the system.

[0079] Data preprocessing parameters: Spatial smoothing kernel size: 6 mm FWHM Gaussian kernel; Temporal filtering range: 0.01–0.1Hz; ROI extraction template: AAL116 template; Regression of confounding variables: Friston-24 motion parameters + WM / CSF principal components + global signal (optional); Space registration: 4×4×4 mm MNI152 standard space.

[0080] Feature encoding network parameters: Clinical feature encoder: two-layer fully connected network (3D input → 32D shallow layer → 64D deep layer), batch normalization (BN) and ReLU activation; fMRI feature encoder: one-dimensional convolution (kernel size = 5, stride = 1, hidden units = 128) to extract time series features; Temporal attention mechanism: time point weight allocation based on Softmax; Spatial attention mechanism: brain region importance matrix based on learnable parameters. .

[0081] Early fusion strategy: After mapping clinical superficial features to the hidden space, fuse them with fMRI features element by element.

[0082] Feature fusion and classification: A bidirectional attention mechanism enables bidirectional interaction between clinical features and fMRI features; the main task classifier uses cross-entropy loss (class weight balancing to handle imbalanced samples); the domain classifier uses gradient inversion layer (GRL) to achieve adversarial domain adaptation.

[0083] Dynamic balancing training strategy: During training, the accuracy of the primary task and the accuracy of the domain task are calculated in real time, and the balancing coefficient is dynamically adjusted according to the ratio of their gradient norms. ; When the accuracy of the main task is >70%, increase the domain adaptive weight; otherwise, decrease it. Smoothing factor The gradient pruning threshold is 5.0, and the optimizer is AdamW (learning rate 1e-4, weight decay 1e-5).

[0084] Experimental results: On the multi-site ADHD dataset, the test set accuracy reached 97.7%, which is about 7.3 percentage points higher than the domain-independent adaptive model; the model AUC improved to 0.96 and the F1-score improved to 0.952, demonstrating excellent cross-site generalization ability; ROI importance analysis shows that the present invention can automatically identify brain regions significantly related to ADHD and has good interpretability.

[0085] The core technical objective of site prediction (domain task) is to eliminate cross-site data distribution differences through "domain adaptive adversarial learning," providing "domain-invariant features" to support neurological disease prediction (main task). Essentially, it enhances the cross-site generalization ability of the main task through "reverse constraints." The specific technical logic is as follows: 1. Root Cause: "Distribution Shift" in Cross-Site Data Interferes with the Primary Task. Differences in scanning equipment (e.g., 3.0T / 1.5T MRI), sequence parameters (e.g., TR / TE values), and participant populations (e.g., age distribution, proportion of underlying diseases) across different sites can lead to "site-specific distributions" in fMRI data and clinical features (e.g., brain region activity intensity is generally higher in site A data and lower in site B data). If such data is directly used to train a disease prediction model, the model is prone to "overfitting to site features" (e.g., mistakenly treating the specific intensity of site A as a disease feature), resulting in a significant drop in diagnostic accuracy on new site data.

[0086] 2. Technical Logic: The system forces the model to learn core disease characteristics through "site prediction adversarial" mechanisms. In this system, site prediction is not an independent task, but rather forms an "adversarial collaborative" relationship with the main task. The goal of a domain task classifier (site prediction) is to "accurately distinguish the site to which a sample belongs", which requires capturing "site-specific features" in the data (such as signal differences caused by devices). The feature extraction module (dual-channel encoding fusion module) under the action of the gradient inversion layer (GRL) must simultaneously satisfy two inverse constraints: (1) enable the main task classifier to accurately diagnose diseases based on its output features (retain "disease-related features"); (2) make it difficult for the domain task classifier to distinguish sites based on its output features (suppress site-specific features). Ultimately, the feature fusion and classification module will be forced to "filter out site-specific interference" and retain only "core features that are not affected by the site and are directly related to the disease" (i.e., "domain-invariant features"), thereby providing more robust feature input for disease prediction (the main task) and solving the problem of "poor generalization ability" of traditional models.

Claims

1. A neurological disease diagnostic system based on spatiotemporal attention and dynamic domain adaptation, characterized in that, The system includes: Data preprocessing module: Collects clinical characteristic data and fMRI time-series data of test subjects from different sites, preprocesses the fMRI time-series data, and extracts data from the fMRI time-series data. Standardized time series of brain regions ; Dual-channel feature coding network module: for clinical feature data and Feature encoding is performed separately, and the encoded information is fused together to obtain global features through an attention mechanism; Feature fusion and classification module: The global features are fused to obtain fused features; the fused features are then input into the main task classifier and the domain task classifier respectively. The main task classifier performs the main task to predict whether the test subject has a neurological disease, and the domain task classifier performs the domain task to predict the site to which the test subject belongs. Dynamic balancing training module: Optimizes the domain task classifier through backpropagation mechanism to reduce domain task loss; adjusts the dynamic balance between the main task and the domain task through dynamic balancing control strategy.

2. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 1, characterized in that, The clinical feature data refers to the identity information of the test subjects, which is not imaging information data.

3. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 2, characterized in that, Feature encoding of clinical feature data specifically involves encoding the clinical feature data into shallow features through a two-layer fully connected network. and deep features : = ； = ； in, For clinical feature vectors, , , and These are the learnable parameters. For activation function, Indicates batch normalization; right Feature encoding specifically involves using a one-dimensional convolutional layer to extract temporal features: ； in, This indicates the extracted time features. This represents one-dimensional convolution; The fusion of encoded information specifically involves projecting shallow features and combining them with... Fusion yields the characteristics of the fused product : ； in, , , Represents the projection weight matrix. This represents the sigmoid function. Represents the gate weight matrix. This indicates element-wise multiplication.

4. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 3, characterized in that, Obtaining global features through an attention mechanism specifically involves learning via a temporal attention mechanism. The importance weights of each time point are assigned to obtain the fMRI temporal feature matrix after temporal attention weighting. By learning the importance weights between brain regions through spatial attention mechanisms, an fMRI spatial feature matrix weighted by spatial attention weights is obtained. By introducing a learnable brain region importance parameter R, global features are obtained through global average pooling. : ， ， in This indicates global tie pooling.

5. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 4, characterized in that, The global features are fused to obtain the fused features as follows: pass Obtain global features attention weight vector ,in, This represents a multilayer perceptron; pass Obtaining deep features attention weight vector ; pass Obtain the weighted global feature matrix ; pass Obtain the weighted deep feature matrix ; pass Obtain the difference feature matrix ; pass Obtain the fusion feature matrix .

6. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 5, characterized in that, The specific operations performed in the main task classifier are as follows: Feature transformation layer: through Perform feature transformation to obtain the output vector after feature transformation. ;in, For learnable weight matrix, For learnable bias terms; Probability calculation layer: through Calculate the probability that the test subject has a neurological disease, where express function, This indicates the probability that the test subject is not a patient with a neurological disease. This indicates the probability that the test subject is a patient with a neurological disease. and It is the output vector after feature transformation. The two elements in the table correspond to the original predicted values ​​of the two diagnostic results: where The original predicted value corresponding to the "non-neurological disease patients" category, The original predicted value corresponding to the category "patient with neurological disease"; Diagnostic decision-making level: through Make diagnostic decisions. This indicates a predictive result regarding whether the test subject has a neurological disorder. This indicates that the test subject does not suffer from a neurological disease. This indicates that the person being tested has a neurological disorder. Indicates a non-neurological disease. It indicates a neurological disorder.

7. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 6, characterized in that, The specific operations performed in the domain task classifier are as follows: Gradient inversion layer: through Obtain the multimodal fusion feature matrix ,in This indicates a gradient inversion layer. Indicates the dynamically adjusted gradient reversal strength; During forward propagation, During back propagation ,in, This represents the total loss function for the main task classifier and the domain task classifier. This indicates the operation of calculating partial derivatives; Feature transformation layer: through Obtain the original prediction vector of the site affiliation of the test personnel. ;in, For learnable weight matrix, For learnable bias terms; Site prediction layer: via Obtain the probability distribution vector of the test subjects belonging to each site. ,in Indicates the number of sites; Site decision-makers: Through Obtain the site of the person to be tested ,in This refers to any one of all sites. Indicates taking The site with the largest probability distribution vector.

8. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 7, characterized in that, The backpropagation mechanism requires forward propagation to occur first, specifically: Forward propagation phase: Domain task classifier receives data. Output the site to which the test subject belongs, and combine it with the real site tag of the test subject to calculate the domain task loss through the cross-entropy loss function; Backpropagation phase: The obtained domain task loss is backpropagated in the domain task classifier, and the parameters of each layer in the domain task classifier are optimized by the optimizer.

9. The spatiotemporal attention and dynamic domain adaptive neurological disease diagnostic system according to claim 7, characterized in that, The dynamic balancing control strategy is implemented through a dynamic balancing controller, which dynamically adjusts the performance based on the primary task and the domain tasks. Specifically: Calculate the accuracy of the main task domain task accuracy ; Calculate the gradient norm ratio : ,in, The gradient represents the domain task loss. The gradient represents the loss of the main task. This indicates the L2 norm operation. As an adjustment factor, when >70% ,otherwise ; Smooth Update : ,in, Indicates the state before the update , Indicates the updated , It is a smoothing factor; Boundary constraints: , , ,in, This is a clipping function, indicating that the updated... The value is limited to and Within the range.

Images