Neurological disease diagnosis system based on spatio-temporal attention and dynamic domain adaptation

Abstract

A neural disease diagnosis system based on space-time attention and dynamic domain adaptation belongs to the technical field of cross-modal medical data adaptive analysis. It solves the technical problem that there is an urgent need for a new system that can simultaneously fuse multi-modal information, model multi-scale space-time features, and have dynamic domain adaptation capability to improve the accuracy and generalization ability of multi-site neural disease intelligent diagnosis. The system comprises: a data preprocessing module for extracting standardized time series of individual brain regions from fMRI time series data; a dual-channel feature encoding network module for obtaining global features through an attention mechanism; a feature fusion and classification module, wherein a main task classifier performs a main task to predict whether a testee has a neural disease, and a domain task classifier performs a domain task to predict the site to which the testee belongs; and a dynamic balance training module for adjusting the dynamic balance of the main task and the domain task through a dynamic balance control strategy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of cross-modal medical data adaptive analysis, and particularly relates to a neural disease diagnosis system based on spatiotemporal attention and dynamic domain adaptation. BACKGROUND

[0002] Resting-state functional magnetic resonance imaging (rs-fMRI) can non-invasively record the spontaneous neural activity patterns of the whole brain and has been widely used in mechanism research and auxiliary diagnosis of neurodevelopmental diseases (such as attention deficit hyperactivity disorder, ADHD). However, the existing technology has the following deficiencies in multi-site fMRI data analysis:

[0003] Significant differences in cross-site data distribution: differences in scanning equipment, sequence parameters and subject population used by different medical institutions result in inconsistent data distribution, causing a significant decrease in model generalization ability.

[0004] Insufficient single-modal feature information: traditional methods mostly rely on single image features, making it difficult to fully utilize clinical questionnaires, behavior scales and other non-imaging information, thus limiting the diagnostic accuracy.

[0005] Insufficient use of spatiotemporal features: most methods model in a single time dimension or spatial dimension, failing to simultaneously capture the dynamic changes of brain activity over time and the spatial connection patterns between brain regions.

[0006] Lack of dynamic training strategies: existing domain adaptation methods mostly use fixed weights, which cannot dynamically adjust the balance between the main task and the domain task according to the task performance during training, resulting in unstable model training.

[0007] Therefore, there is an urgent need for a new system that can simultaneously fuse multi-modal information, model multi-scale spatiotemporal features and have dynamic domain adaptation capability to improve the accuracy and generalization ability of multi-site neural disease intelligent diagnosis. SUMMARY

[0008] To solve the technical problem of the urgent need for a new system that can simultaneously fuse multi-modal information, model multi-scale spatiotemporal features and have dynamic domain adaptation capability to improve the accuracy and generalization ability of multi-site neural disease intelligent diagnosis, the present application provides a neural disease diagnosis system based on spatiotemporal attention and dynamic domain adaptation, which comprises:

[0009] A data preprocessing module: collects clinical feature data and fMRI time series data of test personnel at different sites, and preprocesses the fMRI time series data to extract standardized time series of individual brain regions from the fMRI time series data.

[0010] The dual-channel feature encoding network module: the clinical feature data and the imaging feature data are respectively encoded, and the encoded information is fused to obtain global features through an attention mechanism.

[0011] The feature fusion and classification module: the global features are fused to obtain fused features; the fused features are respectively input into a main task classifier and a domain task classifier; the main task classifier performs a main task to predict whether the to-be-tested person has a neurological disease, and the domain task classifier performs a domain task to predict the site to which the to-be-tested person belongs.

[0012] The dynamic balance training module: the domain task classifier is optimized through a back propagation mechanism to reduce the domain task loss; and a dynamic balance control strategy is used to adjust the dynamic balance between the main task and the domain task.

[0013] Further, the clinical feature data is identity information data of the to-be-tested person, which is non-imaging information data.

[0014] Further, the feature encoding of the clinical feature data specifically includes encoding the clinical feature data into shallow features and deep features through a two-layer fully connected network.

[0015]

[0016]

[0017] wherein, is a clinical feature vector, , , and are learnable parameters, is an activation function, and denotes batch normalization.

[0018] The feature encoding of the imaging feature data specifically includes extracting time features using a one-dimensional convolution layer.

[0019]

[0020] wherein, denotes the extracted time features, and denotes one-dimensional convolution.

[0021] The fusion of the encoded information specifically includes projecting the shallow features and fusing the projected shallow features with the deep features to obtain fused features. ​​​​​​​​​​​

[0022] ;

[0023] in, , , Represents the projection weight matrix. This represents the sigmoid function. Represents the gate weight matrix. This indicates element-wise multiplication.

[0024] Furthermore, obtaining global features through the attention mechanism specifically involves learning through a temporal attention mechanism. The importance weights of each time point are assigned to obtain the fMRI temporal feature matrix after temporal attention weighting. By learning the importance weights between brain regions through spatial attention mechanisms, an fMRI spatial feature matrix weighted by spatial attention weights is obtained. By introducing a learnable brain region importance parameter R, global features are obtained through global average pooling. :

[0025] , ,

[0026] in This indicates global tie pooling.

[0027] Furthermore, the global features are fused to obtain the fused features, specifically as follows:

[0028] pass Obtain global features attention weight vector ,in, This represents a multilayer perceptron;

[0029] pass Obtaining deep features attention weight vector ;

[0030] pass Obtain the weighted global feature matrix ;

[0031] pass Obtain the weighted deep feature matrix ;

[0032] pass Obtain the difference feature matrix ;

[0033] pass Obtain the fusion feature matrix .

[0034] Further, the operations performed in the main task classifier are specifically:

[0035] The feature transformation layer: by performing feature transformation, obtaining the output vector z after feature transformation ; wherein, is a learnable weight matrix, is a learnable bias term;

[0036] The probability calculation layer: by calculating the probability of the to-be-tested person suffering from a neurological disease, wherein represents a function, represents the probability of outputting a to-be-tested person who is not a neurological disease patient, represents the probability of outputting a to-be-tested person who is a neurological disease patient; and are two elements in the output vector z after feature transformation, respectively corresponding to the original prediction values of the two diagnostic results: wherein corresponds to the original prediction value of the “non-neurological disease patient” category, corresponds to the original prediction value of the “neurological disease patient” category;

[0037] The diagnostic decision layer: by making a diagnostic decision, represents the prediction result of whether the to-be-tested person suffers from a neurological disease, represents that the to-be-tested person does not suffer from a neurological disease, represents that the to-be-tested person suffers from a neurological disease, represents non-neurological disease, represents neurological disease.

[0038] Further, the operations performed in the domain task classifier are specifically:

[0039] The gradient inversion layer: by obtaining the multi-modal fusion feature matrix , wherein represents the gradient inversion layer, represents the dynamically adjusted gradient inversion strength;

[0040] During forward propagation, , during back propagation , wherein, represents the total loss function of the main task classifier and the domain task classifier, represents the partial derivative operation;

[0041] The feature transformation layer: by obtain an original prediction vector of the site attribution of the to-be-tested personnel ; wherein, is a learnable weight matrix, is a learnable bias term;

[0042] site prediction layer: through obtain a probability distribution vector of the site attribution of the to-be-tested personnel , wherein represents the number of sites;

[0043] site decision layer: through obtain the site attribution of the to-be-tested personnel , wherein represents any of all sites, represents the site with the maximum probability distribution vector .

[0044] Further, the premise of the back propagation mechanism is to perform forward propagation first, specifically:

[0045] forward propagation stage: the domain task classifier receives , outputs the site attribution of the to-be-tested personnel, and calculates the domain task loss through the cross-entropy loss function combined with the real site label of the to-be-tested personnel;

[0046] back propagation stage: the domain task loss obtained is back propagated in the domain task classifier, and each layer parameter in the domain task classifier is optimized through the optimizer.

[0047] Further, the dynamic balance control strategy is performed through a dynamic balance controller, which dynamically adjusts according to the performance of the main task and the domain task, specifically:

[0048] calculate the main task accuracy and the domain task accuracy ;

[0049] calculate the gradient norm ratio : , wherein, represents the gradient of the domain task loss, represents the gradient of the main task loss, represents the calculation of the L2 norm operation, is an adjustment factor, when >70%, otherwise ;

[0050] smooth update : , wherein, represents the value before updating , represents the value after updating , is a smoothing factor;

[0051] Boundary constraints: , , wherein, is a clipping function, which limits the value of the updated to the range of and .

[0052] The system has the following advantages:

[0053] In view of the problem of significant differences in cross-site data distribution: the use of different scanning equipment, sequence parameters and subject populations by different medical institutions leads to inconsistent data distribution, resulting in a significant decrease in model generalization ability.

[0054] The system effectively solves the problem of inconsistent cross-site data distribution through dynamic adversarial learning of the domain task (site prediction) and the main task (disease prediction).

[0055] In view of the problem of insufficient single modal feature information: traditional methods rely mainly on single image features, making it difficult to fully utilize non-imaging information such as clinical questionnaires and behavior scales, resulting in limited diagnostic accuracy.

[0056] The system fully utilizes imaging (fMRI) and non-imaging (clinical feature) modal information through "dual-channel feature encoding + bidirectional attention fusion", breaking through the limitations of traditional single modal.

[0057] In view of the problem of insufficient spatiotemporal feature utilization: most methods model in a single dimension in the time or space dimension, failing to capture the dynamic changes of brain activity in the time series and the spatial connection patterns between brain regions.

[0058] The system realizes complete capture of brain activity spatiotemporal features through "hierarchical spatiotemporal attention + brain region importance modeling", solving the defects of traditional "single-dimensional modeling".

[0059] In view of the lack of dynamic training strategies: existing domain adaptation methods mostly use fixed weights, which cannot dynamically adjust the balance between the main task and the domain task according to the task performance during training, leading to unstable model training.

[0060] The system replaces the traditional fixed weight training with a "dynamic balance control strategy" to solve the problem of unstable model training. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 Workflow diagram of the neural disease diagnosis system in the embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0063] Embodiment 1,

[0064] The present embodiment provides a neural disease diagnosis system based on spatio-temporal attention and dynamic domain adaptation, which comprises:

[0065] The data preprocessing module collects 3D clinical feature data and fMRI time series data of the test personnel in different sites, and pre-processes the fMRI time series data to extract the standardized time series of brain regions from the fMRI time series data.

[0066] The dual-channel feature encoding network module respectively encodes the 3D clinical feature data and the fMRI time series data, fuses the encoded information, and obtains global features through an attention mechanism.

[0067] The feature fusion and classification module fuses the global features to obtain fusion features, and inputs the fusion features into a main task classifier and a domain task classifier. The main task classifier performs a main task to predict whether the test personnel has a neural disease, and the domain task classifier performs a domain task to predict the site to which the test personnel belongs.

[0068] The dynamic balance training module optimizes the domain task classifier through a back propagation mechanism to reduce the domain task loss, and adjusts the dynamic balance of the main task and the domain task through a dynamic balance control strategy.

[0069] The workflow diagram of the system is shown in Figure 1 The subsequent embodiments will be described in detail in conjunction with Figure 1 the workflow of the system.

[0070] Embodiment 2,

[0071] The present embodiment is a further limitation of embodiment 1, and specifically introduces the data preprocessing module.

[0072] ​​​The data preprocessing module collects 3D clinical feature data and fMRI time series data of the to-be-tested personnel at different sites, and pre-processes the fMRI time series data to extract standardized time series of brain regions from the fMRI time series data . .

[0073] The data preprocessing module collects fMRI time series data and clinical feature data of the to-be-tested personnel from multiple sites.

[0074] The clinical feature data is identity information data of the to-be-tested personnel, such as age, gender, education, and other non-imaging information data.

[0075] The site is the data source of the to-be-tested personnel, for example, the fMRI time series data and clinical feature data of the to-be-tested personnel are obtained through 20 different medical departments, and the 20 different medical departments are 20 sites.

[0076] The process of pre-processing the fMRI time series data is as follows: first, convert the original image to standard NIfTI format and re-orient, if necessary, perform slice timing correction; then use FSL's MCFLIRT to correct head motion for whole brain EPI sequence, and generate brain mask by skull stripping the average functional image; apply a Gaussian smoothing of about 6 mm FWHM in space to improve signal-to-noise ratio, and remove low-frequency drift and high-frequency noise by band-pass filtering (0.01-0.1 Hz) and linear detrending in time; at the same time, construct a mixed variable matrix (including Friston-24 motion parameters, WM / CSF principal components, and optional global signal) and perform regression to reduce physiological and motion artifact interference; quality control section calculates frame displacement (FD) and DVARS index and performs abnormal frame rejection; in the registration aspect, the average functional image is affine aligned to the MNI152 standard space (4x4x4 mm resolution), and the transformation is applied to the entire 4D functional data; finally, based on the AAL template, standardized time series of 116 brain regions are extracted as input for subsequent feature extraction and analysis.

[0077] Embodiment 3,

[0078] This embodiment is a further limitation of embodiment 1, and specifically introduces the dual-channel feature encoding network module.

[0079] The dual-channel feature encoding network module: respectively encodes the 3D clinical feature data and the standardized time series of 116 brain regions, and fuses the encoded information to obtain global features through an attention mechanism.

[0080] Clinical feature encoder: ​

[0081] Clinical feature data is encoded by a two-layer fully connected network into shallow features (32-dimensional) and deep features (64-dimensional):

[0082]

[0083]

[0084] wherein, is the clinical feature vector, denotes the dimension of the clinical feature vector, i.e., the number of input clinical features;

[0085] and are learnable parameters, is an activation function, denotes batch normalization;

[0086] fMRI encoder:

[0087] 1. Extract temporal features using a one-dimensional convolutional layer:

[0088]

[0089] wherein, denotes the extracted temporal features, denotes one-dimensional convolution; is the batch size, is the time step, is the hidden layer dimension 128; the hidden layer dimension H is the number of output channels of the one-dimensional convolutional layer, which is directly defined by the convolutional layer parameters.

[0090] 2. Early fusion: project the shallow features into a hidden space and fuse them with the to obtain the fused features

[0091]

[0092] wherein, , , denotes the projection weight matrix, denotes the sigmoid function, denotes the gating weight matrix, denotes element-wise multiplication;

[0093] ​​​​​​​​​The meaning of the representation is: first, use the projection weight matrix to map the 32-dimensional clinical shallow feature into the 128-dimensional space , whose dimension is derived from the parameter setting of the fMRI encoder one-dimensional convolution layer "hidden layer dimension 128" ; then concatenate and ; The sigmoid function generates a gating weight in the 0-1 interval , which determines the importance of the two; finally, use to weight , (1- ) to weight , and add them after element-wise multiplication processing to obtain the fusion feature , which realizes the dynamic selection and fusion of the two modal features.

[0094] The hidden space in early fusion is a 128-dimensional feature space that adapts to the fMRI time feature matrix dimension, whose dimension is derived from the parameter setting of the fMRI encoder one-dimensional convolution layer "hidden layer dimension 128" dimension is , and the core role is to map the 32-dimensional clinical shallow feature to this space through the projection weight matrix , to realize the dimensional unity with , and meet the needs of subsequent element-wise fusion of the gating mechanism, and the design logic comes from the early fusion goal of cross-modal feature alignment.

[0095] 3. Time attention mechanism:

[0096] Learn the importance weight of each time point in :

[0097] ;

[0098] ;

[0099] where denotes the time attention weight matrix, denotes the normalized exponential function, denotes the neural network, denotes the fMRI time feature matrix after time attention weight distribution;

[0100] 4. Spatial attention mechanism:

[0101] Learn the importance weight between brain regions (ROI):

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] where, Q (Qurry, query vector) represents the current fMRI time characteristics (after time attention processing) ) in the spatial attention calculation as the vector representation of the "query", used for similarity calculation with the "key ( )", to mine the association relationship between different brain regions (ROI) characteristics, which has the same dimension as .

[0108] K (Key, key vector): corresponding to Q, is the "key" vector representation after transformation, through the similarity calculation result with Q, to determine the importance contribution of different brain region characteristics to the current query brain region characteristics, with the same dimension as .

[0109] V (Value, value vector): is the "value" vector representation after transformation, used to aggregate the feature information of different brain regions according to the attention weight calculated by Q and K, to finally generate the features containing spatial correlation information, with the dimension of , where represents the number of brain regions.

[0110] (query weight matrix): is a learnable parameter matrix used to transform Q into Q, which realizes feature transformation through matrix multiplication ( ), its dimension needs to match the output dimension of and the target dimension of Q, which will be dynamically optimized in the model training process according to the spatial attention modeling effect, to accurately capture the spatial dependence between brain regions.

[0111] (key weight matrix): is a learnable parameter matrix that transforms K into K, which completes feature mapping through matrix multiplication ( ), with the same dimension as , the purpose is to make K can effectively calculate the similarity with Q, its parameters will be adjusted constantly with training iteration, to optimize the weight distribution logic of spatial attention. ​​​​

[0112] (Value weight matrix): is a matrix that... Transformed into a learnable parameter matrix of V, through matrix multiplication ( To achieve feature transformation, its dimension must ensure that V is compatible with the spatial attention weight matrix.

[0113] Represents the spatial attention weight matrix. This represents the dimension of the key vector (K), which is consistent with the dimension of the query vector (Q), specifically equal to the hidden layer dimension of the fMRI features (taking a value of 128). Divide by This is to alleviate the problem when the dimensionality is large. To address the issue of extreme numerical values ​​in the calculation results, ensure the spatial attention weight matrix... Reasonable weight allocation improves the stability of the model in capturing spatial correlation information between brain regions.

[0114] Let represent the fMRI spatial feature matrix after spatial attention weighting.

[0115] 5. Explicit modeling of brain region importance:

[0116] Introducing learnable brain region importance parameters Global features are obtained through global average pooling. :

[0117] ,

[0118] in This indicates global tie pooling.

[0119] Example 4

[0120] This embodiment further defines Embodiment 1 and provides a detailed description of the feature fusion and classification module.

[0121] Feature fusion and classification module: The global features are fused to obtain fused features; the fused features are then input into the main task classifier and the domain task classifier respectively. The main task classifier performs the main task to predict whether the test subject has a neurological disease, and the domain task classifier performs the domain task to predict the site to which the test subject belongs.

[0122] Feature fusion module: fuses global fMRI features with deep features.

[0123] Bidirectional attention mechanism:

[0124] pass Obtain global features attention weight vector wherein, denotes a multi-layer perceptron;

[0125] by obtaining a deep feature attention weight vector ;

[0126] by obtaining a weighted global feature matrix ;

[0127] by obtaining a weighted deep feature matrix .

[0128] feature discrepancy learning:

[0129] by obtaining a discrepancy feature matrix ;

[0130] fusion enhancement:

[0131] by obtaining a fusion feature matrix ;

[0132] a main task classifier predicts whether the to-be-tested person has a neurological disease based on the fusion feature matrix , and the operations performed are specifically:

[0133] a feature transformation layer: by performing feature transformation to obtain a feature-transformed output vector ; wherein, is a learnable weight matrix, is a learnable bias term;

[0134] a probability calculation layer: by calculating the probability that the to-be-tested person has ADHD, wherein denotes a function, denotes the probability of outputting that the to-be-tested person is a non-neurological disease patient, denotes the probability of outputting that the to-be-tested person is a neurological disease patient; and are two elements in the feature-transformed output vector z, which respectively correspond to the original prediction values of the two diagnosis results: wherein corresponds to the original prediction value of the “non-neurological disease patient” category, corresponds to the original prediction value of the “neurological disease patient” category;

[0135] a diagnosis decision layer: by performing diagnosis decision, a prediction result indicating whether the to-be-tested person has a neurological disease, a prediction result indicating that the to-be-tested person does not have a neurological disease, a prediction result indicating that the to-be-tested person has a neurological disease; a prediction result indicating a non-neurological disease, a prediction result indicating a neurological disease.

[0136] The operations performed in the domain task classifier are specifically:

[0137] a gradient reversal layer: by obtaining a multi-modal fusion feature matrix , wherein denotes the gradient reversal layer, denotes a dynamically adjusted gradient reversal strength;

[0138] During forward propagation, , during back propagation , wherein denotes a total loss function, denotes a partial derivative operation;

[0139] a feature transformation layer: by obtaining an original prediction vector of a site to which the to-be-tested person belongs ; wherein is a learnable weight matrix, is a learnable bias term;

[0140] a site prediction layer: by obtaining a probability distribution vector of sites to which the to-be-tested person belongs , wherein denotes the number of sites;

[0141] a site decision layer: by obtaining a site to which the to-be-tested person belongs , wherein denotes any one of all sites, denotes a site with the maximum probability distribution vector.

[0142] Embodiment 5,

[0143] This embodiment is a further limitation of Embodiment 1, and specifically introduces a dynamic balance training module.

[0144] a dynamic balance training module: by a back propagation mechanism, the domain task classifier is optimized to reduce the domain task loss; by a dynamic balance control strategy, the dynamic balance of the main task and the domain task is adjusted.

[0145] ​The premise of the backpropagation mechanism is to perform forward propagation first, specifically:

[0146] Forward propagation phase: the domain task classifier receives , outputs the testee's belonging site, combines the testee's real site label, and calculates the domain task loss through the cross-entropy loss function;

[0147] Backpropagation phase: backpropagation is performed in the domain task classifier through the obtained domain task loss, and the parameters in the domain task classifier are optimized through the optimizer.

[0148] The core logic of backpropagation: error backpropagation and parameter update.

[0149] Backpropagation is the core mechanism of model iteration optimization in deep learning, and its basic logic is:

[0150] First, the model prediction result is obtained through forward propagation (forward calculation of data from the input layer to the output layer);

[0151] Then, the loss function value (i.e. error size) is calculated based on the difference between the prediction result and the real label;

[0152] Finally, through the "chain rule", the loss is transmitted from the output layer to each layer, the "contribution degree" (gradient) of each layer parameter to the total loss is calculated, and the parameters (such as weights and biases) are adjusted according to the gradient to minimize the loss.

[0153] The "chain rule" is a general term in the field (cross-modal medical data adaptive analysis and deep learning driven disease diagnosis model field), commonly used in the backpropagation process of multi-layer neural networks. It is the core rule of the derivative of a multi-layer composite function, and in this system, it is mainly used to calculate the gradient of each layer of learnable parameters (such as weights and biases) to the total loss , support to minimize the loss through backpropagation error and parameter adjustment, consistent with the core logic of error backpropagation and parameter update in the document.

[0154] As shown in Figure 1 , the specific role of backpropagation is to connect the domain task classifier and the domain loss gradient, and in the field adaptive process, backpropagation revolves around the optimization of the domain classifier, and the core is to transmit the gradient information of the domain task loss to drive the model parameter update, and the specific process is as follows: Figure 1 Forward propagation phase: prediction and loss calculation of the domain task classifier.

[0155]

[0156] ​The field task classifier receives the cross-site data after feature extraction, fusion, etc., and outputs a site category probability distribution (e.g., the probability that the sample belongs to site A, B, or C); combined with the real site label of the sample, the field task loss is calculated through the cross-entropy loss function (i.e., the prediction error of the field classifier, corresponding Figure 1 to the "calculate field task loss" step in the field performance evaluation).

[0157] Backpropagation phase: backpropagation of loss gradient and parameter update.

[0158] Generation of loss gradient: derivation of field task loss (scalar) with respect to the output layer parameters of the field task classifier. The "output layer parameters" correspond to the learnable parameters (weight matrix and bias term ) in the "feature transformation layer" of the field task classifier. The gradient of the output layer parameters is obtained. Then, through the chain rule, the gradient is backpropagated layer by layer. Here, the gradient is not only the "gradient of the output layer parameters" obtained above, but also the "gradient of all layer parameters of the field task classifier" (i.e., the complete field loss gradient) obtained by backpropagating the output layer parameter gradient through the chain rule.

[0159] Calculate the gradient of all layer parameters of the field task classifier, i.e., the field loss gradient.

[0160] Trigger parameter update: the field loss gradient is backpropagated to the parameters of each layer of the field task classifier, and the optimizer is used to adjust these parameters to reduce the field task loss.

[0161] In summary, Figure 1 the calculation of the field task loss in the field performance evaluation: based on the site category probability output by the site prediction step (e.g., the probability distribution of predicting a certain site) and the real site label of the sample, the field task loss is calculated through the cross-entropy loss function, which is used to measure the error of the field classifier in judging the site attribution and is one of the bases for subsequent parameter update.

[0162] Figure 1 The field performance evaluation in the field performance evaluation: based on the results of calculating the field task loss, combined with the prediction accuracy, confusion matrix, etc., the effect of the field classifier in eliminating the distribution difference of cross-site data is evaluated, which provides reference for dynamically adjusting the model training strategy (such as value).

[0163] Figure 1The main task loss in the calculation is: the main task classifier takes the feature after "fusion enhancement" as input, and outputs the probability distribution of the sample corresponding to the main task target category (such as "disease" in disease diagnosis) through the main task classifier; Then, the probability distribution and the real label of the sample (provided by clinical diagnosis annotation, etc.) are calculated through the cross-entropy loss function (a commonly used loss function in classification tasks, which is commonly known in the art and widely used in medical image diagnosis, machine learning classification tasks, etc. Scene) to obtain the main task loss (scalar), which quantifies the difference between the main task prediction result and the real situation.

[0164] Figure 1 The main task performance evaluation in the calculation is: based on the prediction result (category probability or final judgment category) of the main task classifier and the real label of the sample, the performance is comprehensively evaluated through the existing technical indicators in multiple dimensions: including prediction accuracy (correctly predicted sample number / total sample number), precision (true positive number / predicted positive number), recall (true positive number / actual positive number), F1 score (harmonic mean of precision and recall), and confusion matrix (showing the distribution of true positives, false positives, true negatives, and false negatives). These indicators and calculation methods are widely used in medical image diagnosis, machine learning classification tasks.

[0165] The dynamic balance control strategy is implemented through a dynamic balance controller, which dynamically adjusts the performance of the main task and the domain task , specifically:

[0166] Calculate the main task accuracy and the domain task accuracy ;

[0167] Calculate the gradient norm ratio : , wherein represents the gradient of the domain task loss, represents the gradient of the main task loss, represents the L2 norm operation, is an adjustment factor, when >70%, , otherwise ;

[0168] Smooth update : , wherein represents the before updating, represents the after updating, is a smoothing factor;

[0169] Boundary constraint: , , wherein, is a clipping function, representing limiting the value of the updated within and range.

[0170] The embodiment also provides a method for training the system, using a cross-entropy loss function with class weights to solve the sample imbalance problem:

[0171] wherein, represents the main task loss, represents the sample class weight, represents the sample true neurological disease label, represents the predicted probability of the neurological disease suffered by the sample by the main task classifier;

[0172] Total loss function: wherein, = 1.2 is the main task priority coefficient, represents the domain task loss.

[0173] Gradient clipping is used in the training process to prevent gradient explosion:

[0174] , is a constraint condition for limiting the amplitude of parameter gradient update, and the meanings of the variables are as follows:

[0175] represents a set of learnable parameters in the model, covering the main task classifier weight , the domain classifier weight and the full connection layer weight of the feature encoder (such as the clinical feature encoder and the fMRI feature encoder), which are the core parameters of the model that are constantly optimized through training.

[0176] represents the gradient vector of the learnable parameter , which reflects the influence trend of the change of the learnable parameter on the loss value, and is the key basis for parameter update.

[0177] : Here is the L2 norm (Euclidean norm) operation of the vector, which is used to calculate the module length of the gradient vector , i.e. the overall size of the gradient, which can quantify the absolute amplitude of the gradient.

[0178] ​​ : Threshold of gradient norm, set to 1.0 in the script, which limits the maximum magnitude of gradient update, avoiding the problem of unstable model training (e.g. training oscillation, gradient explosion) caused by too large gradient norm, ensuring smooth parameter update process.

[0179] Training strategy: AdamW optimizer, learning rate 0.0001, weight decay 4, 300 epochs.

[0180] Example 6,

[0181] This embodiment is a further limitation of Example 1, which gives the specific parameter configuration of each module in the system.

[0182] Data preprocessing parameters: spatial smoothing kernel size: 6 mm FWHM Gaussian kernel; time filtering range: 0.01-0.1 Hz; ROI extraction template: AAL116 template; confounding variable regression: Friston-24 motion parameters + WM / CSF principal components + global signal (optional);

[0183] Spatial registration: 4x4x4 mm MNI152 standard space.

[0184] Feature encoding network parameters: clinical feature encoder: two-layer fully connected network (input 3D → 32D shallow layer → 64D deep layer), batch normalization (BN) and ReLU activation; fMRI feature encoder: one-dimensional convolution (kernel size = 5, step = 1, hidden units = 128) to extract time series features; time attention mechanism: time point weight allocation based on Softmax; spatial attention mechanism: brain region importance matrix based on learnable parameters .

[0185] Early fusion strategy: after mapping the clinical shallow features to the hidden space, they are element-wise fused with the fMRI features.

[0186] Feature fusion and classification: bidirectional attention mechanism for bidirectional interaction between clinical features and fMRI features; main task classifier uses cross-entropy loss (class weight balancing to handle unbalanced samples); domain classifier uses gradient reversal layer (GRL) to achieve adversarial domain adaptation.

[0187] Dynamic balance training strategy: real-time calculation of main task accuracy and domain task accuracy during training, and dynamic adjustment of balance coefficient according to the gradient norm ratio ;

[0188] When the main task accuracy is > 70%, increase the domain adaptation weight, otherwise decrease it;

[0189] Smooth factor , gradient clipping threshold = 5.0, and the optimizer is AdamW (learning rate 1e-4, weight decay 1e-5).

[0190] Experimental results: On the multi-site ADHD dataset, the test set accuracy reaches 97.7%, which is about 7.3 percentage points higher than that of the non-domain adaptive model; the model AUC increases to 0.96, and the F1-score increases to 0.952, showing excellent cross-site generalization ability; ROI importance analysis shows that the invention can automatically identify brain regions significantly related to ADHD and has good interpretability.

[0191] The core technology of site prediction (domain task) is to eliminate the distribution difference of cross-site data through "domain adaptive adversarial learning" to provide "domain invariant features" support for neural disease prediction (main task). The essence is to improve the cross-site generalization ability of the main task through "reverse constraint". The specific technical logic is as follows:

[0192] 1. Problem root: "distribution shift" of cross-site data interferes with the main task. Differences in scanning equipment (such as 3.0T / 1.5T MRI), sequence parameters (such as TR / TE values), and subject population (such as age distribution and proportion of underlying diseases) at different sites will cause fMRI data and clinical features to present "site-specific distribution" (such as the overall intensity of brain activity in A-site data being higher, and B-site data being lower). If such data is directly used to train a disease prediction model, the model is prone to "overfitting to site characteristics" (such as mistaking the specific intensity of A-site as a disease feature), resulting in a significant decrease in diagnostic accuracy on new site data.

[0193] 2. Technical logic: Through "site prediction confrontation", the model is forced to learn core disease features. In this system, site prediction is not an independent task, but forms an "adversarial collaboration" relationship with the main task:

[0194] The goal of the domain task classifier (site prediction) is to "accurately distinguish the site to which the sample belongs", and needs to capture "site-specific features" (such as signal differences caused by equipment) in the data;

[0195] The feature extraction module (double-channel encoding fusion module) needs to meet two reverse constraints under the action of the gradient reversal layer (GRL): (1) let the main task classifier accurately diagnose the disease based on its output features (need to retain "disease-related features"); (2) let the domain task classifier be difficult to distinguish the site based on its output features (need to "suppress site-specific features");

[0196] Finally, the feature fusion and classification module will be forced to "filter out site-specific interference" and only retain "core features that do not change with the site and are directly related to the disease" (i.e., "domain-invariant features"), thus providing more robust feature input for disease prediction (the main task) and solving the problem of "poor generalization ability" of traditional models.

Claims

1. A neural disease diagnosis system based on spatio-temporal attention and dynamic domain adaptation, characterized in that, The system comprises: The data preprocessing module collects clinical characteristic data of the to-be-tested personnel at different sites and fMRI time series data, and pre-processes the fMRI time series data to extract standardized time series of brain regions from the fMRI time series data . ​ A dual-channel feature encoding network module: respectively encode the clinical feature data and the imaging feature data, fuse the encoded information, and obtain global features through an attention mechanism respectively encode the clinical feature data and the imaging feature data, fuse the encoded information, and obtain global features through an attention mechanism The feature fusion and classification module: the global features are fused to obtain fused features; the fused features are input into a main task classifier and a domain task classifier respectively, the main task classifier performs a main task to predict whether the to-be-tested person has a neurological disease, and the domain task classifier performs a domain task to predict the site to which the to-be-tested person belongs; A dynamic balance training module: through a back propagation mechanism, the domain task classifier is optimized, and the domain task loss is reduced; through a dynamic balance control strategy, the dynamic balance of the main task and the domain task is adjusted; a total loss function: wherein, =1.2 is a main task priority coefficient, represents a domain task loss, represents a main task loss; The dynamic balance control strategy is implemented by a dynamic balance controller, which dynamically adjusts according to the performance of the main task and the domain task , specifically: Computing the main task accuracy and the domain task accuracy ; Computing gradient norm ratio : where, denotes the gradient of the domain task loss, denotes the gradient of the main task loss, denotes computing L2 norm operation, is an adjustment factor when > 70% , otherwise ; Smooth update : , wherein, denotes the value before the update , denotes the value after the update , is a smoothing factor; Boundary constraints: , , wherein, is a clipping function that limits the value of the updated to be within and .

2. The spatio-temporal attention and dynamic domain adaptation based neural disease diagnosis system of claim 1, wherein, The clinical feature data are identity information data of the to-be-tested person, and are non-imaging information data.

3. The neurological disease diagnosis system of spatiotemporal attention and dynamic domain adaptation according to claim 2, characterized in that, The feature coding of the clinical feature data is specifically coding the clinical feature data into shallow features and deep features through a two-layer fully connected network and deep features : = ； = ； wherein, is a clinical feature vector, , , and are learnable parameters, is an activation function, denotes batch normalization; right Feature encoding specifically involves using a one-dimensional convolutional layer to extract temporal features: ； wherein, denotes the extracted temporal features, denotes a one-dimensional convolution; Fusing the encoded information specifically projecting the shallow features and fusing them with obtaining fused features : ； wherein, , , denotes a projection weight matrix, denotes a sigmoid function, denotes a gating weight matrix, denotes an element-wise multiplication.

4. The neurological disease diagnosis system of spatiotemporal attention and dynamic domain adaptation according to claim 3, characterized in that, Obtaining global features through an attention mechanism specifically involves learning via a temporal attention mechanism. The importance weights of each time point are assigned to obtain the fMRI temporal feature matrix after temporal attention weighting. ; The importance weight between brain regions is learned through a spatial attention mechanism, and an fMRI spatial feature matrix weighted by the spatial attention weight is obtained ; global features are obtained through global average pooling by introducing a learnable brain region importance parameter R : ， ， wherein denotes global average pooling.

5. The neurological disease diagnosis system of spatiotemporal attention and dynamic domain adaptation according to claim 4, characterized in that, The global features are fused to obtain fused features, and the specific process is as follows: By obtaining global features of an attention weight vector wherein, denotes a multi-layer perceptron; By obtaining deep features of the attention weight vector ; By obtaining a weighted global feature matrix ; By obtaining the weighted deep feature matrix ; By obtaining a difference feature matrix ; By obtaining a fusion feature matrix .

6. The spatio-temporal attention and dynamic domain adaptation based neural disease diagnosis system of claim 5, wherein, The operation performed in the main task classifier is specifically as follows: Feature transformation layer: by performing feature transformation to obtain an output vector after feature transformation ; wherein, is a learnable weight matrix, is a learnable bias term; Probability calculation layer: through Calculate the probability that the test subject has a neurological disease, where express function, This indicates the probability that the test subject is not a patient with a neurological disease. This indicates the probability that the test subject is a patient with a neurological disease. and It is the output vector after feature transformation. The two elements in the table correspond to the original predicted values ​​of the two diagnostic results: where The original predicted value corresponding to the "non-neurological disease patients" category, The original predicted value corresponding to the category "patient with neurological disease"; diagnostic decision layer: by making a diagnostic decision, a prediction result indicating whether the person to be tested suffers from a neurological disease, a prediction result indicating that the person to be tested does not suffer from a neurological disease, a prediction result indicating that the person to be tested suffers from a neurological disease, a prediction result indicating a non-neurological disease, a prediction result indicating a neurological disease.

7. The spatio-temporal attention and dynamic domain adaptation based neural disease diagnosis system of claim 6, wherein, The operation performed in the domain task classifier is specifically as follows: Gradient reversal layer: by obtaining a multi-modal fusion feature matrix wherein denotes a gradient reversal layer, denotes a dynamically adjusted gradient reversal strength; At the time of forward propagation, At the time of backward propagation wherein, denotes the total loss function of the main task classifier and the domain task classifier, denotes the partial derivative operation; Feature transformation layer: by obtaining an original prediction vector for site attribution of a person under test ; wherein, is a learnable weight matrix, is a learnable bias term; Site prediction layer: by obtaining a probability distribution vector of the sites to which the person under test belongs wherein denotes the number of sites; Site decision layer: by obtaining the home site of the person to be tested wherein represents any of the sites, represents the site with the largest probability distribution vector. from the set of sites.

8. The neurological disease diagnosis system of spatiotemporal attention and dynamic domain adaptation according to claim 7, characterized in that, The precondition of the back propagation mechanism is forward propagation, and the specific process is as follows: Forward propagation phase: the domain task classifier receives , outputs the belonging site of the to-be-tested personnel, combines the real site label of the to-be-tested personnel, and calculates the domain task loss through a cross-entropy loss function. In the back propagation stage, the obtained domain task loss is used for back propagation in the domain task classifier, and the parameters of each layer in the domain task classifier are optimized by an optimizer.

Images