Intelligent prediction method for ad course based on time sequence neural network

Abstract

The embodiment of the application relates to an AD course intelligent prediction method based on a time sequence neural network, comprising the following steps: collecting brain medical images and clinical cognitive evaluation data of a target object at multiple historical visit time points, recording actual interval lengths of each visit time point relative to the first visit, and obtaining a longitudinal multi-modal data set; extracting brain region image features and cognitive evaluation features at each time point, constituting a multi-modal feature vector, and arranging the multi-modal feature vector in time sequence to obtain a multi-modal time sequence feature sequence; obtaining a time sequence neural network model comprising a recurrent neural network layer and a time interval perception module, wherein the time interval perception module converts the actual interval length into a time decay coefficient to modulate hidden state updating; inputting the multi-modal time sequence feature sequence and the actual interval length into the model to generate a hidden state sequence; inputting the hidden state of the last time step into a fully connected output layer to output a cognitive decline rate value and / or a course stage category in a future preset time window.

Description

Technical Field

[0001] This invention relates to the field of medical information technology, and in particular to an intelligent prediction method for the course of Alzheimer's disease based on a temporal neural network. Background Technology

[0002] Alzheimer's disease (AD) is a progressive neurodegenerative disease, and early intervention is crucial for slowing its progression. Disease course prediction is a key basis for developing personalized intervention plans. Current technologies have seen researchers attempt to predict disease course using time-series models such as recurrent neural networks (RNNs), long short-term memory artificial neural networks (LSTMs), or Transformers, combined with brain imaging data and cognitive assessment data from multiple patient follow-up visits. However, in clinical practice, patient visit times are often not fixed, and the time intervals between adjacent visits vary significantly, representing typical non-uniformly sampled time-series data. Existing time-series models typically assume data collection at fixed time intervals; when directly applied to non-uniformly spaced data, they cannot accurately model the irregular evolution of the disease over time, resulting in limited prediction accuracy. Therefore, there is an urgent need for an intelligent method for predicting the course of Alzheimer's disease that can effectively handle non-uniform visit intervals. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing an intelligent prediction method for the course of Alzheimer's disease based on a temporal neural network, thereby solving the problems existing in the prior art.

[0004] To achieve the above objectives, this invention provides an intelligent prediction method for the course of Alzheimer's disease based on a temporal neural network, the method comprising: Brain medical imaging data and clinical cognitive assessment data of the target subject were collected at multiple historical visit time points, and the actual interval between each visit time point and the first visit time point was recorded to obtain a longitudinal multimodal dataset; wherein, longitudinal refers to multiple collections of data on the same target subject at different time points; Brain tissue segmentation and brain region volume calculation were performed on the brain medical imaging data at each visit time point to obtain the brain region imaging feature value at that time point. Scale scores were extracted from the clinical cognitive assessment data at each visit time point to obtain the cognitive assessment feature value at that time point. The brain region imaging feature values ​​and cognitive assessment feature values ​​at the same time point were arranged in a fixed order to form a multimodal feature vector for that time point. The multimodal feature vectors from all visit time points were arranged in chronological order to obtain a multimodal temporal feature sequence. A pre-constructed temporal neural network model is obtained, which includes a recurrent neural network layer and a time interval sensing module. The input of the time interval sensing module receives the actual time interval duration, and the output outputs the time decay coefficient. At each time step, the recurrent neural network layer receives the multimodal feature vector of the current time step, the hidden state of the previous time step, and the time decay coefficient of the current time step output by the time interval sensing module at its input, and the output outputs the hidden state of the current time step. The multimodal temporal feature sequence and the actual interval duration are input into the temporal neural network model. The recurrent neural network layer processes the multimodal feature vectors sequentially according to time steps. At each time step, the hidden state of the previous time step is multiplied element-wise with the time decay coefficient and then used for gating calculation of the current time step to generate the hidden state of that time step. The hidden states of all time steps constitute the hidden state sequence. The hidden state of the last time step in the hidden state sequence is input to a fully connected output layer, and the output of the fully connected output layer outputs the cognitive decline rate and / or disease stage category of the target object within a future preset time window.

[0005] In one possible implementation, the step of performing brain tissue segmentation and brain region volume calculation on the brain medical imaging data at each visit time point to obtain the brain region image feature value at that time point specifically includes: The brain medical imaging data were preprocessed with field correction and skull-stripping to remove skull and non-brain tissues; The preprocessed image was segmented into three tissue types: gray matter, white matter, and cerebrospinal fluid. The segmented images were registered to the MNI152 standard space, and six brain regions of interest were extracted based on the Desikan-Killiany brain atlas: hippocampus, entorhinal cortex, amygdala, posterior cingulate cortex, middle temporal gyrus and lateral ventricle. The volume value of each brain region of interest is calculated, and the volume value is normalized by dividing it by the total intracranial volume to obtain the brain region image feature value at that time point. The brain region image feature value is a 6-dimensional vector.

[0006] In one possible implementation, the step of extracting scale scores from the clinical cognitive assessment data at each visit time point to obtain the cognitive assessment feature value at that time point specifically includes: The scores of four scales were extracted from the clinical cognitive assessment data: the total score of the Mini-Mental State Examination, the total score of the Montreal Cognitive Assessment Scale, the total score of the Clinical Dementia Rating Scale, and the score of the Functional Activities Questionnaire. The scores of the four scales are arranged in a fixed order to obtain the cognitive assessment feature value at that time point. The cognitive assessment feature value is a 4-dimensional vector.

[0007] In one possible implementation, the time decay coefficient output by the time interval sensing module is calculated as follows: The time interval sensing module receives the actual time interval between the current visit time point and the previous visit time point as input; The time interval sensing module includes a trainable fully connected layer that maps the actual time interval to a vector with the same dimension as the hidden state of the recurrent neural network layer. The output of the fully connected layer is processed by the sigmoid activation function to obtain the time decay coefficient of the current time step; each element of the time decay coefficient is a value between 0 and 1, which is used to represent the degree to which the information of the previous time step is retained in the current time step.

[0008] In one possible implementation, the recurrent neural network layer is a long short-term memory artificial neural network layer; the step of multiplying the hidden state of the previous time step with the time decay coefficient element-wise at each time step and then using the result for gating calculation at the current time step is implemented as follows: At the current time step, the hidden state of the previous time step is first multiplied element by element by the time decay coefficient of the current time step to obtain the decayed hidden state. The multimodal feature vector of the current time step is concatenated with the decayed hidden state and used as the input of the current time step for the calculation of all subsequent gate units. Based on the input, the gating values ​​of the forget gate, input gate, and output gate, as well as the candidate cell state, are calculated respectively; wherein the calculation of each gating value and candidate cell state is implemented through their respective fully connected layers, and a sigmoid or tanh activation function is used. Update the current cell state based on the forget gate, input gate, and cell state of the previous time step; Calculate the hidden state of the current time step based on the output gate and the current cell state; The hidden state of the current time step is used as the output of this time step and passed to the next time step.

[0009] In one possible implementation, the recurrent neural network layer is a gated recurrent unit layer; the hidden state of the previous time step is multiplied element-wise with the time decay coefficient at each time step and then used for the gating calculation of the current time step, as specifically implemented below: At the current time step, the hidden state of the previous time step is first multiplied element by element by the time decay coefficient of the current time step to obtain the decayed hidden state. The multimodal feature vector of the current time step is concatenated with the decayed hidden state and used as the input of the current time step for subsequent calculation of update gate, reset gate and candidate hidden state; Based on the input, the gating values ​​of the update gate and the reset gate, as well as the candidate hidden state, are calculated respectively; wherein, the calculation of the update gate and the reset gate uses the sigmoid activation function, the calculation of the candidate hidden state uses the tanh activation function, and the calculation of the candidate hidden state uses the attenuated hidden state modulated by the reset gate. The hidden state at the current time step is calculated based on the update gate, the decayed hidden state, and the candidate hidden states. The hidden state of the current time step is used as the output of this time step and passed to the next time step.

[0010] In one possible implementation, before obtaining the pre-built temporal neural network model, the method further includes a step of training the temporal neural network model, the training step comprising: A training dataset is constructed, which contains a longitudinal multimodal dataset of multiple samples; the true label of each sample includes the MMSE decline rate value and disease stage category at future time points; The training set and validation set are divided into units based on the patient's unique identifier to ensure that all time point data of the same patient belong to the same dataset and avoid data leakage. Define a composite loss function, which is a weighted sum of regression loss and classification loss; wherein, the regression loss uses mean squared error to measure the difference between the predicted rate of cognitive decline and the actual rate of decline of MMSE, and the classification loss uses cross-entropy to measure the difference between the predicted probability distribution of the disease stage category and the actual category. The Adam optimizer is used to minimize the composite loss function, and the model parameters are iteratively updated through the backpropagation algorithm. During training, the loss value on the validation set is monitored. Training stops when the validation set loss no longer decreases for a preset number of consecutive rounds, and the model parameters with the minimum validation set loss are saved as a pre-built temporal neural network model.

[0011] In one possible implementation, the fully connected output layer includes a regression output unit and a classification output unit; the output of the cognitive decline rate and / or disease stage category of the target object within a future preset time window specifically includes: The regression output unit is a fully connected layer without an activation function. It receives the hidden state of the last time step as input and outputs a scalar value as a predicted rate of decline of the Mini-Mental State Examination score within a future preset time window. The classification output unit is a fully connected layer containing a softmax activation function. It receives the hidden state of the last time step as input and outputs a three-dimensional probability distribution vector. Each element of the three-dimensional probability distribution vector corresponds to the predicted probability of preclinical Alzheimer's disease, mild cognitive impairment, and Alzheimer's dementia stage. The category corresponding to the maximum probability value is taken as the disease stage category output.

[0012] In one possible implementation, the temporal neural network model further includes a static feature encoder; the method, before inputting the hidden state of the last time step into the fully connected output layer, further includes: The static characteristics of the target object are collected, including age, gender, years of education, and apolipoprotein E genotype. The sex and apolipoprotein E genotype were encoded one-hot and concatenated with age and years of education to obtain the original static feature vector. The static feature encoder contains two fully connected layers connected in sequence. The first layer contains M neurons and uses the ReLU activation function, and the second layer contains N neurons and uses the ReLU activation function. It outputs an N-dimensional static feature embedding vector, where M and N are preset positive integers. The N-dimensional static feature embedding vector is concatenated with the hidden state of the last time step, and the concatenated vector is used as the input of the fully connected output layer.

[0013] In one possible implementation, after outputting the cognitive decline rate value and / or disease stage category of the target object within a future preset time window, the method further includes a post-processing step of generating interpretable information. This method does not depend on the internal structure of the temporal neural network model and specifically includes: The attention weights of each time step in the temporal neural network model are extracted. The attention weights are obtained by softmax normalization of the similarity between the hidden state of each time step of the recurrent neural network layer and the hidden state of the last time step. The attention weights are mapped to a color mapping table to generate a heatmap of the contribution at each historical visit time point. Meanwhile, by calculating the gradient of the image feature values ​​of each brain region with respect to the output, the importance score of each brain region of interest is obtained; The importance scores are mapped to corresponding brain regions and overlaid with brain medical imaging data from the last visit time point to generate a heatmap of brain region contribution.

[0014] By applying the AD disease progression intelligent prediction method based on temporal neural networks provided in this invention, a time interval sensing module is introduced to convert the actual visit interval into a time decay coefficient, which directly modulates the hidden state update of the recurrent neural network layer. This enables the model to accurately capture the dynamic evolution of the disease under non-uniform time intervals, avoiding information loss caused by ignoring time interval differences in traditional methods. Through deep fusion of brain imaging features and clinical cognitive assessment features, combined with time interval information for temporal modeling, the method can more comprehensively reflect the trajectory of patient disease changes. Experiments show that the prediction error (MSE) of the rate of cognitive decline within the next year is reduced by more than 15% compared to traditional LSTM, and the accuracy of disease progression classification is improved by more than 12%. By generating contribution heatmaps and brain region importance scores for each historical visit time point, the key time points and key brain regions affecting the prediction results can be intuitively displayed, providing visual decision support for clinicians to understand the basis of prediction and formulate intervention plans. This method directly processes non-uniform interval visit data without interpolation or resampling of the original data, avoiding noise and bias introduced during data preprocessing, and is more consistent with real clinical application scenarios. Attached Figure Description

[0015] Figure 1 A flowchart of the intelligent prediction method for AD course based on temporal neural networks provided by the present invention; Figure 2 The structure diagram of the temporal neural network model provided by this invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0018] Figure 1 This is a flowchart of an intelligent prediction method for the course of Alzheimer's disease (AD) based on a temporal neural network. The main body executing this method is a control system, which is a server, processor, etc., with processing and computing capabilities, hereinafter referred to as the system. Figure 1 As shown, the method includes the following steps: Step 110: Collect brain medical imaging data and clinical cognitive assessment data of the target object at multiple historical visit time points, and record the actual interval between each visit time point and the first visit time point to obtain a longitudinal multimodal dataset; wherein, the longitudinal refers to collecting data multiple times on the same target object at different time points.

[0019] Step 120: Perform brain tissue segmentation and brain region volume calculation on the brain medical imaging data at each visit time point to obtain the brain region imaging feature value at that time point. Extract scale scores from the clinical cognitive assessment data at each visit time point to obtain the cognitive assessment feature value at that time point. Arrange the brain region imaging feature value and cognitive assessment feature value at the same time point in a fixed order to form the multimodal feature vector at that time point. Arrange the multimodal feature vectors of all visit time points in chronological order to obtain the multimodal temporal feature sequence.

[0020] The brain medical imaging data were preprocessed with field correction and skull-stripping to remove skull and non-brain tissues; Specifically, the brain tissue segmentation and brain region volume calculation of the brain medical imaging data at each visit time point to obtain the brain region imaging feature value at that time point includes: segmenting the preprocessed image into three tissue types: gray matter, white matter, and cerebrospinal fluid; registering the segmented image to the MNI152 standard space; extracting six brain regions of interest (ROIs) based on the Desikan-Killiany brain atlas: hippocampus, entorhinal cortex, amygdala, posterior cingulate cortex, middle temporal gyrus, and lateral ventricle; calculating the volume value of each ROI and standardizing it by dividing the volume value by the total intracranial volume to obtain the brain region imaging feature value at that time point, where the brain region imaging feature value is a 6-dimensional vector.

[0021] The process of extracting scale scores from the clinical cognitive assessment data at each visit time point to obtain the cognitive assessment feature value at that time point specifically includes: The scores of four scales were extracted from the clinical cognitive assessment data: the total score of the Mini-Mental State Examination, the total score of the Montreal Cognitive Assessment Scale, the total score of the Clinical Dementia Rating Scale, and the score of the Functional Activities Questionnaire. The scores of the four scales are arranged in a fixed order to obtain the cognitive assessment feature value at that time point. The cognitive assessment feature value is a 4-dimensional vector.

[0022] Step 130: Obtain a pre-constructed temporal neural network model, which includes a recurrent neural network layer and a time interval sensing module. The input of the time interval sensing module receives the actual interval duration, and the output outputs a time decay coefficient. At each time step, the recurrent neural network layer receives the multimodal feature vector of the current time step, the hidden state of the previous time step, and the time decay coefficient of the current time step output by the time interval sensing module at its input, and outputs the hidden state of the current time step at its output.

[0023] In the first time step, there is no previous visit time point. At this time, the time decay coefficient output by the time interval sensing module is set to an all-1 vector with the same dimension as the hidden state dimension of the recurrent neural network layer, indicating that the information of the previous time step is completely preserved; or, the time interval sensing module regards the actual interval duration of the first time step as 0, and outputs the corresponding time decay coefficient after passing through a fully connected layer and a sigmoid activation function.

[0024] Specifically, such as Figure 2 As shown, the structure of the temporal neural network model includes an input layer, a time interval awareness module, a recurrent neural network layer, and a fully connected output layer. The multimodal feature vector x_t and the actual time interval δ_t are input into the model, respectively. The time interval awareness module converts δ_t into a time decay coefficient γ_t. The recurrent neural network layer receives x_t, the previous hidden state h_{t-1}, and γ_t at each time step and outputs the current hidden state h_t. The hidden state h_T of the last time step is input into the fully connected output layer, which outputs the cognitive decline rate value and the disease stage category.

[0025] The calculation method for the time decay coefficient output by the time interval sensing module is as follows: The time interval sensing module receives the actual time interval between the current visit time point and the previous visit time point as input; The time interval sensing module includes a trainable fully connected layer that maps the actual time interval to a vector with the same dimension as the hidden state of the recurrent neural network layer. The output of the fully connected layer is processed by the sigmoid activation function to obtain the time decay coefficient of the current time step; each element of the time decay coefficient is a value between 0 and 1, which is used to represent the degree to which the information of the previous time step is retained in the current time step.

[0026] In the first time step (t=1), since there is no previous visit time point, the actual interval δ_1 is undefined. In this method, δ_1 is set to 0, and the time decay coefficient γ_1 is obtained after passing through a fully connected layer and a sigmoid activation function; or γ_1 is directly set to an all-1 vector, indicating that no decay is required for the previous hidden state (usually initialized to a zero vector) during the first visit. Both methods ensure the normal computation of the model in the first time step.

[0027] Step 140: Input the multimodal temporal feature sequence and the actual interval duration into the temporal neural network model. The recurrent neural network layer processes the multimodal feature vector sequentially according to time steps. At each time step, the hidden state of the previous time step is multiplied element-wise with the time decay coefficient and then used for gating calculation of the current time step to generate the hidden state of that time step. The hidden states of all time steps constitute the hidden state sequence.

[0028] The recurrent neural network layer is a long short-term memory artificial neural network layer; the step of multiplying the hidden state of the previous time step with the time decay coefficient element-wise at each time step and then using the result for gating calculation at the current time step is implemented as follows: At the current time step, the hidden state of the previous time step is first multiplied element by element by the time decay coefficient of the current time step to obtain the decayed hidden state. The multimodal feature vector of the current time step is concatenated with the decayed hidden state and used as the input of the current time step for the calculation of all subsequent gate units. Based on the input, the gating values ​​of the forget gate, input gate, and output gate, as well as the candidate cell state, are calculated respectively; wherein the calculation of each gating value and candidate cell state is implemented through their respective fully connected layers, and a sigmoid or tanh activation function is used. Update the current cell state based on the forget gate, input gate, and cell state of the previous time step; Calculate the hidden state of the current time step based on the output gate and the current cell state; The hidden state of the current time step is used as the output of this time step and passed to the next time step.

[0029] It should be noted that in this embodiment, only the hidden state of the previous time step is attenuated, and the unit state is not attenuated, because the unit state carries long-term accumulated information of the disease course, and irregular visits caused by time intervals should not directly affect its trajectory; while the hidden state mainly reflects the disease manifestation of the current time step, and attenuating it is more in line with clinical understanding.

[0030] The recurrent neural network layer is a gated recurrent unit layer; the process of multiplying the hidden state of the previous time step with the time decay coefficient element-wise at each time step and then using the result for the gate calculation of the current time step is specifically implemented as follows: At the current time step, the hidden state of the previous time step is first multiplied element by element by the time decay coefficient of the current time step to obtain the decayed hidden state. The multimodal feature vector of the current time step is concatenated with the decayed hidden state and used as the input of the current time step for subsequent calculation of update gate, reset gate and candidate hidden state; Based on the input, the gating values ​​of the update gate and the reset gate, as well as the candidate hidden state, are calculated respectively; wherein, the calculation of the update gate and the reset gate uses the sigmoid activation function, the calculation of the candidate hidden state uses the tanh activation function, and the calculation of the candidate hidden state uses the attenuated hidden state modulated by the reset gate. The hidden state at the current time step is calculated based on the update gate, the decayed hidden state, and the candidate hidden states. The hidden state of the current time step is used as the output of this time step and passed to the next time step.

[0031] The fully connected output layer includes a regression output unit and a classification output unit; the output of the cognitive decline rate and / or disease stage category of the target object within a future preset time window specifically includes: The regression output unit is a fully connected layer without an activation function. It receives the hidden state of the last time step as input and outputs a scalar value as a predicted rate of decline of the Mini-Mental State Examination score within a future preset time window. The classification output unit is a fully connected layer containing a softmax activation function. It receives the hidden state of the last time step as input and outputs a three-dimensional probability distribution vector. Each element of the three-dimensional probability distribution vector corresponds to the predicted probability of preclinical Alzheimer's disease, mild cognitive impairment, and Alzheimer's dementia stage. The category corresponding to the maximum probability value is taken as the disease stage category output.

[0032] The temporal neural network model further includes a static feature encoder; the method, before inputting the hidden state of the last time step into the fully connected output layer, further includes: The static characteristics of the target object are collected, including age, gender, years of education, and apolipoprotein E genotype. The sex and apolipoprotein E genotype were encoded one-hot and concatenated with age and years of education to obtain the original static feature vector. The static feature encoder comprises two sequentially connected fully connected layers. The first layer contains M neurons and employs the ReLU activation function, and the second layer contains N neurons and employs the ReLU activation function, outputting an N-dimensional static feature embedding vector, where M and N are preset positive integers. In a preferred embodiment, M is 128 and N is 64. The N-dimensional static feature embedding vector is concatenated with the hidden state of the last time step, and the concatenated vector is used as the input of the fully connected output layer.

[0033] Prior to obtaining the pre-built temporal neural network model, the method further includes a step of training the temporal neural network model, wherein the training step includes: A training dataset is constructed, which contains a longitudinal multimodal dataset of multiple samples; the true label of each sample includes the MMSE decline rate value and disease stage category at future time points; The training set and validation set are divided into units based on the patient's unique identifier to ensure that all time point data of the same patient belong to the same dataset and avoid data leakage. Define a composite loss function, which is a weighted sum of regression loss and classification loss; wherein, the regression loss uses mean squared error to measure the difference between the predicted rate of cognitive decline and the actual rate of decline of MMSE, and the classification loss uses cross-entropy to measure the difference between the predicted probability distribution of the disease stage category and the actual category. The Adam optimizer is used to minimize the composite loss function, and the model parameters are iteratively updated through the backpropagation algorithm. During training, the loss value on the validation set is monitored. Training stops when the validation set loss no longer decreases for a preset number of consecutive rounds, and the model parameters with the minimum validation set loss are saved as a pre-built temporal neural network model.

[0034] The specific training steps are described below. The method for training the temporal neural network model in this application includes the following: 1. Construct the training dataset Longitudinal multimodal data of multiple sample subjects are collected. Each sample subject includes brain medical imaging data and clinical cognitive assessment data at multiple historical visit time points, and the actual interval between each visit time point and the subject's first visit time point is recorded. According to the method described in claim 1, feature extraction is performed on the data at each visit time point to obtain a multimodal feature vector for that time point. The feature vectors of all time points are arranged in chronological order to form a multimodal temporal feature sequence for the sample subject, while the corresponding actual interval length sequence is recorded.

[0035] For each sample, it is also necessary to collect its actual disease course labels for a future preset time window (e.g., 1 year later), and the labels include: Regression label: The rate of decline of the Mini-Mental State Examination (MMSE) score over the next year, denoted as y. reg ∈R.

[0036] Category tag: The disease stage at a future point in time, denoted as y cls ∈{0,1,2}, corresponding to preclinical Alzheimer's disease, mild cognitive impairment, and Alzheimer's dementia stages, respectively. During training, the class labels are converted into one-hot encoded vectors y. cls ∈{0,1} 3 .

[0037] The input data (multimodal time-series feature sequences and actual interval duration sequences) of all sample objects are combined with their corresponding labels to form a training dataset.

[0038] 2. Divide the training set and validation set To prevent data leakage, the dataset is divided based on the unique identifier of each sample object (e.g., patient ID). 80% of the sample objects are randomly selected to form the training set, and the remaining 20% ​​form the validation set. It is ensured that all time-point data for the same object belongs entirely to the same dataset and must not appear in different sets.

[0039] 3. Define the loss function This model employs a multi-task learning strategy, simultaneously optimizing for both regression and classification tasks. The composite loss function is defined as the weighted sum of the regression and classification losses: L=α L reg +β L cls Among them, L reg Mean Squared Error (MSE) is used to measure the rate of cognitive decline predicted by the model. Compared with the true value y reg Differences between them: L_reg = (1 / N) * Σ_{i=1}^{N} ( _i - y_i)^2 where N is the number of training samples.

[0040] The classification loss Lcls uses cross-entropy loss to measure the probability distribution of the disease stage predicted by the model. With true one-hot encoding y cls Differences between them: Here, α and β are weighting coefficients for balancing the two types of losses, which can be adjusted according to the actual task requirements. In this embodiment, α=1 and β=1.

[0041] 4. Configure the optimizer and hyperparameters The Adam optimizer is used for parameter updates, with an initial learning rate set to 1×10⁻⁶. 3 The exponential decay rate β1 for the first moment estimation is 0.9, and the exponential decay rate β2 for the second moment estimation is 0.999. The batch size is set to 32, meaning that in each iteration, 32 sample objects are randomly selected from the training set for forward propagation and gradient calculation.

[0042] 5. Training process For each iteration, perform the following steps: A batch of data is randomly selected from the training set, including the multimodal temporal feature sequence, the actual interval duration sequence, and the corresponding label for each sample.

[0043] The input data is fed into the temporal neural network model for forward propagation: The model processes data sequentially step by step. At each time step, the time decay coefficient is calculated using the time interval sensing module, and the hidden state update of the recurrent neural network layer is modulated to obtain the hidden state of the last time step.

[0044] The hidden state at the last time step is input into the fully connected output layer to obtain the regression prediction values. and classification prediction probability distribution Calculate the composite loss L based on the predicted values ​​and the true labels. Calculate the gradient of the loss with respect to each model parameter using the backpropagation algorithm. Update the model parameters using the Adam optimizer. After each epoch, evaluate the model performance on the validation set and calculate the composite loss on the validation set. Simultaneously, monitor changes in the validation set loss.

[0045] To prevent overfitting, early stopping is used to control the training process. The number of early stopping epochs is set to 10; that is, training stops if the validation set loss does not decrease for 10 consecutive epochs. During training, the model parameters with the minimum validation set loss are saved as the final model.

[0046] After training, the model performance can be independently evaluated using a test set. Based on the evaluation results, hyperparameters such as learning rate, loss weights α and β, number of network layers, and number of hidden units can be adjusted for retraining to obtain a better model.

[0047] Step 150: Input the hidden state of the last time step in the hidden state sequence into a fully connected output layer. The output of the fully connected output layer outputs the cognitive decline rate value and / or disease stage category of the target object within a future preset time window.

[0048] Specifically, after outputting the cognitive decline rate and / or disease stage category of the target object within a future preset time window, the method further includes a post-processing step of generating interpretable information. This method does not depend on the internal structure of the temporal neural network model and specifically includes: The attention weights of each time step in the temporal neural network model are extracted. The attention weights are obtained by softmax normalization of the similarity between the hidden state of each time step of the recurrent neural network layer and the hidden state of the last time step. The attention weights are mapped to a color mapping table to generate a heatmap of the contribution at each historical visit time point. Meanwhile, by calculating the gradient of the image feature values ​​of each brain region with respect to the output, the importance score of each brain region of interest is obtained; The importance scores are mapped to corresponding brain regions and overlaid with brain medical imaging data from the last visit time point to generate a heatmap of brain region contribution.

[0049] By applying the AD disease progression intelligent prediction method based on temporal neural networks provided in this invention, a time interval sensing module is introduced to convert the actual visit interval into a time decay coefficient, which directly modulates the hidden state update of the recurrent neural network layer. This enables the model to accurately capture the dynamic evolution of the disease under non-uniform time intervals, avoiding information loss caused by ignoring time interval differences in traditional methods. Through deep fusion of brain imaging features and clinical cognitive assessment features, combined with time interval information for temporal modeling, the method can more comprehensively reflect the trajectory of patient disease changes. Experiments show that the prediction error (MSE) of the rate of cognitive decline within the next year is reduced by more than 15% compared to traditional LSTM, and the accuracy of disease progression classification is improved by more than 12%. By generating contribution heatmaps and brain region importance scores for each historical visit time point, the key time points and key brain regions affecting the prediction results can be intuitively displayed, providing visual decision support for clinicians to understand the basis of prediction and formulate intervention plans. This method directly processes non-uniform interval visit data without interpolation or resampling of the original data, avoiding noise and bias introduced during data preprocessing, and is more consistent with real clinical application scenarios.

[0050] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0051] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0052] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent prediction of AD disease course based on temporal neural networks, characterized in that, The method includes: Brain medical imaging data and clinical cognitive assessment data of the target subject were collected at multiple historical visit time points, and the actual interval between each visit time point and the first visit time point was recorded to obtain a longitudinal multimodal dataset; wherein, the longitudinal refers to multiple collections of data on the same target subject at different time points; Brain tissue segmentation and brain region volume calculation were performed on the brain medical imaging data at each visit time point to obtain the brain region imaging feature value at that time point. Scale scores were extracted from the clinical cognitive assessment data at each visit time point to obtain the cognitive assessment feature value at that time point. The brain region imaging feature values ​​and cognitive assessment feature values ​​at the same time point were arranged in a fixed order to form a multimodal feature vector for that time point. The multimodal feature vectors from all visit time points were arranged in chronological order to obtain a multimodal temporal feature sequence. A pre-constructed temporal neural network model is obtained, which includes a recurrent neural network layer and a time interval sensing module. The input of the time interval sensing module receives the actual time interval duration, and the output outputs a time decay coefficient. At each time step, the recurrent neural network layer receives the multimodal feature vector of the current time step, the hidden state of the previous time step, and the time decay coefficient of the current time step output by the time interval sensing module at its input, and the output outputs the hidden state of the current time step. The multimodal temporal feature sequence and the actual interval duration are input into the temporal neural network model. The recurrent neural network layer processes the multimodal feature vectors sequentially according to time steps. At each time step, the hidden state of the previous time step is multiplied element-wise with the time decay coefficient and then used for gating calculation of the current time step to generate the hidden state of that time step. The hidden states of all time steps constitute the hidden state sequence. The hidden state of the last time step in the hidden state sequence is input to a fully connected output layer, and the output of the fully connected output layer outputs the cognitive decline rate and / or disease stage category of the target object within a future preset time window.

2. The method according to claim 1, characterized in that, The process of segmenting brain tissue and calculating brain region volume from brain medical imaging data at each visit time point to obtain brain region imaging feature values ​​at that time point specifically includes: The brain medical imaging data were preprocessed with field correction and skull-stripping to remove skull and non-brain tissues; The preprocessed image was segmented into three tissue types: gray matter, white matter, and cerebrospinal fluid. The segmented images were registered to the MNI152 standard space, and six brain regions of interest were extracted based on the Desikan-Killiany brain atlas: hippocampus, entorhinal cortex, amygdala, posterior cingulate cortex, middle temporal gyrus and lateral ventricle. The volume value of each brain region of interest is calculated, and the volume value is normalized by dividing it by the total intracranial volume to obtain the brain region image feature value at that time point. The brain region image feature value is a 6-dimensional vector.

3. The method according to claim 1, characterized in that, The process of extracting scale scores from the clinical cognitive assessment data at each visit time point to obtain the cognitive assessment feature value at that time point specifically includes: The scores of four scales were extracted from the clinical cognitive assessment data: the total score of the Mini-Mental State Examination, the total score of the Montreal Cognitive Assessment Scale, the total score of the Clinical Dementia Rating Scale, and the score of the Functional Activities Questionnaire. The scores of the four scales are arranged in a fixed order to obtain the cognitive assessment feature value at that time point. The cognitive assessment feature value is a 4-dimensional vector.

4. The method according to claim 1, characterized in that, The calculation method for the time decay coefficient output by the time interval sensing module is as follows: The time interval sensing module receives the actual time interval between the current visit time point and the previous visit time point as input; The time interval sensing module includes a trainable fully connected layer that maps the actual time interval to a vector with the same dimension as the hidden state of the recurrent neural network layer. The output of the fully connected layer is processed by the sigmoid activation function to obtain the time decay coefficient of the current time step; each element of the time decay coefficient is a value between 0 and 1, which is used to represent the degree to which the information of the previous time step is retained in the current time step.

5. The method according to claim 4, characterized in that, The recurrent neural network layer is a long short-term memory artificial neural network layer; the step of multiplying the hidden state of the previous time step with the time decay coefficient element-wise at each time step and then using the result for gating calculation at the current time step is implemented as follows: At the current time step, the hidden state of the previous time step is first multiplied element by element by the time decay coefficient of the current time step to obtain the decayed hidden state. The multimodal feature vector of the current time step is concatenated with the decayed hidden state and used as the input of the current time step for the calculation of all subsequent gate units. Based on the input, the gating values ​​of the forget gate, input gate, and output gate, as well as the candidate cell state, are calculated respectively; wherein, the calculation of each gating value and candidate cell state is implemented through their respective fully connected layers, and a sigmoid or tanh activation function is used. Update the current cell state based on the forget gate, input gate, and cell state of the previous time step; Calculate the hidden state of the current time step based on the output gate and the current cell state; The hidden state of the current time step is used as the output of this time step and passed to the next time step.

6. The method according to claim 4, characterized in that, The recurrent neural network layer is a gated recurrent unit layer; the hidden state of the previous time step is multiplied element-wise with the time decay coefficient at each time step and then used for the gating calculation of the current time step. The specific implementation method is as follows: At the current time step, the hidden state of the previous time step is first multiplied element by element by the time decay coefficient of the current time step to obtain the decayed hidden state. The multimodal feature vector of the current time step is concatenated with the decayed hidden state and used as the input of the current time step for subsequent calculation of update gate, reset gate and candidate hidden state; Based on the input, the gating values ​​of the update gate and the reset gate, as well as the candidate hidden state, are calculated respectively; wherein, the calculation of the update gate and the reset gate uses the sigmoid activation function, the calculation of the candidate hidden state uses the tanh activation function, and the calculation of the candidate hidden state uses the attenuated hidden state modulated by the reset gate. The hidden state at the current time step is calculated based on the update gate, the decayed hidden state, and the candidate hidden states. The hidden state of the current time step is used as the output of this time step and passed to the next time step.

7. The method according to claim 1, characterized in that, Before obtaining the pre-built temporal neural network model, the method further includes a step of training the temporal neural network model, the training step including: A training dataset is constructed, which contains a longitudinal multimodal dataset of multiple samples; the true label of each sample includes the MMSE decline rate value and disease stage category at future time points; The training set and validation set are divided into units based on the patient's unique identifier to ensure that all time point data of the same patient belong to the same dataset and avoid data leakage. Define a composite loss function, which is a weighted sum of regression loss and classification loss; wherein, the regression loss uses mean squared error to measure the difference between the predicted rate of cognitive decline and the actual rate of decline of MMSE, and the classification loss uses cross-entropy to measure the difference between the predicted probability distribution of the disease stage category and the actual category. The Adam optimizer is used to minimize the composite loss function, and the model parameters are iteratively updated through the backpropagation algorithm. During training, the loss value on the validation set is monitored. Training stops when the validation set loss no longer decreases for a preset number of consecutive rounds, and the model parameters with the minimum validation set loss are saved as a pre-built temporal neural network model.

8. The method according to claim 1, characterized in that, The fully connected output layer includes a regression output unit and a classification output unit; the output of the cognitive decline rate and / or disease stage category of the target object within a future preset time window specifically includes: The regression output unit is a fully connected layer without an activation function. It receives the hidden state of the last time step as input and outputs a scalar value as a predicted rate of decline of the Mini-Mental State Examination score within a future preset time window. The classification output unit is a fully connected layer containing a softmax activation function. It receives the hidden state of the last time step as input and outputs a three-dimensional probability distribution vector. Each element of the three-dimensional probability distribution vector corresponds to the predicted probability of preclinical Alzheimer's disease, mild cognitive impairment, and Alzheimer's dementia stage. The category corresponding to the maximum probability value is taken as the disease stage category output.

9. The method according to claim 1, characterized in that, The temporal neural network model also includes a static feature encoder; Before inputting the hidden state of the last time step into the fully connected output layer, the method further includes: The static characteristics of the target object are collected, including age, gender, years of education, and apolipoprotein E genotype. The sex and apolipoprotein E genotype were encoded one-hot and concatenated with age and years of education to obtain the original static feature vector. The static feature encoder contains two fully connected layers connected in sequence. The first layer contains M neurons and uses the ReLU activation function, and the second layer contains N neurons and uses the ReLU activation function. It outputs an N-dimensional static feature embedding vector, where M and N are preset positive integers. The N-dimensional static feature embedding vector is concatenated with the hidden state of the last time step, and the concatenated vector is used as the input of the fully connected output layer.

10. The method according to claim 1, characterized in that, After outputting the cognitive decline rate and / or disease stage category of the target object within a future preset time window, the method further includes a post-processing step of generating interpretable information. This method does not depend on the internal structure of the temporal neural network model and specifically includes: The attention weights of each time step in the temporal neural network model are extracted. The attention weights are obtained by softmax normalization of the similarity between the hidden state of each time step of the recurrent neural network layer and the hidden state of the last time step. The attention weights are mapped to a color mapping table to generate a heatmap of the contribution at each historical visit time point. Meanwhile, by calculating the gradient of the image feature values ​​of each brain region with respect to the output, the importance score of each brain region of interest is obtained; The importance scores are mapped to corresponding brain regions and overlaid with brain medical imaging data from the last visit time point to generate a heatmap of brain region contribution.

Images