Early-stage Alzheimer's disease imaging feature analysis method based on improved ResNet-Transformer model

Abstract

The invention discloses an early Alzheimer's disease imaging feature analysis method based on an improved ResNet-Transform model, and belongs to the field of deep learning and medical image analysis. In order to solve the problem that an existing network model is difficult to capture global feature information of a medical image in an early Alzheimer's disease task, an improved ResNet-50 model and an improved Transformer module are constructed, the improved ResNet-50 model and the improved Transformer module are connected in parallel, and then a global average pooling layer, a third full connection layer, a Softmax layer and a third output layer are connected in series, so that the overall feature information of the medical image can be captured in the early Alzheimer's disease task. According to the method, an improved ResNet-Transformer model is obtained, the improved ResNet-Transformer model is trained by applying a medical image set, and the trained model can capture global feature information of an AD image, so that the recognition accuracy is improved, and the lightweight of the model is realized.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and early disease symptom prediction, and more specifically, to a method for early Alzheimer's disease image feature analysis based on an improved ResNet-Transformer model. Background Technology

[0002] According to the latest data from the United Nations, the global population aged 65 and over is projected to reach approximately 10% in 2024, and a high of around 15% in 2050. This trend could lead to a surge in the number of people suffering from Alzheimer's Disease (AD). Currently, there are approximately 52 million people with Alzheimer's disease worldwide, and this number is projected to reach around 130 million by 2050.

[0003] Alzheimer's disease (AD) is a chronic neurodegenerative disease with insidious symptoms. Common clinical manifestations include cognitive decline and memory impairment, and it can even lead to irreversible brain damage or death. Currently, there is no effective treatment to completely cure the disease. However, intervention at the onset of early symptoms can lead to better prognosis and significantly improve the patient's quality of life. With the development of deep learning, analyzing early AD imaging features using network models has recently become a hot research topic in the field of artificial intelligence.

[0004] In 2015, the deep residual network (ResNet) based on residual modules, proposed by He et al., successfully solved the gradient vanishing or exploding problems of traditional convolutional neural networks (CNNs). It still achieves good performance and efficiency even with very deep layers. ResNet's application in early AD medical image recognition can better understand the detailed features of images. However, limited by the size of the convolutional kernel, ResNet cannot effectively utilize information outside the receptive field, making it difficult to capture the global feature information of AD. Furthermore, how to further reduce the number of ResNet parameters and computational complexity during model construction remains a hot research direction.

[0005] Transformer is a sequence feature extraction model based on attention mechanism proposed by the Google team in 2017. It uses multiple self-attention mechanism to highlight the key features of AD images and the long-range dependencies between similar blocks in the image. It can provide global information and capture the dependencies between distant sequences. However, Transformer may ignore the local features and background information of AD images, thus limiting the applicability of the model.

[0006] In summary, designing a network model with high recognition accuracy and good generalization ability is crucial for early image analysis of AD.

[0007] Chinese patent application CN113989551A, published on January 28, 2022, describes an Alzheimer's disease classification method based on an improved ResNet network. It proposes optimizing the ResNet model to identify medical images and predict AD, but this method still struggles to capture the global feature information of AD images. Chinese patent application CN118711800A, published on September 27, 2024, describes a method for predicting coronary heart disease based on depthwise separable convolution and attention mechanisms. It proposes a depthwise separable convolution method for lightweight model design, but fails to significantly reduce the number of parameters and computational complexity. Summary of the Invention

[0008] 1. Technical problems to be solved

[0009] To address the shortcomings of existing network models in capturing global feature information of AD images, failing to reduce computational complexity, and lacking lightweight design in early Alzheimer's disease tasks, this invention provides an image feature analysis method for early Alzheimer's disease based on an improved ResNet-Transformer model. By applying the improved ResNet-Transformer fusion model to process AD images, it can capture global feature information of AD images, improve prediction accuracy, and achieve model lightweight design.

[0010] 2. Technical Solution

[0011] The objective of this invention is achieved through the following technical solutions.

[0012] This invention provides a method for image feature analysis of early Alzheimer's disease based on an improved ResNet-Transformer model, the specific steps of which are as follows:

[0013] Step S100: Obtain AD images, preprocess the AD images, and label each preprocessed AD image as one of the following categories: normal, mild cognitive impairment, or suspected Alzheimer's disease, to obtain an AD image dataset.

[0014] The dataset comes from a neuroimaging database, which contains AD images mainly of two types: MRI (magnetic resonance imaging) and PET (positron emission tomography).

[0015] Methods for preprocessing AD images include filtering out abnormal samples, using spatial domain denoising techniques to reduce image noise, enhancing the dataset by color transformation and / or geometric transformation to increase the data volume, and performing a unified format conversion on the image dataset.

[0016] Preferably, Gaussian filtering technology is used. Through linear smoothing filtering, the image pixel and its neighboring pixel values ​​are weighted and averaged. The weights are set according to Gaussian distribution, with pixels closer to the center pixel having higher weights and vice versa.

[0017] Preferably, JPEG format is used to uniformly convert the AD images. The AD images have a pixel size of 224x224, and these images retain key information, serving as the foundation for model learning and completing various tasks.

[0018] Furthermore, when classifying and labeling AD image samples according to the patient's health status corresponding to the image, the labeling categories include normal, mild cognitive impairment, or suspected Alzheimer's disease.

[0019] The AD image dataset is divided into a training set, a validation set, and a test set.

[0020] Preferably, the AD image dataset is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5.

[0021] Step S200: Construct an improved ResNet-50 model as the backbone feature extraction network.

[0022] Choosing the ResNet-50 model as the backbone feature extraction network, the improvements to the ResNet-50 model in this invention are as follows: First, a kernel replacement method is used to replace some of the traditional convolutional kernel structures in the ResNet-50 model by designing new convolutional kernel structures. Specifically, the kernel replacement method involves replacing one 7x7 convolutional kernel with a stride of 2 in the traditional ResNet-50 model with three 3x3 concatenated convolutional kernels.

[0023] The improvement of ResNet-50 in this invention lies in replacing all standard convolutions in the residual units of the residual module in the traditional ResNet-50 with depthwise separable convolutions. Depthwise separable convolutions include depthwise convolutions and pointwise convolutions. Each residual unit after the replacement includes m 3x3 convolutional kernels with a stride of 2, a batch normalization layer, an optimized ReLU(x) activation function, and n convolutional layers. Residual connections are then established between the residual units.

[0024] The further improvement of ResNet-50 as the backbone feature extraction network in this invention is that the GAM module is placed after all the residual units in the ResNet-50 model, and the first fully connected layer and the first output layer are connected in series after the GAM module.

[0025] Step S300: Build the improved Transformer module.

[0026] The specific improvement of the traditional Transformer module in this invention lies in the following: a parallel pooling layer is obtained by connecting a 3x3 max pooling layer with a stride of 2 and an average pooling layer with a stride of 2 in parallel. This parallel pooling layer is then concatenated and placed after the multi-head self-attention mechanism of the encoder and decoder in the traditional Transformer module. The multi-head self-attention mechanism and the parallel pooling layer together constitute a pooling-based multi-head self-attention mechanism. Finally, a second fully connected layer and a second output layer are concatenated at the end of the Transformer module.

[0027] Step S400: Connect the improved ResNet-50 model and the improved Transformer module in parallel, and then connect the global average pooling layer, the third fully connected layer, the Softmax layer and the third output layer in series to obtain the improved ResNet-Transformer model.

[0028] Step S500: Train the improved ResNet-Transformer model using the AD image dataset to obtain the trained improved ResNet-Transformer model.

[0029] The network training process is as follows: A set of AD image data from the training set is simultaneously input into both the improved ResNet-50 model and the improved Transformer module. Forward propagation yields predicted values. Based on the predicted values ​​and the true labels, the difference between the probability distribution predicted by the model and the probability distribution of the true labels is calculated. Then, backpropagation updates the parameters, completing one round of training. After each round of training, the model performance is evaluated and monitored using validation set data (which does not participate in parameter updates). Training stops when the validation set loss no longer decreases, indicating model convergence and training completion. Finally, the performance of the final ResNet-Transformer model is evaluated using test set data, resulting in the trained improved ResNet-Transformer model.

[0030] The specific process of using the training and validation set data through the ResNet-50 model and the Transformer module is as follows:

[0031] The process of processing AD image data using the improved ResNet-50 model is as follows:

[0032] First, the preprocessed AD image data with a pixel size of 224x224 is convolved by three concatenated convolution kernels of size 3x3 and stride 2, and then max pooling is performed by a max pooling layer of size 3x3 and stride 2.

[0033] Next, the AD image data after the above operations is transmitted to the residual unit for processing. The depthwise separable convolution structure in the residual unit includes depthwise convolution and pointwise convolution. The AD image data is first subjected to depthwise convolution. The method is as follows: m convolution kernels of size 3x3 and stride 2 are assigned to the m input channels of the AD image data for independent convolution operations (the channels do not interfere with each other). After convolution, m channel data after depthwise convolution are obtained.

[0034] The data from the m channels after depthwise convolution is first standardized using a batch normalization layer, and then a nonlinear transformation is performed on the data using an optimized ReLU(x) activation function.

[0035]

[0036] Where x is the channel data after standardization, max is the maximum value function, and c is a parameter that is always greater than 0.

[0037] The channel data after optimizing the ReLU(x) activation function enters the pointwise convolution process. The pointwise convolution stage includes n convolutional layers, each containing m 1×1 kernels with a stride of 2. First, the m channel data after optimizing the ReLU(x) activation function are convolved with the m kernels in the first convolutional layer of the pointwise convolution stage, outputting m channel data after passing through the first convolutional layer. These m channel data are then summed and aggregated into one channel data. In the same way, all m channel data after optimizing the ReLU(x) activation function are sequentially convolved with the n×m kernels in the pointwise convolution stage and aggregated. Finally, the entire pointwise convolution stage will produce n channels of output data.

[0038] After the AD image data undergoes processing by all the residual units mentioned above, it is then processed by the GAM module to interact with global context information and local features. Finally, the output data of the ResNet-50 network model channels are connected and integrated using the first fully connected layer, and the first output layer outputs feature L. ResNet .

[0039] AD image data is processed by an improved Transformer module and the second output layer outputs feature L. TransformerThe specific processing steps are as follows: First, the AD image is divided into multiple small blocks to obtain patches. Then, the patches are converted into a format that can be processed by the Transformer module, resulting in tensor format patches. Subsequently, the tensor format patches are transformed into a high-dimensional space through an input embedding layer, converting the image region features into a feature matrix that the Transformer module can process more effectively.

[0040] The feature matrix is ​​fed into the encoder part of the Transformer. The core of the encoder is a pooling-based multi-head self-attention mechanism. This mechanism first performs specific operations on the input feature matrix to obtain the three inputs required by the pooling-based multi-head self-attention mechanism: the query matrix Q, the key matrix K, and the value matrix V. The specific operation is achieved through linear mapping, that is, multiplying the input feature matrix with a learnable weight matrix to project the input feature matrix into different subspaces, thus obtaining Q, K, and V. After obtaining Q, K, and V, the pooling-based multi-head self-attention mechanism further performs calculations, first calculating the dot product of Q and K, with the formula:

[0041]

[0042] Here, M(Q,K) is the result of the dot product of Q and K. T Let be the transpose of the key matrix K, and T be the transpose operation. k To query the dimensions of vectors in matrix Q and key matrix K, where K is the key matrix.

[0043] Then, M(Q,K) is normalized using the Softmax function to determine the weights corresponding to each V. Finally, these weights are multiplied by their corresponding V values ​​to obtain the attention mechanism for the correlation between different positions in the input sequence, calculated as follows:

[0044] Attention(M(Q,K),V)=Softmax(M(Q,K))V

[0045] Here, Attention(M(Q,K),V) is the attention mechanism for the correlation between different positions in the input sequence, and also represents the weighted feature representation. The Softmax function is used in the attention mechanism to transform M(Q,K) into a probability distribution.

[0046] Next, the weighted feature representations are input into a parallel pooling layer for pooling operations. The results from both max pooling and average pooling operations are then merged using the following formula:

[0047] F output=(1-α)×MaxPool(F1)+α×AvgPool(F2)

[0048] Among them, F output To fuse the output feature representations of MaxPool(F1) and AvgPool(F2), α is a weight parameter that controls the importance of max pooling and average pooling. F1 is the input feature representation of the max pooling layer, and MaxPool(F1) is the output feature representation of F1 after the max pooling operation. F2 is the input feature representation of the average pooling layer, and AvgPool(F2) is the output feature representation of F2 after the average pooling operation.

[0049] These feature representations, each in its own independent subspace, undergo attention computation through different linear transformations to generate a series of output feature representation heads. Each head maps to an attention pattern within a specific subspace. These heads are then concatenated along the feature dimension and fused into a comprehensive feature representation. Further, linear transformations are applied to these concatenated heads using matrix multiplication and addition to synthesize and adjust the feature representations. This transformation process aims to fuse all attention information, ultimately forming a fully fused feature representation MultiHead(M(Q,K),V), as shown in the following equation:

[0050] MultiHead(M(Q,K),V)=Concat(head1,head2,…,head h W o

[0051] head = Attention(M(Q,K),V)

[0052] Here, Concat is a concatenation operation, where head1, head2, ..., head h These are the feature representations obtained through attention calculation in the 1st, 2nd...hth subspaces, where h is a pre-defined number of heads determined based on experiments and experience. W o It is a learnable weight matrix.

[0053] Following the multi-head self-attention and parallel pooling layers, a feedforward neural network is cascaded in. The feedforward neural network consists of two linear layers and a non-linear activation function. Residual connections and layer normalization are introduced in each layer of the multi-head self-attention and feedforward networks.

[0054] The feature representations obtained after encoder processing enter the decoder stage. In the initial sub-layer of the decoder, a multi-head self-attention mechanism based on pooling with masking is used. Specifically, the feature representations of each head are calculated through a multi-head self-attention process, and then these feature representations are further pooled through parallel pooling layers. When calculating the attention weights, the weights of information after the current processing position are set to minimum values. Subsequently, the feature representations are again monitored by a multi-head self-attention process to focus on the encoder output.

[0055] The decoder also integrates feedforward neural networks, residual connections, and layer normalization structures.

[0056] The feature representations are processed through an improved Transformer module, then connected and integrated by a second fully connected layer, and finally the second output layer outputs the feature L. Transformer .

[0057] After processing by the ResNet-50 model and the Transformer module, the first output layer outputs feature L. ResNet and the output feature L from the second output layer Transformer Perform a full connection. The specific process is as follows:

[0058] Output feature L of the first output layer ResNet Second output layer output feature L Transformer The data is then passed to a 3x3 global average pooling layer with a stride of 2 for feature compression and dimensionality reduction, and then connected through a third fully connected layer.

[0059] The first output layer outputs feature L. ResNet Second output layer output features L Transformer The fusion characteristics after the third fully connected layer are:

[0060] L combined =1 / a*L ResNet +1 / b*L Transformer

[0061] Among them, L combined For fusion features, 'a' is the adjustment of L. ResNet The weighting parameter of the proportion, b is the adjustment L Transformer The weighting parameters of the proportion.

[0062] The fused feature L after passing through the third fully connected layer combined The input is fed into the Softmax layer and the third output layer for processing.

[0063] The Softmax layer and the third output layer affect L combined The specific processing procedure is as follows:

[0064] First, L combined After being converted into relative probability values ​​by the Softmax layer, L combined The sum of the predicted values ​​for each category after transformation by the Softmax layer is 1.

[0065] Using the cross-entropy loss function CE loss (Cross-Entropy loss function) Calculate the loss. In each iteration, calculate the gradient of the loss function with respect to the model parameters, and then update the parameters based on the gradient. Convergence occurs when the network model's loss function value no longer decreases significantly or reaches a preset stopping condition. After model training is complete, all parameters need to be saved.

[0066]

[0067] Among them, CE loss It is the cross-entropy loss function, where s is the number of samples, i is any value from 1 to the total number of samples s, and p i Let q be the probability distribution of the true label. i This represents the probability distribution predicted by the model.

[0068] The predicted values ​​output from the third output layer are mapped and classified, with the predicted values ​​falling into categories such as normal, mild cognitive impairment, or suspected Alzheimer's disease. After model training is complete, the model's performance is evaluated using test set data, resulting in a trained network model.

[0069] Step S600: Collect AD images and apply the trained network model to classify the collected AD images into one of the following categories: normal, mild cognitive impairment, or suspected Alzheimer's disease.

[0070] 3. Beneficial effects

[0071] Compared to existing technologies, the advantages of this invention are: it can improve model computation speed while maintaining accuracy, and reduce the number of parameters required for feature map convolution calculations. The main contributions are:

[0072] 1. A fusion learning model based on an improved ResNet-Transformer dual model is proposed to better learn detailed information and global features in AD patient images, thereby improving the accuracy of global feature extraction from AD images.

[0073] 2. By combining the kernel substitution method and the depthwise separable convolution method, ResNet-50 is made lightweight, which reduces the number of parameters during convolution and improves the speed of the model in classifying AD images.

[0074] 3. Integrating the Global Attention Mechanism (GAM) module into the improved ResNet-50 allows the ResNet-50 model to focus more on important features, significantly improving the model's performance when processing AD images.

[0075] 4. By combining average pooling and max pooling operations into a parallel pooling layer and placing it after the self-attention layer in the Transformer module, the key texture features of the AD image are preserved to the maximum extent and the number of parameters is reduced. Attached Figure Description

[0076] Figure 1 This is an overall flowchart of the image feature analysis method for early Alzheimer's disease based on the improved ResNet-Transformer model of the present invention;

[0077] Figure 2 This is a structural diagram of the residual unit of the present invention;

[0078] Figure 3 This is a diagram showing the connection relationships between residual units in this invention;

[0079] Figure 4 This is a structural diagram of the improved Transformer module of the present invention;

[0080] Figure 5 Diagram of the improved ResNet-Transformer dual-model network structure. Detailed Implementation

[0081] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0082] Example

[0083] like Figure 1 As shown, this invention provides a method for early Alzheimer's disease image feature analysis based on an improved ResNet-Transformer model, the steps of which are as follows:

[0084] Step S100: Obtain AD images, preprocess the AD images, and label each preprocessed AD image as one of the following categories: normal, mild cognitive impairment, or suspected Alzheimer's disease, to obtain an AD image dataset.

[0085] The dataset comes from a neuroimaging database, which contains AD images mainly of two types: MRI (magnetic resonance imaging) and PET (positron emission tomography).

[0086] The preprocessing process includes the following sub-steps:

[0087] First, the dataset comes from a neuroimaging database, which contains AD images mainly of two types: MRI (magnetic resonance imaging) and PET (positron emission tomography).

[0088] Abnormal samples are screened and removed, including those with low image quality, incomplete data, or that do not meet the research requirements.

[0089] Then, spatial domain denoising techniques are used to reduce noise in the image. For example, the mature Gaussian filtering technique is used. Gaussian filtering uses linear smoothing to perform a weighted average of the image pixel and its neighboring pixel values. The weights are set according to the Gaussian distribution. The closer the pixel is to the center pixel, the higher the weight, and vice versa.

[0090] After noise reduction, the dataset is enhanced by color transformation (including adjusting color attributes such as image brightness, contrast, and saturation) and / or geometric transformation (including operations such as rotating, scaling, and flipping the image) to generate more image data of different forms to expand the data volume. This prevents the model from becoming overly dependent on a small number of specific samples and improves the model's generalization ability and environmental adaptability.

[0091] Meanwhile, JPEG format was used to uniformly convert the AD images. The AD images have a pixel size of 224x224, and these images retain key information, forming the basis for model learning and completing various tasks.

[0092] In the final stage of preprocessing, AD image samples are classified and labeled according to the corresponding patient health status. The labeling classification includes normal, mild cognitive impairment, or suspected Alzheimer's disease.

[0093] The preprocessed AD image dataset is divided into a training set, a validation set, and a test set. Preferably, the AD image dataset is divided into the training set, validation set, and test set in a ratio of 7:1.5:1.5.

[0094] Step S200: Construct an improved ResNet-50 as the backbone feature extraction network.

[0095] ResNet-50 was chosen as the backbone feature extraction network. The traditional ResNet-50 initial structure includes an input layer, a 7x7 convolutional kernel with a stride of 2, a pooling layer, four residual blocks, a fully connected layer, and an output layer. The four residual blocks have different structures, and each residual block contains multiple residual units. The number of residual units in these four residual blocks are 3, 4, 6, and 3, respectively.

[0096] The improvement of ResNet-50 in this invention application lies in the following: First, a kernel replacement method is adopted to replace part of the traditional convolutional kernel structure of the ResNet-50 model by designing a new convolutional kernel structure. Specifically, the kernel replacement method involves replacing a single 7x7 convolutional kernel with a stride of 2 in the traditional ResNet-50 model with three 3x3 convolutional kernels connected in series. This further optimizes computation and enhances the ability to extract specific features.

[0097] The improvement of ResNet-50 in this invention application lies in replacing all standard convolutions in the residual units of the residual module in the traditional ResNet-50 with depthwise separable convolutions. For example... Figure 2 As shown, the residual unit comprises depthwise separable convolutions, batch normalization layers, and an optimized ReLU(x) activation function. Depthwise separable convolutions include depthwise convolutions and pointwise convolutions. First, depthwise convolutions are used to focus on feature extraction for each channel, capturing the unique information of different channels and significantly reducing the number of parameters and computational cost. Then, pointwise convolutions are used with 1x1 kernels and a stride of 2 to fuse information between channels, extracting more representative features and further reducing computational complexity.

[0098] Each replaced residual unit includes m 3x3 convolutional kernels with a stride of 2, a batch normalization layer, an optimized ReLU(x) activation function, and n convolutional layers, where the values ​​of m and n are adjusted according to the model depth. In some embodiments, m and n are 3 or 4. Figure 3 As shown, residual connections are then made between each residual unit to alleviate the gradient vanishing problem and accelerate the network convergence speed.

[0099] The further improvement of ResNet-50 as the backbone feature extraction network in this invention lies in the following: the Global Attention Mechanism (GAM) module is placed after all the residual units in the ResNet-50 model, and the output of the residual units is further filtered and enhanced to enable the network to better utilize the learned features.

[0100] Finally, the first fully connected layer and the first output layer are concatenated after the GAM module to obtain the improved ResNet-50 as the backbone feature extraction network.

[0101] Step S300: Build the improved Transformer module.

[0102] The specific improvement of the traditional Transformer module in this invention lies in: a parallel pooling layer is obtained by connecting a 3x3 max pooling layer with a stride of 2 and an average pooling layer with a size of 3x3 and a stride of 2 in parallel. For example... Figure 4 As shown, the parallel pooling layer is then concatenated and placed after the multi-head self-attention mechanism of the encoder and decoder in the traditional Transformer module. The multi-head self-attention mechanism and the parallel pooling layer together constitute a pooling-based multi-head self-attention mechanism.

[0103] Placing pooling techniques after the multi-head self-attention mechanism of the Transformer module effectively reduces its computational complexity and expands its receptive range. Among pooling techniques, max pooling tends to filter out the main features within a local region, which is highly advantageous for capturing key information in images or sequences. In contrast, average pooling, through averaging, focuses more on reflecting the overall statistical characteristics of a local region, reflecting its general properties. Combining these two pooling methods allows for the simultaneous acquisition of multiple different types of feature information, thus covering multi-scale information from salient features to global characteristics. Furthermore, this combination enhances robustness to local anomalies and noise, thereby improving the robustness and generalization ability of the Transformer module.

[0104] Furthermore, by concatenating a second fully connected layer and a second output layer at the end of the Transformer module, an improved Transformer module is obtained.

[0105] Step S400: Connect the improved ResNet-50 model and the improved Transformer module in parallel, and then connect the global average pooling layer, the third fully connected layer, the Softmax layer and the third output layer in series to obtain the improved ResNet-Transformer model.

[0106] The Transformer module in the model can capture long-range dependencies and provide richer global contextual information for ResNet-50, improving the model's ability to express features and its robustness. Compared to the Transformer module, ResNet-50 is better at extracting local features and can effectively learn feature representations at different levels, providing a solid foundation for subsequent integration with the Transformer module.

[0107] Step S500: Train the improved ResNet-Transformer model using the AD image dataset to obtain the trained improved ResNet-Transformer model.

[0108] The network training process is as follows: using a set of AD image data from the training set, such as... Figure 5As shown, the improved ResNet-Transformer model is simultaneously input from both the improved ResNet-50 model and the improved Transformer module. Forward propagation yields predicted values. The difference between the model's predicted probability distribution and the true label probability distribution is calculated based on the predicted values ​​and the true label probability distribution. Backpropagation then updates the parameters, completing one round of training. After each training round, the model performance is evaluated and monitored using validation set data (which does not participate in parameter updates). Training stops when the validation set loss no longer decreases, indicating model convergence and training completion. Finally, the performance of the final ResNet-Transformer model is evaluated using test set data, resulting in the trained improved ResNet-Transformer model.

[0109] The specific process of using the training and validation set data through the ResNet-50 model and the Transformer module is as follows:

[0110] The process of processing AD image data using the improved ResNet-50 network model is as follows:

[0111] First, the preprocessed AD image data with a pixel size of 224x224 is convolved by three concatenated convolution kernels of size 3x3 and stride 2, and then max pooling is performed by a max pooling layer of size 3x3 and stride 2.

[0112] Next, the AD image data after the above operations is transmitted to the residual unit for processing. The depthwise separable convolution structure in the residual unit includes depthwise convolution and pointwise convolution. The AD image data is first subjected to depthwise convolution. The method is as follows: the m input channels of the AD image data are assigned m convolution kernels of size 3x3 and stride 2 for independent convolution operations (the channels do not interfere with each other). After convolution, m channel data after depthwise convolution are obtained.

[0113] The data from the m channels after depthwise convolution is first standardized using a batch normalization layer, and then a nonlinear transformation is performed on the data using an optimized ReLU(x) activation function.

[0114]

[0115] Where x is the standardized channel data, max is the maximum value function, and c is a parameter that is always greater than 0. The value of c is adjusted and confirmed according to the data category to improve the model's ability to distinguish between different categories.

[0116] The channel data after optimizing the ReLU(x) activation function enters the pointwise convolution process. The pointwise convolution stage includes n convolutional layers, each containing m 1×1 kernels with a stride of 2. First, the m channel data after optimizing the ReLU(x) activation function are convolved with the m kernels in the first convolutional layer of the pointwise convolution stage, outputting m channel data after passing through the first convolutional layer. These m channel data are then summed and aggregated into one channel data. In the same way, all m channel data after optimizing the ReLU(x) activation function are sequentially convolved with the n×m kernels in the pointwise convolution stage and aggregated. Finally, the entire pointwise convolution stage will produce n channels of output data.

[0117] After the AD image data is processed by all the residual units mentioned above, it is then processed by the GAM module to interact with global context information and local features.

[0118] Finally, the output feature information of the ResNet-50 network model is connected and integrated using the first fully connected layer, and the first output layer outputs feature L. ResNet .

[0119] When AD image data passes through the improved Transformer module, the specific processing procedure is as follows:

[0120] First, the AD image is divided into multiple small patches. These patches are then transformed into a format that the Transformer module can process, resulting in tensor-format patches. Subsequently, the tensor-format patches are transformed into a high-dimensional space through an input embedding layer, converting the image region features into a feature matrix that the Transformer module can process more efficiently.

[0121] The feature matrix is ​​fed into the encoder part of the Transformer. The core of the encoder is a pooling-based multi-head self-attention mechanism. This mechanism first performs specific operations on the input feature matrix to obtain the three inputs required by the pooling-based multi-head self-attention mechanism: the query matrix Q, the key matrix K, and the value matrix V. The specific operation is achieved through linear mapping, that is, multiplying the input feature matrix with a learnable weight matrix to project the input feature matrix into different subspaces, thus obtaining Q, K, and V. After obtaining Q, K, and V, the pooling-based multi-head self-attention mechanism further performs calculations, first calculating the dot product of Q and K, with the formula:

[0122]

[0123] Here, M(Q,K) is the result of the dot product of Q and K, which reflects the degree of correlation or similarity between different positions. T Let be the transpose of the key matrix K, and T be the transpose operation. d k To query the dimensions of vectors in matrix Q and key matrix K, where K is the key matrix and d k This can prevent the dot product result from being too large, thus keeping the attention score within a reasonable range.

[0124] Then, M(Q,K) is normalized using the Softmax function to determine the weights corresponding to each V. Finally, these weights are multiplied by their corresponding V values ​​to obtain the attention mechanism for the correlation between different positions in the input sequence, calculated as follows:

[0125] Attention(M(Q,K),V)=Softmax(M(Q,K))V

[0126] Here, Attention(M(Q,K),V) is the attention mechanism for the correlation between different positions in the input sequence, and also represents the weighted feature representation. The Softmax function is used in the attention mechanism to transform M(Q,K) into a probability distribution.

[0127] Next, the weighted feature representations are input into a parallel pooling layer for pooling operations. The results from both max pooling and average pooling operations are then merged using the following formula:

[0128] F output =(1-α)×MaxPool(F1)+α×AvgPool(F2)

[0129] Among them, F output To fuse the output feature representations of MaxPool(F1) and AvgPool(F2), α is a weight parameter that controls the importance of max pooling and average pooling. F1 is the input feature representation of the max pooling layer, and MaxPool(F1) is the output feature representation of F1 after the max pooling operation. F2 is the input feature representation of the average pooling layer, and AvgPool(F2) is the output feature representation of F2 after the average pooling operation.

[0130] These feature representations, each in its own independent subspace, undergo attention computation through different linear transformations to generate a series of output feature representation heads. Each head maps to an attention pattern within a specific subspace. These heads are then concatenated along the feature dimension and fused into a comprehensive feature representation. Further, linear transformations are applied to these concatenated heads using matrix multiplication and addition to synthesize and adjust the feature representations. This transformation process aims to fuse all attention information, ultimately forming a fully fused feature representation MultiHead(M(Q,K),V), as shown in the following equation:

[0131] MultiHead(M(Q,K),V)=Concat(head1,head2,…,head h W o

[0132] head = Attention(M(Q,K),V)

[0133] Here, Concat is a concatenation operation, where head1, head2, ..., head h These are the feature representations obtained through attention calculation in the 1st, 2nd...hth subspaces, where h is a pre-defined number of heads determined based on experiments and experience. W o It is a learnable weight matrix, which is optimized during model training using the backpropagation algorithm.

[0134] Following multi-head self-attention and parallel pooling layers, a feedforward neural network is cascaded. This network comprises two linear layers and a non-linear activation function for further feature processing and to enhance the model's non-linear expressive power, thereby better capturing complex features in the image. Residual connections and layer normalization are introduced in each layer of the multi-head self-attention and feedforward networks. Residual connections help prevent gradient vanishing and maintain direct information transfer. Layer normalization helps to standardize features, making model training more stable.

[0135] The feature representations processed by the encoder enter the decoder stage. In the initial sub-layer of the decoder, a multi-head self-attention mechanism based on pooling with a mask is employed. This mechanism is similar to the pooling-based multi-head self-attention mechanism in the encoder, but differs in that it restricts the model to only focus on information up to the current processing position. This design aims to prevent the undue leakage of future information during sequence generation. Specifically, the feature representations of each head are calculated through a multi-head self-attention process, and then these feature representations are further pooled through parallel pooling layers. When calculating the attention weights, the weights of information after the current processing position are set to minimum values. Subsequently, the feature representations again focus on the encoder's output through another multi-head self-attention layer, thus facilitating the construction of the target sequence.

[0136] The decoder also integrates feedforward neural networks, residual connections, and layer normalization structures.

[0137] The feature representations are processed through an improved Transformer module, then connected and integrated by a second fully connected layer, and finally the second output layer outputs the feature L. Transformer .

[0138] After processing by the ResNet-50 model and the Transformer module, the first output layer outputs feature L. ResNet and the output feature L from the second output layer Transformer Perform a full connection. The specific process is as follows:

[0139] Output feature L of the first output layer ResNet Second output layer output features L Transformer The data is then passed to a 3x3 global average pooling layer with a stride of 2 for feature compression and dimensionality reduction, and then connected through a third fully connected layer.

[0140] The first output layer outputs feature L. ResNet Second output layer output features L Transformer The fusion characteristics after the third fully connected layer are:

[0141] L combined =1 / a*L ResNet +1 / b*L Transformer

[0142] Among them, L combined For fusion features, a is the adjustment of L ResNet The weighting parameter of the proportion, b is the adjustment L Transformer The weighting parameters of the proportion.

[0143] The fused feature L after passing through the third fully connected layer combinedThe input is fed into the Softmax layer and the third output layer for processing.

[0144] The Softmax layer and the third output layer affect L combined The specific processing procedure is as follows:

[0145] First, L combined After being converted into relative probability values ​​by the Softmax layer, L combined The sum of the predicted values ​​for each category after transformation by the Softmax layer is 1.

[0146] Using the cross-entropy loss function CE loss The Cross-Entropy loss function calculates the loss. This function continuously adjusts the model parameters to minimize the loss function by measuring the difference between the model's predicted values ​​and the true values. Specifically, in each iteration, the gradient of the loss function with respect to the model parameters is calculated, and the parameters are updated based on the gradient. This evaluates the difference between the model's prediction and the true results under the current parameters, prompting the model's predictions to be closer to the true results. Generally, convergence occurs when the network model's loss function value no longer decreases significantly or reaches a preset stopping condition. After model training, all parameters need to be saved. These parameters can be used for subsequent prediction tasks, or the saved parameters can be loaded to continue training or fine-tuning the model when needed.

[0147]

[0148] Among them, CE loss It is the cross-entropy loss function, where s is the number of samples, i is any value from 1 to the total number of samples s, and p i Let q be the probability distribution of the true label. i This represents the probability distribution predicted by the model.

[0149] The predicted values ​​output from the third output layer are mapped and categorized, with the result falling into three categories: normal, mild cognitive impairment, or suspected Alzheimer's disease. If the classification is normal, the corresponding imaging features may indicate that the subject's cognitive function is normal, without obvious signs of cognitive impairment, such as normal memory, thinking ability, and language ability. This can be used for screening the general population to distinguish normal individuals for targeted health management and prevention measures. If the predicted value falls into mild cognitive impairment, the corresponding imaging features may show that the subject is in the early stages of cognitive decline, but has not yet reached the level of severe cognitive impairment such as Alzheimer's disease. If the classification result falls into suspected Alzheimer's disease, the corresponding imaging features indicate that the subject has a certain probability of having Alzheimer's disease, requiring further medical examination and confirmation.

[0150] After the model training is completed, the performance of the model is evaluated using test set data to obtain the trained network model.

[0151] Step S600: Collect AD images and apply the trained network model to classify the collected AD images into one of the following categories: normal, mild cognitive impairment, or suspected Alzheimer's disease.

[0152] Through experiments, the improved network demonstrates advantages over previous networks and existing technologies in its ability to comprehensively extract features from medical images. The model's image processing speed is significantly improved. In predicting Alzheimer's disease (AD), the model shows significant optimization in both accuracy and speed, enabling patients to receive more timely treatment. To reduce the number of parameters in convolution, this invention proposes combining kernel substitution and depthwise separable convolution to lightweight ResNet-50, thereby improving the model's speed in predicting AD.

[0153] The present invention and its embodiments have been described above illustratively. This description is not restrictive, and the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. The accompanying drawings are only one embodiment of the present invention, and the actual structure is not limited thereto. No reference numerals in the claims should limit the scope of the claims. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the scope of protection of the present invention. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Multiple elements stated in the product claims may also be implemented by a single element through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for image feature analysis of early Alzheimer's disease based on an improved ResNet-Transformer model, the specific steps of which are as follows: Acquire AD images, preprocess the AD images, and label each preprocessed AD image as one of the following categories: normal, mild cognitive impairment, or suspected Alzheimer's disease, to obtain an AD image dataset; An improved ResNet-50 was constructed as the backbone feature extraction network; Build an improved Transformer module; The improved ResNet-50 model and the improved Transformer module are connected in parallel, and then the global average pooling layer, the third fully connected layer, the Softmax layer and the third output layer are connected in series to obtain the improved ResNet-Transformer model. The improved ResNet-Transformer model is trained using the AD image dataset to obtain the trained improved ResNet-Transformer model; Images of Alzheimer's disease (AD) are collected, and a trained network model is used to classify the collected AD images into one of the following categories: normal, mild cognitive impairment, or suspected Alzheimer's disease.

2. The method for early Alzheimer's disease image feature analysis based on an improved ResNet-Transformer model according to claim 1, characterized in that: The method for preprocessing AD images includes filtering out abnormal samples, using spatial domain denoising techniques to reduce image noise, enhancing the dataset and expanding the data volume through color transformation and / or geometric transformation, and performing a unified format conversion on the image dataset.

3. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 2, characterized in that: The improvement of the ResNet-50 model lies in replacing the traditional ResNet-50 model's single convolutional kernel with a size of 7x7 and a stride of 2 with three cascaded 3x3 kernels.

4. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 3, characterized in that, The improvements to the ResNet-50 model are as follows: all standard convolutions in the residual units of the residual module in the traditional ResNet-50 are replaced with depthwise separable convolutions; depthwise separable convolutions include depthwise convolutions and pointwise convolutions; each residual unit after the replacement includes m convolutional kernels of size 3x3 with stride of 2, batch normalization operation layers, optimized ReLU(x) activation function and n convolutional layers; and residual connections are then made between the residual units.

5. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 4, characterized in that, The improvement of the ResNet-50 model lies in placing the GAM module after all the residual units in the ResNet-50 model, and then concatenating the first fully connected layer and the first output layer after the GAM module.

6. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 5, characterized in that, The improved Transformer module improves upon the traditional Transformer module by: connecting a 3x3 max pooling layer with a stride of 2 and an average pooling layer with a stride of 2 in parallel to obtain a parallel pooling layer; then concatenating the parallel pooling layer after the multi-head self-attention mechanism of the encoder and decoder in the traditional Transformer module; the multi-head self-attention mechanism and the parallel pooling layer together constitute a pooling-based multi-head self-attention mechanism; and finally concatenating a second fully connected layer and a second output layer at the end of the Transformer module.

7. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 6, characterized in that, AD image data is processed by an improved ResNet-50 model and the first output layer outputs feature L. ResNet The process is as follows: First, the preprocessed AD image data with a pixel size of 224x224 is convolved using three concatenated 3x3 kernels with a stride of 2, followed by max pooling using a 3x3 max pooling layer with a stride of 2. Then, the AD image data after these operations is transferred to a residual unit for processing. The depthwise separable convolutional structure in the residual unit includes depthwise convolution and pointwise convolution. The AD image data is first subjected to depthwise convolution by assigning m 3x3 kernels with a stride of 2 to each of the m input channels of the AD image data for independent convolution operations, resulting in m channels of data after depthwise convolution. The m channels of data after depthwise convolution are then standardized using a batch normalization layer, and then a nonlinear transformation is performed on the data using an optimized ReLU(x) activation function. Where x is the channel data after standardization, max is the maximum value function, and c is a parameter that is always greater than 0; The channel data after optimizing the ReLU(x) activation function enters the pointwise convolution process: The pointwise convolution stage includes n convolutional layers, each of which includes m convolutional kernels of size 1×1 and stride 2. First, the m channel data after optimizing the ReLU(x) activation function are convolved with the m convolutional kernels in the first convolutional layer of the pointwise convolution stage, respectively, to output m channel data after passing through the first convolutional layer. Then, these m channel data are summed and aggregated into one channel data. In the same way, all m channel data after optimizing the ReLU(x) activation function are convolved with the n×m convolutional kernels in the pointwise convolution stage and aggregated. Finally, the entire pointwise convolution stage will produce n channel output data. After the AD image data is processed by all the residual units mentioned above, it is then processed by the GAM module to interact with global context information and local features. Finally, the output data of the channels of the ResNet-50 network model is connected and integrated by the first fully connected layer, and the first output layer outputs feature L. ResNet .

8. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 7, characterized in that, AD image data is processed by an improved Transformer module and the second output layer outputs feature L. Transformer The specific processing steps are as follows: First, the AD image data is divided into multiple small blocks to obtain patches. Then, the patches are transformed into a format that the Transformer module can process, resulting in tensor format patches. Subsequently, the tensor format patches are transformed into a high-dimensional space through an input embedding layer, converting the image region features into a feature matrix that the Transformer module can process more effectively. The feature matrix is ​​fed into the encoder part of the Transformer, the core of which is a pooling-based multi-head self-attention mechanism. The feature representation obtained after encoder processing enters the decoder stage. In the initial sub-layer of the decoder, a pooling-based masked multi-head self-attention mechanism is used. Subsequently, the feature representation is again fed into the encoder output through a multi-head self-attention mechanism. The decoder also integrates a feedforward neural network, residual connections, and layer normalization structures. After passing through the improved Transformer module, the feature representation is then connected and integrated by the second fully connected layer. Finally, the second output layer outputs feature L. Transformer .

9. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 8, characterized in that, It also includes the following step: after processing by the ResNet-50 model and the Transformer module, the first output layer outputs the feature L. ResNet and the output feature L from the second output layer Transformer The specific process for performing a full connection is as follows: Output feature L of the first output layer ResNet Second output layer output features L Transformer The data is then passed to a 3x3 global average pooling layer with a stride of 2 for feature compression and dimensionality reduction, and then connected through a third fully connected layer. The first output layer outputs feature L. ResNet Second output layer output features L Transformer The fusion characteristics after the third fully connected layer are: L combined =1 / a*L ResNet +1 / b*L Transformer Among them, L combined For fusion features, a is the adjustment of L ResNet The weighting parameter of the proportion, b is the adjustment L Transformer The weighting parameters of the proportion; The fused feature L after passing through the third fully connected layer combined The input is fed into the Softmax layer and the third output layer for processing; The Softmax layer and the third output layer affect L combined The specific processing procedure is as follows: First, L combined After being converted into relative probability values ​​by the Softmax layer, L combined After transformation by the Softmax layer, the sum of the predicted values ​​for each category is 1, and the predicted values ​​are output by the third output layer.

10. The method for early Alzheimer's disease image feature analysis based on the improved ResNet-Transformer model according to claim 8, characterized in that, When training the improved ResNet-Transformer model, the cross-entropy loss function CE is used. loss Calculate the loss; at each iteration, calculate the gradient of the loss function with respect to the model parameters, and then update the parameters based on the gradient. The network model converges when the loss function value no longer decreases significantly or reaches the preset stopping condition; after the model training is completed, all parameters need to be saved. Among them, CE loss It is the cross-entropy loss function, where s is the number of samples, i is any value from 1 to the total number of samples s, and p i Let q be the probability distribution of the true label. i This represents the probability distribution predicted by the model.

Images