GAN-enhanced ad prediction model data optimization method

Abstract

The embodiment of the application relates to a GAN enhanced AD prediction model data optimization method, comprising: acquiring real multi-modal image data of AD patients and corresponding clinical cognitive evaluation data; constructing a conditional generative adversarial network comprising a multi-scale perception generator and a pathological feature discriminator; inputting random noise vectors, clinical cognitive evaluation data and brain region anatomical labels into the generator after fusion to generate initial synthetic images meeting constraint conditions in anatomical structures and pathological features; optimizing generator parameters through adversarial training to obtain target synthetic images; mixing the target synthetic images with real images to construct a class-balanced enhanced training set; and training an AD prediction model using the enhanced training set. The application generates high-quality synthetic images through a dual mechanism of anatomical guidance and pathological constraint, effectively solves the problems of AD medical data scarcity and class imbalance, significantly improves the robustness, accuracy and explainability of the classification model, and has good clinical application value.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a data optimization method for an AD prediction model enhanced by GAN. Background Technology

[0002] Alzheimer's disease (AD) is a progressive neurodegenerative disease. Early and accurate diagnosis is crucial for slowing disease progression and improving patients' quality of life.

[0003] Neuroimaging techniques such as magnetic resonance imaging (MRI) and positron emission tomography (PET) have been widely used in computer-aided diagnosis of Alzheimer's disease (AD). Deep learning models such as convolutional neural networks (CNNs) have achieved excellent performance in AD classification tasks. However, existing technologies have the following limitations: First, due to factors such as patient privacy protection, high data collection costs, and difficulty in obtaining rare pathological samples, publicly available AD image datasets typically have limited sample sizes. This is especially true for a few categories, such as moderate dementia, where the sample size is severely insufficient, resulting in poor recognition ability of the trained classification models for minority classes and limited generalization performance.

[0004] Second, existing data augmentation methods for Generative Adversarial Networks (GANs) lack clinical guidance. While traditional GANs can generate synthetic images for data augmentation, the generation process relies solely on random noise and does not consider the pathological features and anatomical constraints of Alzheimer's disease (AD). The generated images may be visually realistic, but the atrophy patterns in key brain regions (such as the hippocampus and entorhinal cortex) do not match the actual pathology, making it difficult to effectively improve the performance of downstream classification tasks.

[0005] Third, AD diagnosis typically requires combining multidimensional information from structural magnetic resonance imaging (sMRI), fluorodeoxyglucose positron emission tomography (FDG-PET), and clinical cognitive assessment scales (such as the Mini-Mental State Examination (MMSE) and the Clinical Dementia Rating Scale (CDR)). Existing methods often use only single-modal data or simply stitch together multimodal features, failing to achieve deep intermodal information fusion and guidance.

[0006] Fourth, deep learning models are often considered "black boxes" because their decision-making processes lack transparency and are difficult for clinicians to trust. Existing methods pay little attention to the interpretability of the models and fail to establish a correlation between the decision-making processes and pathological changes in key brain regions of Alzheimer's disease (AD).

[0007] Therefore, there is an urgent need for a data optimization method that can combine multimodal clinical data, generate high-quality synthetic images under the dual constraints of anatomical structure and pathological features, and improve the performance and interpretability of classification models. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by providing a data optimization method for AD prediction models based on GAN enhancement, thereby solving the problems existing in the prior art.

[0009] To achieve the above objectives, this invention provides a data optimization method for GAN-enhanced AD prediction models, the method comprising: To acquire real multimodal imaging data and corresponding clinical cognitive assessment data of Alzheimer's patients; A conditional generative adversarial network is constructed; the conditional generative adversarial network includes a multi-scale perceptual generator and a pathological feature discriminator; The random noise vector is fused with the clinical cognitive assessment data and the brain region anatomical labels pre-generated based on standard brain atlases. The fused feature vector is then used as input to the multi-scale perception generator to generate initial synthetic image data that is consistent with the distribution of the real multimodal image data, conforms to the brain region anatomical label definition in terms of anatomical structure, and matches the clinical cognitive assessment data in terms of pathological features. The real multimodal image data is used as a positive sample, and the initial synthetic image data is used as a negative sample. Both are input into the pathological feature discriminator for adversarial training to optimize the parameters of the multi-scale perception generator until the pathological feature discriminator can no longer distinguish between the real multimodal image data and the initial synthetic image data. The output of the multi-scale perception generator after parameter optimization is the target synthetic image data. The target synthetic image data is mixed with the real multimodal image data to construct a class-balanced augmented training set, such that the maximum difference between the number of samples in each class does not exceed a preset proportion threshold. The initial neural network classification model is trained using the enhanced training set to obtain an Alzheimer's disease prediction model.

[0010] In one possible implementation, acquiring real multimodal image data and corresponding clinical cognitive assessment data of Alzheimer's patients specifically includes: Structural magnetic resonance imaging data and fluorodeoxyglucose positron emission tomography data from multiple subjects were collected as the real multimodal imaging data; Each subject underwent neuropsychological scale assessment to obtain multidimensional scoring data, including Mini-Mental State Examination scores, total scores of the Clinical Dementia Rating Scale, and scores of the Alzheimer's Disease Rating Scale-Cognitive Subscale, as the clinical cognitive assessment data. The structural magnetic resonance imaging data and fluorodeoxyglucose positron emission tomography data were sequentially spatially registered to the standard Montreal Neurological Institute space, skew field correction, grayscale normalization, and cranial dissection preprocessing. The preprocessed multimodal image data and the multidimensional scoring data are associated and stored according to the subject identifiers to form a paired dataset.

[0011] In one possible implementation, constructing a conditional generative adversarial network includes: A multi-scale perception generator is constructed, which adopts an encoder-decoder architecture. The encoder part includes multiple parallel residual blocks with different porosity to extract multi-scale features of different receptive fields. The decoder part fuses the features of the corresponding layer of the encoder through skip connections, and introduces an attention gating module based on brain region anatomical labels at the skip connections to enhance the expression of key brain region features. A pathological feature discriminator is constructed, which is a fully convolutional network. Its output layer includes a first branch and a second branch in parallel. The first branch outputs the probability that the input image is real or synthetic through a global average pooling layer and a fully connected layer. The second branch outputs the classification probability of the disease stage to which the input image belongs through a global average pooling layer and a fully connected layer. The disease stages include normal, mild cognitive impairment and Alzheimer's disease. The conditional generative adversarial network is constructed by jointly optimizing the multi-scale perception generator and the pathological feature discriminator through a weighted sum of adversarial loss function and auxiliary classification loss function.

[0012] In one possible implementation, the step of fusing random noise vectors with clinical cognitive assessment data and brain region anatomical labels, and inputting this data into a multi-scale perception generator to generate initial synthetic image data, specifically includes: The random noise vector is copied and expanded into a noise feature map with the same spatial size as the brain region anatomical label; The clinical cognitive assessment data is converted into a one-dimensional conditional vector output by a fully connected layer, and the one-dimensional conditional vector is copied and expanded into a clinical conditional feature map with the same spatial size as the brain region anatomical labels. The noise feature map, the clinical condition feature map, and the brain region anatomical labels are concatenated along the channel dimension to obtain a multi-channel fusion tensor. The multi-channel fusion tensor is input into the multi-scale sensing generator; The multi-scale perception generator encodes and decodes the multi-channel fused tensor using a three-dimensional convolutional kernel. During the encoding process, multi-scale features are extracted through convolutional layers with different dilation rates. During the decoding process, spatial details are restored through skip connections. Finally, the generator outputs initial synthetic image data with the same size and modality as the real multimodal image data.

[0013] In one possible implementation, the step of inputting real multimodal image data and initial synthetic image data into a pathological feature discriminator for adversarial training to optimize generator parameters specifically includes: A joint loss function is constructed, wherein the joint loss function includes a weighted sum of adversarial loss and pathological classification loss; the adversarial loss uses Wasserstein distance to calculate the score difference between the pathological feature discriminator's output of real images and synthetic images; the pathological classification loss uses the cross-entropy loss function to calculate the error between the disease stage predicted by the pathological feature discriminator for the input image and the real disease stage labeled by the corresponding clinical cognitive assessment data. In each iteration of training, the generator parameters are first fixed, and the pathological feature discriminator parameters are updated with the goal of minimizing the joint loss of the pathological feature discriminator. Then, the pathological feature discriminator parameters are fixed, and the generator parameters are updated with the goal of minimizing the joint loss of the generator. The joint loss of the generator is the negative value of the pathological feature discriminator's score on the synthetic image plus a pathological classification loss term. The joint loss of the generator also includes a perceptual loss term, which is calculated based on the distance between the feature maps of the initial synthetic image data and the real multimodal image data at at least one network layer by the pre-trained medical image feature extraction network. Repeat the above iterative process until the average score difference between the pathological feature discriminator and the synthetic image is less than a preset threshold, and the pathological classification accuracy of the pathological feature discriminator for the synthetic image is lower than a preset random level threshold.

[0014] In one possible implementation, the step of mixing the target synthetic image data with real multimodal image data to construct a class-balanced augmented training set specifically includes: The distribution of sample numbers for different disease categories in the real multimodal image data is statistically analyzed, and categories with sample numbers below a preset equilibrium threshold are identified as minority categories. Based on the preset category balancing strategy, calculate the category and quantity of target synthetic image data required to supplement the sample data of each category to achieve the desired balance ratio; The multi-scale perception generator, after training convergence, is invoked to generate a corresponding number of target synthetic image data based on the clinical cognitive assessment data and brain region anatomical labels corresponding to the minority class. The quality consistency of the generated target synthetic image data is checked, and outlier samples that differ significantly from the feature distribution of similar real image data are removed. The verified target synthetic image data is merged with the real multimodal image data, and the merged dataset is randomly rearranged to construct an enhanced training set in which the distribution of the number of samples in each category meets the preset balance requirements.

[0015] In one possible implementation, training the initial neural network classification model using an enhanced training set to obtain an Alzheimer's disease prediction model specifically includes: A multimodal fusion classification network is constructed as the initial neural network classification model. The multimodal fusion classification network includes an image feature extraction branch and a clinical feature fusion layer. The image feature extraction branch adopts a two-stream three-dimensional convolutional neural network structure. The first stream input is the structural magnetic resonance imaging data in the enhanced training set, and the second stream input is the fluorodeoxyglucose positron emission tomography data in the enhanced training set. Structural features and metabolic features are extracted respectively, and then fused in the deep layer by feature splicing or addition to obtain a multimodal image feature vector. The clinical feature fusion layer takes the clinical cognitive assessment data corresponding to the enhanced training set as input, maps it to a clinical feature vector through a fully connected network, and concatenates the clinical feature vector with the multimodal image feature vector to obtain a joint feature representation; The joint feature representation is input into a classification head consisting of multiple fully connected layers, and finally outputs the probability that the input sample belongs to normal, mild cognitive impairment, or Alzheimer's disease through a softmax layer. The multimodal fusion classification network is trained in an end-to-end manner using the cross-entropy loss function to minimize the error between the predicted probability and the true label, and the model parameters after training convergence are saved as the final Alzheimer's disease prediction model.

[0016] In one possible implementation, before feature fusion of the random noise vector with clinical cognitive assessment data and brain region anatomical labels, the method further includes: For subjects with missing scores on the scales, multiple imputation was used to fill in the missing values ​​based on the score distribution of the complete sample, resulting in a complete clinical cognitive assessment data vector. Gaussian noise was added to the complete clinical cognitive assessment data vector to generate multiple perturbation versions of the clinical condition vector, thereby increasing the generator's robustness to slight fluctuations in clinical scores during training.

[0017] In one possible implementation, the adversarial training process further includes: The initial synthetic image data and the corresponding real multimodal image data are respectively input into a pre-trained medical image feature extraction network to obtain their feature maps in multiple network layers. Calculate the L1 distance between the initial synthetic image data and the real multimodal image data in each selected layer feature map, and use it as the perceptual loss term; The perceptual loss term is added to the joint loss function of the generator and weighted and summed with the adversarial loss and pathological classification loss to constrain the generated image to maintain consistency with the real image in high-level semantic features.

[0018] In one possible implementation, after obtaining the Alzheimer's disease prediction model, the method further includes: The test sample is input into the Alzheimer's disease prediction model to obtain the predicted probability and the corresponding category activation heatmap output by the model. The category activation heatmap is generated by the gradient weighted class activation mapping method. The category activation heatmap is overlaid with the brain region anatomical labels of the test samples, and the overlap between high-response areas in the heatmap and key brain regions of AD is statistically analyzed; wherein, the key brain regions of AD include the hippocampus and the entorhinal cortex. When the overlap is lower than a preset threshold, the classification head of the prediction model is fine-tuned until the model's decision basis matches clinical prior knowledge, thus obtaining an interpretable Alzheimer's disease prediction model.

[0019] By applying the GAN-enhanced AD prediction model data optimization method provided in this invention, and using clinical cognitive assessment data and brain region anatomical labels as conditional constraints input to a multi-scale perception generator, the generated synthetic images not only match the real images in pixel distribution but also conform to the brain region label constraints in anatomical structure and match the clinical scores in pathological features. Compared with traditional unconstrained GANs, the images generated by this invention have higher structural similarity and can accurately reflect brain atrophy patterns at different disease stages. Furthermore, this application generates targeted synthetic images of minority classes by statistically analyzing the distribution of samples of each category in real data, constructing a class-balanced enhancement training set. Further, this application uses a two-stream three-dimensional convolutional neural network to process sMRI and FDG-PET data respectively, and integrates clinical cognitive assessment features to fully utilize the complementarity of multimodal information. The classification model trained on the enhancement training set shows a significant improvement in classification accuracy compared to a model using only real data, and exhibits better robustness to fluctuations in data distribution. Furthermore, by generating a category activation heatmap and overlaying it with brain region anatomical labels, the key brain regions on which the model's decisions are based are visualized. When the overlap between high-response regions and key AD brain regions (hippocampus, entorhinal cortex) is below a preset threshold, the model is fine-tuned to ensure that the model's decision-making aligns with prior clinical knowledge. This makes the model output not only accurate but also interpretable, enhancing clinicians' trust. Furthermore, this application employs multiple imputation to fill in missing scale scores and adds Gaussian noise to complete scores to generate perturbation versions, enabling the generator to adapt to common score fluctuations in clinical assessments and ensuring stable quality of the generated synthetic images under different clinical conditions. Further, this application introduces perceptual consistency constraints and a quality verification mechanism based on a pre-trained medical image feature extraction network to ensure that the synthetic images maintain consistency with real images in high-level semantic features, eliminating outliers with significantly different feature distributions and guaranteeing the overall quality of the enhanced training set. In summary, this invention, through an anatomically guided and pathologically constrained generative adversarial network, effectively solves the problems of scarce and imbalanced AD medical data, generating high-quality, clinically usable synthetic images, significantly improving the performance, robustness, and interpretability of AD prediction models, and demonstrating promising clinical application prospects. Attached Figure Description

[0020] Figure 1 Flowchart of the GAN-enhanced AD prediction model data optimization method provided by the present invention; Figure 2 A schematic diagram of the multi-scale perception generator structure provided by the present invention; Figure 3 A schematic diagram of the pathological feature discriminator provided by the present invention; Figure 4 This is a schematic diagram of the Alzheimer's disease prediction model provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0023] Figure 1 This is a flowchart of a GAN-enhanced Alzheimer's disease (AD) prediction model data optimization method. The execution entity of this method is a control system, which includes servers, processors, etc., with processing and computing capabilities (hereinafter referred to as the system). It should be noted that the core of the method described in this invention lies in optimizing the training data through GAN enhancement. The resulting Alzheimer's disease prediction model is a direct product of this optimization method, rather than an improvement on the prediction model itself. Figure 1 As shown, the method includes the following steps: Step 110: Obtain real multimodal image data and corresponding clinical cognitive assessment data of Alzheimer's patients.

[0024] Specifically, acquiring real multimodal image data and corresponding clinical cognitive assessment data of Alzheimer's patients includes: Structural magnetic resonance imaging data and fluorodeoxyglucose positron emission tomography data from multiple subjects were collected as the real multimodal imaging data; Each subject underwent neuropsychological scale assessment to obtain multidimensional scoring data, including Mini-Mental State Examination scores, total scores of the Clinical Dementia Rating Scale, and scores of the Alzheimer's Disease Rating Scale-Cognitive Subscale, as the clinical cognitive assessment data. The structural magnetic resonance imaging data and fluorodeoxyglucose positron emission tomography data were sequentially spatially registered to the standard Montreal Neurological Institute space, skew field correction, grayscale normalization, and cranial dissection preprocessing. The preprocessed multimodal image data and the multidimensional scoring data are associated and stored according to the subject identifiers to form a paired dataset.

[0025] The clinical cognitive assessment data described in this application includes at least three scores: the Mini-Mental State Examination (MMSE), the Clinical Dementia Rating Scale (CDR), and the Alzheimer's Disease Rating Scale-Cognitive Subscale (ADAS-Cog). This complete vector composed of these three scores is used in subsequent feature fusion, adversarial training, and classification model training. For subjects lacking a particular score, multiple imputation is employed, i.e., missing values ​​are filled based on the score distribution of the complete sample.

[0026] Step 120: Construct a conditional generative adversarial network; the conditional generative adversarial network includes a multi-scale perceptual generator and a pathological feature discriminator.

[0027] The construction of a conditional generative adversarial network specifically includes: Construct a multi-scale sensing generator, such as Figure 2 As shown, the multi-scale perception generator adopts an encoder-decoder architecture and contains residual blocks with different porosity rates; a pathological feature discriminator is constructed, such as... Figure 3 As shown, the pathological feature discriminator is a fully convolutional network that outputs true / false probabilities and disease stage classification probabilities. The encoder portion includes multiple parallel residual blocks with different dilatancy rates to extract multi-scale features from different receptive fields. The decoder portion fuses features from the corresponding layer of the encoder via skip connections and introduces an attention gating module based on brain region anatomical labels at the skip connections to enhance the expression of key brain region features. A pathological feature discriminator is constructed, which is a fully convolutional network. Its output layer includes a first branch and a second branch in parallel. The first branch outputs the probability that the input image is real or synthetic through a global average pooling layer and a fully connected layer. The second branch outputs the classification probability of the disease stage to which the input image belongs through a global average pooling layer and a fully connected layer. The disease stages include normal, mild cognitive impairment and Alzheimer's disease. The conditional generative adversarial network is constructed by jointly optimizing the multi-scale perception generator and the pathological feature discriminator through a weighted sum of adversarial loss function and auxiliary classification loss function.

[0028] It should be noted that the conditional generative adversarial network described in this invention is not a general cGAN architecture, but an improved cGAN specifically designed for the characteristics of AD medical data. Its unique features are: ① using clinical cognitive assessment data as the global pathological condition; ② using brain region anatomical labels as the local spatial condition; ③ capturing lesion features at different scales through a multi-scale perception generator; and ④ jointly optimizing true / false discrimination and disease stage classification through a pathological feature discriminator.

[0029] like Figure 2As shown in the figure, the generator adopts an encoder-decoder architecture. The encoder includes parallel residual blocks with different hole rates (d=1,2,4) for extracting multi-scale features. The decoder fuses with the encoder features through skip connections and sets an attention gating module based on brain region anatomical labels at the skip connections to enhance the feature expression of key brain regions such as the hippocampus and entorhinal cortex.

[0030] The multi-scale perception generator is a dedicated generative network designed specifically for the multi-scale characteristics of Alzheimer's disease (AD) lesions and the importance of key brain regions. Its encoder extracts pathological features at different scales through parallel multi-cavitation rate residual blocks, while its decoder enhances the feature representation of key brain regions by introducing an attention gating module based on brain region anatomical labels, thereby achieving refined control over the generated images in terms of anatomical structure and pathological features. The multi-scale perception generator described in this invention is a generative network specifically designed for the characteristics of medical images; the "multi-scale perception" in its name encompasses the following two meanings: First, multi-scale feature extraction. AD pathological changes exhibit different spatial scale characteristics in different brain regions and at different disease stages: early AD often manifests as subtle atrophy of the entorhinal cortex (small-scale feature), intermediate AD as significant reduction in hippocampal volume (medium-scale feature), and late AD as diffuse atrophy of the entire brain (large-scale feature). To comprehensively capture these multi-scale pathological changes, this invention designs multiple parallel residual blocks in the encoder section, each with a different porosity (d=1,2,4), thereby obtaining receptive fields of different sizes without increasing computational load, achieving multi-scale feature extraction from local details to global structure.

[0031] Second, enhanced perception of key brain regions. Medical image generation requires not only global realism but also ensuring the anatomical accuracy of key diagnostic areas. To this end, this invention introduces an attention-gating module based on brain region anatomical tags at the skip connections of the decoder. This module utilizes segmentation masks of brain region anatomical tags, such as the hippocampus and entorhinal cortex, as spatial guidance to calculate the feature importance weight of each spatial location, enhancing the expression of features in key brain regions and suppressing features in irrelevant regions. This design ensures the fidelity of the anatomical structure of key brain regions in AD in the generated images.

[0032] In one example, the generator structure based on a 96×96×96 input size is as follows: The encoder contains four downsampling blocks, each consisting of a 3×3×3 convolution with a stride of 2 and 64-128-256-512 channels, followed by batch normalization and LeakyReLU activation. Parallel dilated residual blocks with dilation rates of 1, 2, and 4 are set in layers 2, 3, and 4, respectively. Each dilated residual block contains two 3×3×3 convolutional layers with the same dilation rate, and the residual connections directly add the input to the output.

[0033] The decoder contains four upsampling blocks, each of which is a transposed convolution with a stride of 2 to restore the spatial size, with the number of channels halved step by step (512-256-128-64).

[0034] Before each upsampling block in the decoder, features from the corresponding encoder layer are fused via skip connections, with an attention gating module introduced before fusion. The calculation formula for the attention gating module is: attention = sigmoid(W_a*(concat(encoded features, brain region anatomical label features))+b_a), and the output is the attention weight, which is multiplied by the encoded features before being fed into the decoder. The final output layer uses a 1×1×1 convolution to reduce the number of channels to 1, and then performs Tanh activation to generate the synthetic image.

[0035] The following is an example of the specific structure of the discriminator: The shared feature extraction part consists of four 3×3×3 convolutional layers with 64-128-256-512 channels. Each layer is followed by batch normalization and LeakyReLU activation, with a stride of 2, achieving spatial downsampling to 6×6×6. Global average pooling yields a 512-dimensional feature vector.

[0036] The first branch (true / false discrimination) is a fully connected layer of 512-256-1, with no activation in the last layer, outputting a real score for use in Wasserstein loss.

[0037] The second branch (disease classification) consists of a fully connected layer 512-256-3, and the last layer is Softmax.

[0038] The pathological feature discriminator is a dual-task output network. Its first branch performs traditional true / false discrimination, and its second branch performs auxiliary classification of disease stages. By jointly optimizing the adversarial loss and the pathological classification loss, the discriminator not only distinguishes between real and synthetic images, but also forces the generator to learn pathological features related to disease progression, thereby generating synthetic images that are clinically reliable.

[0039] Figure 3This is a schematic diagram of the pathological feature discriminator provided by the present invention. As shown in the figure, the discriminator uses a fully convolutional network to extract shared features, which are then split into two parallel branches after global average pooling: the first branch (left) activates the probability of the input image being real or synthetic through a fully connected layer and a sigmoid function; the second branch (right) activates the probability of the input image belonging to a disease stage, including the classification probability of normal / mild cognitive impairment / Alzheimer's disease, through a fully connected layer and a softmax function. The pathological feature discriminator of the present invention is a dual-task discriminator network specifically designed for medical image generation tasks. The pathological features in its name embody the following core design ideas: First, the discriminator features a dual-task design. Conventional GAN ​​discriminators only need to output the probability of the input image being real or synthetic, such as in a binary classification task. This invention adds a second branch—a disease stage classification task—requiring the discriminator to simultaneously output the probability that the input image belongs to normal (NC), mild cognitive impairment (MCI), or Alzheimer's disease (AD), such as in a tri-class classification task. The purpose of this design is: This imposes stronger constraints on the generator: the generator must not only generate realistic images, but also images that can be accurately classified into the correct disease stage, forcing the generator to learn the inherent laws of disease evolution, rather than merely imitating at the pixel level.

[0040] Regularizing the discriminator itself: The auxiliary classification task provides additional supervision signals for the discriminator's feature extraction layer, which helps the discriminator learn more discriminative pathological features and improve the accuracy of true / false discrimination.

[0041] Second, the implementation of a two-branch structure. For example... Figure 3 As shown, the pathological feature discriminator uses a fully convolutional network as the shared feature extraction backbone, followed by a global average pooling layer to compress spatial features into a feature vector. This feature vector is then branched into two parallel fully connected branches: The first branch (true / false discrimination branch) includes multiple fully connected layers. The final output layer uses the Sigmoid activation function, and the output range is [0,1], representing the probability that the input image is a true image.

[0042] The second branch (disease classification branch) includes multiple fully connected layers. The final output layer uses the Softmax activation function, and the sum of the three class probabilities (NC / MCI / AD) is 1.

[0043] The two branches share the underlying feature extraction network, but perform task-specific parameter learning at higher levels, thus promoting mutual learning across multiple tasks.

[0044] Third, the design of the joint optimization loss function. The training objective of the pathological feature discriminator is to minimize the joint loss function: L_D = L_adv + λ·L_cls In this design, L_adv represents the adversarial loss using Wasserstein distance, L_cls represents the pathological classification cross-entropy loss, and λ is the balancing weight coefficient, which can be set to 10. This joint optimization ensures that the discriminator, while focusing on true / false discrimination, does not neglect the modeling of pathological features. It is important to note that the multi-scale perception generator and the pathological feature discriminator of this invention are co-designed: the generator's attention gating module relies on brain region anatomical labels to ensure the accuracy of the anatomical structure in the generated image, while the discriminator's second branch supervises the accuracy of the pathological features in the generated image. Together, they constitute a dual guarantee mechanism of anatomical guidance and pathological constraint, which is the core innovation that distinguishes this invention from existing technologies.

[0045] It should be noted that the conditional generative adversarial network described in this invention, comprising a multi-scale perceptual generator and a pathological feature discriminator, is one of two independent networks used sequentially with the subsequent Alzheimer's disease prediction model: the former is used to generate synthetic images to construct a class-balanced augmented training set, belonging to the data optimization stage; the latter is trained on this augmented training set as the final diagnostic classifier. They are not the same network and do not share parameters.

[0046] Step 130: The random noise vector is fused with the clinical cognitive assessment data and the brain region anatomical labels pre-generated based on the standard brain atlas. The fused feature vector is then input into the multi-scale perception generator to generate initial synthetic image data that is consistent with the distribution of the real multimodal image data, conforms to the brain region anatomical label definition in terms of anatomical structure, and matches the clinical cognitive assessment data in terms of pathological features.

[0047] In this embodiment, the brain region anatomical labels are generated using an Automated Anatomical Labeling (AAL) atlas. The specific steps are as follows: Structural magnetic resonance imaging data for each subject is mapped to a standard MNI space via nonlinear registration; the AAL atlas, comprising 116 brain regions, is inversely mapped back to the individual space to obtain the brain region attribution label for each voxel; a binary mask is generated for each brain region, and all brain region masks are superimposed along the channel dimension to form a brain region anatomical label tensor with dimensions of 96×96×96×116. This label tensor serves as a spatial constraint, guiding the generator to produce image features consistent with anatomical structures in key brain regions, such as the hippocampus and entorhinal cortex.

[0048] Specifically, the step of fusing random noise vectors with clinical cognitive assessment data and brain region anatomical labels, and inputting this data into a multi-scale perception generator to generate initial synthetic image data, includes: The random noise vector is copied and expanded into a noise feature map with the same spatial size as the brain region anatomical label; The clinical cognitive assessment data is converted into a one-dimensional conditional vector output by a fully connected layer, and the one-dimensional conditional vector is copied and expanded into a clinical conditional feature map with the same spatial size as the brain region anatomical labels. The noise feature map, the clinical condition feature map, and the brain region anatomical labels are concatenated along the channel dimension to obtain a multi-channel fusion tensor. The multi-channel fusion tensor is input into the multi-scale sensing generator; The multi-scale perception generator encodes and decodes the multi-channel fused tensor using a three-dimensional convolutional kernel. During the encoding process, multi-scale features are extracted through convolutional layers with different dilation rates. During the decoding process, spatial details are restored through skip connections. Finally, the generator outputs initial synthetic image data with the same size and modality as the real multimodal image data.

[0049] In a preferred embodiment of the present invention, the brain region anatomical labels are generated using an Automated Anatomical Labeling (AAL) atlas, comprising 116 brain regions. Each channel corresponds to a binary mask of one brain region, with a spatial size of 96×96×96 voxels. The feature fusion specifically includes: The 1×1×1×100 dimension random noise vector is copied and expanded into a 96×96×96×100 noise feature map with the same spatial size as the brain region anatomical labels; Clinical cognitive assessment data, including MMSE, CDR, and ADAS-Cog scores, are input into a fully connected layer, mapped to a 1×1×1×32 one-dimensional conditional vector, and then replicated and expanded to a 96×96×96×32 clinical conditional feature map. The noise feature map (96×96×96×100), the clinical condition feature map (96×96×96×32), and the brain region anatomical labels (96×96×96×116) are spliced ​​in the channel dimension to obtain a multi-channel fusion tensor of 96×96×96×248. Input the multi-channel fused tensor Figure 2 The multi-scale sensing generator shown outputs initial synthetic image data of the same size (96×96×96×1) as the real structural magnetic resonance imaging data after encoding-decoding processing.

[0050] It should be noted that the spatial dimensions of 96×96×96 voxels, 100-dimensional noise vector, 32-dimensional conditional vector, and 116-dimensional number of anatomical label channels mentioned above are preferred parameters in the embodiments of the present invention. In actual applications, they can be adjusted according to specific data characteristics and computing resources, and do not constitute a limitation on the present invention.

[0051] Step 140: The real multimodal image data is used as a positive sample, and the initial synthetic image data is used as a negative sample. Both are input into the pathological feature discriminator for adversarial training to optimize the parameters of the multi-scale perception generator until the pathological feature discriminator can no longer distinguish between the real multimodal image data and the initial synthetic image data; wherein, the output of the multi-scale perception generator after parameter optimization is the target synthetic image data.

[0052] Specifically, the step of inputting real multimodal image data and initial synthetic image data into the pathological feature discriminator for adversarial training to optimize the generator parameters includes: A joint loss function is constructed, wherein the joint loss function includes a weighted sum of adversarial loss and pathological classification loss; the adversarial loss uses Wasserstein distance to calculate the score difference between the pathological feature discriminator's output of real images and synthetic images; the pathological classification loss uses the cross-entropy loss function to calculate the error between the disease stage predicted by the pathological feature discriminator for the input image and the real disease stage labeled by the corresponding clinical cognitive assessment data. In each iteration of training, the generator parameters are first fixed, and the pathological feature discriminator parameters are updated with the goal of minimizing the joint loss of the pathological feature discriminator. Then, the pathological feature discriminator parameters are fixed, and the generator parameters are updated with the goal of minimizing the joint loss of the generator. The joint loss of the generator is the negative value of the pathological feature discriminator's score on the synthetic image plus a pathological classification loss term. The joint loss of the generator also includes a perceptual loss term, which is calculated based on the distance between the feature maps of the initial synthetic image data and the real multimodal image data at at least one network layer by the pre-trained medical image feature extraction network. Repeat the above iterative process until the average score difference between the pathological feature discriminator and the synthetic image is less than a preset threshold, and the pathological classification accuracy of the pathological feature discriminator for the synthetic image is lower than a preset random level threshold.

[0053] The adversarial training employs a joint loss function: L_D = L_adv + λ·L_cls L_G= -L_adv + μ·L_cls + ν·L_percep In the formula, L_adv is the Wasserstein adversarial loss, whose score range is usually between [-1, 1]; L_cls is the pathological classification cross-entropy loss; and L_percep is the perceptual loss. λ, μ, and ν are balancing weight coefficients, which are set to 10, 1, and 5 respectively in this embodiment. It should be noted that the above weight coefficients are preferred values, and in actual applications, they can be adjusted according to the balance requirements between generation quality and classification accuracy. For example, the value range of λ can be 5-20, the value range of μ can be 0.5-5, and the value range of ν can be 1-10.

[0054] During training, an alternating update strategy was adopted, updating the generator parameters once after every 5 updates to the discriminator parameters. After 200 iterations of training, the average score difference between the discriminator's output of real and synthetic images was less than the preset threshold of 0.1, and the pathological classification accuracy of the discriminator for synthetic images was lower than the preset random level threshold of 0.35. Based on the three-class classification random guess accuracy of 33.3%, training was stopped, and the multi-scale perception generator with optimized parameters was obtained.

[0055] The perceptual loss L_percep is used to constrain the generated image to maintain consistency with the real image in high-level semantic features, and its calculation method is as follows: A three-dimensional residual network (3D-ResNet) pre-trained on large medical image datasets, such as BraTS and IXI, is used as a feature extractor. Its first, third, and fifth layers are selected as feature extraction layers, and the network parameters are kept fixed during the perceptual loss calculation.

[0056] The initial synthetic image data I_syn and the corresponding real multimodal image data I_real are input into the pre-trained network to obtain feature maps for each selected layer, denoted as φ1(I_syn), φ3(I_syn), φ5(I_syn) and φ1(I_real), φ3(I_real), φ5(I_real). Here, φ_l(·) represents the feature extraction function of the l-th layer, and the size of the output feature map depends on the network structure of that layer.

[0057] For each selected layer, the L1 distance between the synthetic image and the real image in that layer's feature map is calculated. The L1 distance is more robust to outliers and preserves clearer edge information compared to the L2 distance. It is calculated as the sum of the absolute values ​​of the element-wise differences: D_l = ||φ_l(I_syn) - φ_l(I_real)||1, where ||·||1 represents the L1 norm, and l = 1, 3, 5.

[0058] The L1 distances of each layer are weighted and summed to obtain the final perceptual loss: L_percep = w1·D1 + w3·D3 + w5·D5, where w1, w3, and w5 are the weight coefficients of each layer, which are set to 0.2, 0.3, and 0.5 respectively in this embodiment, and the sum of the weights is 1. This weight setting makes the perceptual loss focus more on high-level semantic features (the 5th layer has the highest weight), while also taking into account shallow texture features (the 1st and 3rd layers).

[0059] The calculated L_percep is multiplied by the balancing weight coefficient ν and then added to the joint loss function of the generator: L_G = -L_adv + μ·L_cls + ν·L_percep, where ν can be set to 5 (see the preferred embodiment in step 140).

[0060] By introducing perceptual loss, the generator is guided to produce synthetic data that is not only similar to real images in pixel distribution, but also consistent with real images in high-level semantic features, such as brain region contours and texture patterns, thereby further improving the quality and clinical usability of synthetic images.

[0061] Step 150: Mix the target synthetic image data with the real multimodal image data to construct a class-balanced augmented training set, such that the maximum difference between the number of samples in each class does not exceed a preset ratio threshold.

[0062] The preset ratio threshold is an empirical value derived from multiple experiments, and this application does not limit it. The step of mixing the target synthetic image data with real multimodal image data to construct a class-balanced augmented training set specifically includes: The distribution of sample numbers for different disease categories in the real multimodal image data is statistically analyzed, and categories with sample numbers below a preset equilibrium threshold are identified as minority categories. Based on the preset category balancing strategy, calculate the category and quantity of target synthetic image data required to supplement the sample data of each category to achieve the desired balance ratio; The multi-scale perception generator, after training convergence, is invoked to generate a corresponding number of target synthetic image data based on the clinical cognitive assessment data and brain region anatomical labels corresponding to the minority class. The quality consistency of the generated target synthetic image data is checked, and outlier samples that differ significantly from the feature distribution of similar real image data are removed. The verified target synthetic image data is merged with the real multimodal image data, and the merged dataset is randomly rearranged to construct an enhanced training set in which the distribution of the number of samples in each category meets the preset balance requirements.

[0063] In this embodiment, the preset balance requirement is defined as the ratio of the number of samples in each category to the category with the most samples being no less than 0.8, that is, the maximum difference does not exceed 20%. In practical applications, this ratio threshold can be set according to task requirements, for example, between 0.7 and 1.0.

[0064] The quality consistency verification can employ kernel density estimation to calculate the probability density of generated samples in the feature space, eliminating outliers whose density is below the 5th percentile of the true sample distribution. This embodiment uses the 5th percentile as the outlier threshold. In practical applications, this threshold can be adjusted within the range of 1%-10% based on the total sample size and data distribution characteristics of the training set; a lower percentile, such as 1%, can be used when the sample size is large, while it can be appropriately relaxed to 10% when the sample size is small.

[0065] Step 160: Train the initial neural network classification model using the enhanced training set to obtain an Alzheimer's disease prediction model.

[0066] The step of training the initial neural network classification model using an enhanced training set to obtain an Alzheimer's disease prediction model specifically includes: A multimodal fusion classification network is constructed as the initial neural network classification model. The multimodal fusion classification network includes an image feature extraction branch and a clinical feature fusion layer. The image feature extraction branch adopts a two-stream three-dimensional convolutional neural network structure. The first stream input is the structural magnetic resonance imaging data in the enhanced training set, and the second stream input is the fluorodeoxyglucose positron emission tomography data in the enhanced training set. Structural features and metabolic features are extracted respectively, and then fused in the deep layer by feature splicing or addition to obtain a multimodal image feature vector. The clinical feature fusion layer takes the clinical cognitive assessment data corresponding to the enhanced training set as input, maps it to a clinical feature vector through a fully connected network, and concatenates the clinical feature vector with the multimodal image feature vector to obtain a joint feature representation; The joint feature representation is input into a classification head consisting of multiple fully connected layers, and finally outputs the probability that the input sample belongs to normal, mild cognitive impairment, or Alzheimer's disease through a softmax layer. The multimodal fusion classification network is trained in an end-to-end manner using the cross-entropy loss function to minimize the error between the predicted probability and the true label, and the model parameters after training convergence are saved as the final Alzheimer's disease prediction model.

[0067] like Figure 4 As shown, in this embodiment, the specific structure of the multimodal fusion classification network is as follows: Each stream contains 5 three-dimensional convolutional blocks. Each convolutional block consists of two 3×3×3 convolutional layers with channel numbers of 32, 64, 128, 256, and 512 respectively, followed by batch normalization, ReLU activation, and 2×2×2 max pooling.

[0068] The fifth pooling layer outputs a 512-dimensional feature vector, and the two stream features are fused by element-wise addition.

[0069] Clinical cognitive assessment data (MMSE, CDR, ADAS-Cog) are input into a three-layer fully connected network (3→64→32) to obtain a 32-dimensional clinical feature vector.

[0070] The fused image features (512 dimensions) and clinical features (32 dimensions) are concatenated into a 544-dimensional joint feature, which is then passed through fully connected layers 544→256→128→3 in sequence. Finally, the Softmax layer outputs the three probabilities.

[0071] Furthermore, before fusing the random noise vector with clinical cognitive assessment data and brain region anatomical labels, the process also includes: For subjects with missing scores on the scales, multiple imputation was used to fill in the missing values ​​based on the score distribution of the complete sample, resulting in a complete clinical cognitive assessment data vector. Gaussian noise was added to the complete clinical cognitive assessment data vector to generate multiple perturbation versions of the clinical condition vector, thereby increasing the generator's robustness to slight fluctuations in clinical scores during training.

[0072] Furthermore, the adversarial training process also includes: The initial synthetic image data and the corresponding real multimodal image data are respectively input into a pre-trained medical image feature extraction network to obtain their feature maps in multiple network layers. Calculate the L1 distance between the initial synthetic image data and the real multimodal image data in each selected layer feature map, and use it as the perceptual loss term; The perceptual loss term is added to the joint loss function of the generator and weighted and summed with the adversarial loss and pathological classification loss to constrain the generated image to maintain consistency with the real image in high-level semantic features.

[0073] Furthermore, after obtaining the Alzheimer's disease prediction model, the method further includes: The test sample is input into the Alzheimer's disease prediction model to obtain the predicted probability and the corresponding category activation heatmap output by the model. The category activation heatmap is generated by the gradient weighted class activation mapping method. The category activation heatmap is overlaid with the brain region anatomical labels of the test samples, and the overlap between high-response areas in the heatmap and key brain regions of AD is statistically analyzed; wherein, the key brain regions of AD include the hippocampus and the entorhinal cortex. When the overlap is lower than a preset threshold, the classification head of the prediction model is fine-tuned until the model's decision basis matches clinical prior knowledge, thus obtaining an interpretable Alzheimer's disease prediction model.

[0074] By applying the GAN-enhanced AD prediction model data optimization method provided in this invention, and using clinical cognitive assessment data and brain region anatomical labels as conditional constraints input to a multi-scale perception generator, the generated synthetic images not only match the real images in pixel distribution but also conform to the brain region label constraints in anatomical structure and match the clinical scores in pathological features. Compared with traditional unconstrained GANs, the images generated by this invention have higher structural similarity and can accurately reflect brain atrophy patterns at different disease stages. Furthermore, this application generates targeted synthetic images of minority classes by statistically analyzing the distribution of samples of each category in real data, constructing a class-balanced enhancement training set. Further, this application uses a two-stream three-dimensional convolutional neural network to process sMRI and FDG-PET data respectively, and integrates clinical cognitive assessment features to fully utilize the complementarity of multimodal information. The classification model trained on the enhancement training set shows a significant improvement in classification accuracy compared to a model using only real data, and exhibits better robustness to fluctuations in data distribution. Furthermore, by generating a category activation heatmap and overlaying it with brain region anatomical labels, the key brain regions on which the model's decisions are based are visualized. When the overlap between high-response regions and key AD brain regions (hippocampus, entorhinal cortex) is below a preset threshold, the model is fine-tuned to ensure that the model's decision-making aligns with prior clinical knowledge. This makes the model output not only accurate but also interpretable, enhancing clinicians' trust. Furthermore, this application employs multiple imputation to fill in missing scale scores and adds Gaussian noise to complete scores to generate perturbation versions, enabling the generator to adapt to common score fluctuations in clinical assessments and ensuring stable quality of the generated synthetic images under different clinical conditions. Further, this application introduces perceptual consistency constraints and a quality verification mechanism based on a pre-trained medical image feature extraction network to ensure that the synthetic images maintain consistency with real images in high-level semantic features, eliminating outliers with significantly different feature distributions and guaranteeing the overall quality of the enhanced training set. In summary, this invention, through an anatomically guided and pathologically constrained generative adversarial network, effectively solves the problems of scarce and imbalanced AD medical data, generating high-quality, clinically usable synthetic images, significantly improving the performance, robustness, and interpretability of AD prediction models, and demonstrating promising clinical application prospects.

[0075] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0076] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0077] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data optimization method for an AD prediction model enhanced by GAN, characterized in that, The method includes: To acquire real multimodal imaging data and corresponding clinical cognitive assessment data of Alzheimer's patients; Construct a conditional generative adversarial network; the conditional generative adversarial network includes a multi-scale perceptual generator and a pathological feature discriminator; The random noise vector is fused with the clinical cognitive assessment data and the brain region anatomical labels pre-generated based on standard brain atlases. The fused feature vector is then used as input to the multi-scale perception generator to generate initial synthetic image data that is consistent with the distribution of the real multimodal image data, conforms to the brain region anatomical label definition in terms of anatomical structure, and matches the clinical cognitive assessment data in terms of pathological features. The real multimodal image data is used as a positive sample, and the initial synthetic image data is used as a negative sample. Both are input into the pathological feature discriminator for adversarial training to optimize the parameters of the multi-scale perception generator until the pathological feature discriminator can no longer distinguish between the real multimodal image data and the initial synthetic image data. The output of the multi-scale perception generator after parameter optimization is the target synthetic image data. The target synthetic image data is mixed with the real multimodal image data to construct a class-balanced augmented training set, such that the maximum difference between the number of samples in each class does not exceed a preset proportion threshold. The initial neural network classification model is trained using the enhanced training set to obtain an Alzheimer's disease prediction model.

2. The method according to claim 1, characterized in that, The acquisition of real multimodal imaging data and corresponding clinical cognitive assessment data of Alzheimer's patients specifically includes: Structural magnetic resonance imaging data and fluorodeoxyglucose positron emission tomography data from multiple subjects were collected as the real multimodal imaging data; Each subject underwent neuropsychological scale assessment to obtain multidimensional scoring data, including Mini-Mental State Examination scores, total scores of the Clinical Dementia Rating Scale, and scores of the Alzheimer's Disease Rating Scale-Cognitive Subscale, as the clinical cognitive assessment data. The structural magnetic resonance imaging data and fluorodeoxyglucose positron emission tomography data were sequentially spatially registered to the standard Montreal Neurological Institute space, skew field correction, grayscale normalization, and cranial dissection preprocessing. The preprocessed multimodal image data and the multidimensional scoring data are associated and stored according to the subject identifiers to form a paired dataset.

3. The method according to claim 1, characterized in that, The construction of a conditional generative adversarial network includes: A multi-scale perception generator is constructed, which adopts an encoder-decoder architecture. The encoder part includes multiple parallel residual blocks with different porosity to extract multi-scale features of different receptive fields. The decoder part fuses the features of the corresponding layer of the encoder through skip connections, and introduces an attention gating module based on brain region anatomical labels at the skip connections to enhance the expression of key brain region features. A pathological feature discriminator is constructed, which is a fully convolutional network. Its output layer includes a first branch and a second branch in parallel. The first branch outputs the probability that the input image is real or synthetic through a global average pooling layer and a fully connected layer. The second branch outputs the classification probability of the disease stage to which the input image belongs through a global average pooling layer and a fully connected layer. The disease stages include normal, mild cognitive impairment and Alzheimer's disease. The conditional generative adversarial network is constructed by jointly optimizing the multi-scale perception generator and the pathological feature discriminator through a weighted sum of adversarial loss function and auxiliary classification loss function.

4. The method according to claim 1, characterized in that, The process of fusing random noise vectors with clinical cognitive assessment data and brain region anatomical labels, and inputting this data into a multi-scale perception generator to generate initial synthetic image data, specifically includes: The random noise vector is copied and expanded into a noise feature map with the same spatial size as the brain region anatomical label; The clinical cognitive assessment data is converted into a one-dimensional conditional vector output by a fully connected layer, and the one-dimensional conditional vector is copied and expanded into a clinical conditional feature map with the same spatial size as the brain region anatomical labels. The noise feature map, the clinical condition feature map, and the brain region anatomical labels are concatenated along the channel dimension to obtain a multi-channel fusion tensor. The multi-channel fusion tensor is input into the multi-scale sensing generator; The multi-scale perception generator encodes and decodes the multi-channel fused tensor using a three-dimensional convolutional kernel. During the encoding process, multi-scale features are extracted through convolutional layers with different dilation rates. During the decoding process, spatial details are restored through skip connections. Finally, the generator outputs initial synthetic image data with the same size and modality as the real multimodal image data.

5. The method according to claim 1, characterized in that, The step of inputting real multimodal image data and initial synthetic image data into the pathological feature discriminator for adversarial training to optimize the generator parameters specifically includes: A joint loss function is constructed, wherein the joint loss function includes a weighted sum of adversarial loss and pathological classification loss; the adversarial loss uses Wasserstein distance to calculate the score difference between the pathological feature discriminator's output of real images and synthetic images; the pathological classification loss uses the cross-entropy loss function to calculate the error between the disease stage predicted by the pathological feature discriminator for the input image and the real disease stage labeled by the corresponding clinical cognitive assessment data. In each iteration of training, the generator parameters are first fixed, and the pathological feature discriminator parameters are updated with the goal of minimizing the joint loss of the pathological feature discriminator. Then, the pathological feature discriminator parameters are fixed, and the generator parameters are updated with the goal of minimizing the joint loss of the generator. The joint loss of the generator is the negative value of the pathological feature discriminator's score on the synthetic image plus a pathological classification loss term. The joint loss of the generator also includes a perceptual loss term, which is calculated based on the distance between the feature maps of the initial synthetic image data and the real multimodal image data at at least one network layer by the pre-trained medical image feature extraction network. Repeat the above iterative process until the average score difference between the pathological feature discriminator and the synthetic image is less than a preset threshold, and the pathological classification accuracy of the pathological feature discriminator for the synthetic image is lower than a preset random level threshold.

6. The method according to claim 1, characterized in that, The process of mixing synthetic target image data with real multimodal image data to construct a class-balanced augmented training set specifically includes: The distribution of sample numbers for different disease categories in the real multimodal image data is statistically analyzed, and categories with sample numbers below a preset equilibrium threshold are identified as minority categories. Based on the preset category balancing strategy, calculate the category and quantity of target synthetic image data required to supplement the sample data of each category to achieve the desired balance ratio; The multi-scale perception generator, after training convergence, is invoked to generate a corresponding number of target synthetic image data based on the clinical cognitive assessment data and brain region anatomical labels corresponding to the minority class. The quality consistency of the generated target synthetic image data is checked, and outlier samples that differ significantly from the feature distribution of similar real image data are removed. The verified target synthetic image data is merged with the real multimodal image data, and the merged dataset is randomly rearranged to construct an enhanced training set in which the distribution of the number of samples in each category meets the preset balance requirements.

7. The method according to claim 1, characterized in that, The step of training the initial neural network classification model using an enhanced training set to obtain an Alzheimer's disease prediction model specifically includes: A multimodal fusion classification network is constructed as the initial neural network classification model. The multimodal fusion classification network includes an image feature extraction branch and a clinical feature fusion layer. The image feature extraction branch adopts a two-stream three-dimensional convolutional neural network structure. The first stream input is the structural magnetic resonance imaging data in the enhanced training set, and the second stream input is the fluorodeoxyglucose positron emission tomography data in the enhanced training set. Structural features and metabolic features are extracted respectively, and then fused in the deep layer by feature splicing or addition to obtain a multimodal image feature vector. The clinical feature fusion layer takes the clinical cognitive assessment data corresponding to the enhanced training set as input, maps it to a clinical feature vector through a fully connected network, and concatenates the clinical feature vector with the multimodal image feature vector to obtain a joint feature representation; The joint feature representation is input into a classification head consisting of multiple fully connected layers, and finally outputs the probability that the input sample belongs to normal, mild cognitive impairment, or Alzheimer's disease through a softmax layer. The multimodal fusion classification network is trained in an end-to-end manner using the cross-entropy loss function to minimize the error between the predicted probability and the true label, and the model parameters after training convergence are saved as the final Alzheimer's disease prediction model.

8. The method according to claim 1, characterized in that, Before fusing the random noise vector with clinical cognitive assessment data and brain region anatomical labels, the process also includes: For subjects with missing scores on the scales, multiple imputation was used to fill in the missing values ​​based on the score distribution of the complete sample, resulting in a complete clinical cognitive assessment data vector. Gaussian noise was added to the complete clinical cognitive assessment data vector to generate multiple perturbation versions of the clinical condition vector, thereby increasing the generator's robustness to slight fluctuations in clinical scores during training.

9. The method according to claim 1, characterized in that, The adversarial training process also includes: The initial synthetic image data and the corresponding real multimodal image data are respectively input into a pre-trained medical image feature extraction network to obtain their feature maps in multiple network layers. Calculate the L1 distance between the initial synthetic image data and the real multimodal image data in each selected layer feature map, and use it as the perceptual loss term; The perceptual loss term is added to the joint loss function of the generator and weighted and summed with the adversarial loss and pathological classification loss to constrain the generated image to maintain consistency with the real image in high-level semantic features.

10. The method according to claim 1, characterized in that, After obtaining the Alzheimer's disease prediction model, the method further includes: The test sample is input into the Alzheimer's disease prediction model to obtain the predicted probability and the corresponding category activation heatmap output by the model. The category activation heatmap is generated by the gradient weighted class activation mapping method. The category activation heatmap is overlaid with the brain region anatomical labels of the test samples, and the overlap between high-response areas in the heatmap and key brain regions of AD is statistically analyzed; wherein, the key brain regions of AD include the hippocampus and the entorhinal cortex. When the overlap is lower than a preset threshold, the classification head of the prediction model is fine-tuned until the model's decision basis matches clinical prior knowledge, thus obtaining an interpretable Alzheimer's disease prediction model.

Images