A dual-encoder contrast method, device and program product based on whole brain voxel level

Abstract

The application relates to the field of intelligent medical treatment, in particular to a double-encoder comparison method, equipment and program product based on whole-brain voxel levels. The method comprises the following steps: acquiring a three-dimensional whole-brain image dataset; constructing a double-encoder comparison decoding model by using the three-dimensional whole-brain image dataset; the construction process of the double-encoder comparison decoding model comprises the following steps: grouping normal or diseased three-dimensional whole-brain images, selecting an arbitrary normal three-dimensional whole-brain image as a reference image, inputting the reference image and other normal or diseased three-dimensional whole-brain images into a to-be-trained encoder model in parallel to obtain output features, comparing the output features to obtain a whole-brain difference classification result; repeating the steps of updating the to-be-trained encoder model and comparison calculation according to the reference image and other normal or diseased three-dimensional whole-brain images until a preset stop condition is reached, and obtaining a double-encoder comparison decoding model. The application has good clinical application value.

Description

Technical Field

[0001] This application relates to the field of intelligent healthcare, specifically to a method, device, program product, and computer-readable storage medium for comparing dual encoders based on the whole-brain voxel level. Background Technology

[0002] The neuropsychiatric symptoms (NPS) spectrum of Alzheimer's disease (AD) is a common and highly heterogeneous group of clinical symptoms, including at least 12 clinical symptoms (phenotypes), such as hallucinations, delusions, agitation / aggression, depression, anxiety, euphoria, apathy, disinhibition, irritability, abnormal motor behavior, sleep disturbances / nocturnal behavior abnormalities, and appetite / eating behavior disorders. NPS persists throughout the entire AD course; over 90% of AD patients exhibit at least one NPS. Individuals often display different behavioral symptoms and genetic variations, frequently presenting different combinations of symptoms (phenotypes), significantly limiting the effectiveness of drug treatment and the development of new drugs, severely impacting patients' quality of life, and imposing a significant burden on society and families. Currently, NPS diagnosis heavily relies on clinical scales, such as the Neuropsychiatric Questionnaire (NPI, NPI-Q, BEHAVE-AD, etc.). Assessment results are significantly influenced by caregiver emotions, culture, and familiarity with the patient, resulting in significant individual variability and unstable outcomes.

[0003] In recent years, the integration of medicine and engineering based on multimodal neuroimaging has innovatively applied engineering principles and technologies (such as machine learning algorithms, complex network analysis, and high-performance computing) to the medical field. This has been more conducive to revealing the neural activity patterns of synergistic interactions between brain regions, and has promoted breakthroughs in the diagnosis and treatment of complex neuropsychiatric diseases and technological development. Understanding the heterogeneity of neuroanatomy and clarifying NPS diagnosis and targeted behavioral interventions may be key to improving patients' quality of life. However, there are currently no definitively established neuroanatomical variations associated with different NPS phenotypes. Moreover, current NPS imaging research evidence is limited and mostly concentrated in the traditional research paradigm of single-scale or pre-defined regions of interest. It is influenced by factors such as researchers' subjective thinking, patient age, and disease stage, making it impossible to accurately determine the extent to which individual differences in NPS are due to anatomical abnormalities. Summary of the Invention

[0004] To address the above problems, this invention provides a dual encoder comparison method based on the whole-brain voxel level, specifically including:

[0005] Obtain a 3D whole-brain imaging dataset;

[0006] A dual encoder contrast decoding model was constructed using the aforementioned three-dimensional whole-brain image dataset. This dual encoder contrast decoding model was used to compare whole-brain image data for difference recognition and classification.

[0007] The construction process of the dual encoder contrast decoding model is as follows: the three-dimensional whole-brain images are grouped into normal or diseased groups, and any normal three-dimensional whole-brain image is selected as the reference image. The reference image is input into the encoder model to be trained in parallel with other normal or diseased three-dimensional whole-brain images to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the encoder model to be trained and the comparison calculation steps are repeated until the preset stopping condition is reached to obtain the dual encoder contrast decoding model.

[0008] When the reference image is input to the encoder to be trained in parallel with other normal or diseased 3D whole-brain images, a second comparison is also included. After the reference image is compared with other normal 3D whole-brain images, the second comparison calculation is performed and the reference image is input to the encoder to be trained for the other normal 3D whole-brain images to obtain a first output feature. After the reference image is compared with diseased 3D whole-brain images, the reference image is input to the encoder to be trained for the diseased 3D whole-brain images to obtain a second output feature. The reference image is input to the encoder to be trained for the reference image to obtain a third output feature. The first output feature and the third output feature are compared and calculated, and the second output feature and the third output feature are compared and calculated to obtain the whole-brain differential classification result.

[0009] Optionally, the reference image and the diseased three-dimensional whole-brain image are compared in a second way to obtain similar features and difference features. The similar features and difference features are then input into the encoder to be trained of the diseased three-dimensional whole-brain image. The encoder to be trained of the diseased three-dimensional whole-brain image encodes the similar features and difference features in parallel to obtain similar coded features and difference coded features. The similar coded features and difference coded features are then fused and decoded to obtain the third output feature.

[0010] The encoder is any one or more of the following: variational autoencoder, vector quantization variational autoencoder, adversarial autoencoder, sparse autoencoder, conditional variational autoencoder, Beta-VAE;

[0011] Optionally, the encoder is a variational autoencoder. The reference image and other normal or diseased three-dimensional whole-brain images are input in parallel into the variational autoencoder model to be trained to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the variational autoencoder model to be trained and the comparison calculation steps are repeated until a preset stopping condition is reached to obtain a dual encoder comparison decoding model.

[0012] Optionally, the encoder includes an encoding layer and a decoding layer. The reference image and other normal or diseased three-dimensional whole-brain images are encoded by feature compression in their respective encoders through the encoding layer, and then reconstructed by the decoding layer to obtain output features. The encoding layer includes a three-dimensional convolution module. The feature is compressed to the latent space through the three-dimensional convolution module to calculate the feature probability distribution parameters and obtain compressed features. The compressed features are then sampled and reconstructed by the three-dimensional convolution module of the decoding layer to obtain output features.

[0013] Optionally, the dual encoder contrast decoding model further includes morphological analysis, which calculates local volume change features by reconstructing the output features from the decoding layer and the input image of the encoder, fuses the output features and the local volume change features to obtain fused features, and compares and calculates the fused features to obtain whole-brain differential classification results.

[0014] Alternatively, the reference image is compared with other normal 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the other normal 3D whole-brain images to obtain a first output feature. The first output feature is then compared with the input image of the encoder to obtain a first local volume change feature. The first output feature and the first local volume change feature are fused to obtain a first fusion feature. The reference image is compared with diseased 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the diseased 3D whole-brain images to obtain a second output feature. The second output feature is then compared with the input image of the encoder to obtain a second local volume change feature. The second output feature and the second local volume change feature are fused to obtain a second fusion feature. The reference image is input into the encoder to be trained on the reference image to obtain a third output feature. The first fusion feature and the third output feature are compared and calculated, and the second fusion feature and the third output feature are compared and calculated to obtain a whole-brain differential classification result.

[0015] The comparison calculation performs feature classification by calculating the distance between features, and obtains the whole brain difference classification result;

[0016] Optionally, the reference image is input into the encoder to be trained to obtain the reference global output features, and the other normal or diseased three-dimensional whole brain images are input into the encoder to be trained to obtain the normal or diseased global output features. The reference global output features and the normal or diseased global output features are compared and calculated to obtain the whole brain differential classification results.

[0017] Optionally, in the second comparison, the reference image and other normal 3D whole-brain images are first sampled to obtain a reference patch image and other normal 3D whole-brain patch images. Any one of the reference patch images is selected as the second reference image. Then, the second reference image is compared with other reference patch images or other normal 3D whole-brain patch images in a second comparison calculation, and then input into the encoder to be trained of the other normal 3D whole-brain images to obtain the first output feature. The reference image and diseased 3D whole-brain images are first sampled to obtain a reference patch image and diseased 3D whole-brain patch images. Any one of the reference patch images is selected as the third reference image. Then, the third reference image is compared with other reference patch images or diseased 3D whole-brain patch images in a second comparison calculation, and then input into the encoder to be trained of the diseased 3D whole-brain images to obtain the second output feature. The reference image is input into the encoder to be trained to obtain the third output feature. The first output feature and the third output feature are compared and calculated, and the second output feature and the third output feature are compared and calculated to obtain the whole-brain differential classification result.

[0018] Optionally, the second comparison calculation involves calculating the image feature distance between the second reference image and other reference patch images or other normal three-dimensional whole-brain patch images to obtain a first internal classification result, and inputting the first internal classification into the encoder to be trained of other normal three-dimensional whole-brain images to obtain a first output feature; or the third reference image involves calculating the image feature distance between other reference patch images or diseased three-dimensional whole-brain patch images to obtain a second internal classification result, and inputting the second internal classification into the encoder to be trained of diseased three-dimensional whole-brain images to obtain a second output feature.

[0019] The purpose of this invention is to provide a brain feature classification method based on a whole-brain voxel dual encoder, comprising:

[0020] Acquire whole-brain imaging data of the test subject;

[0021] The whole-brain imaging data is input into a dual-encoder contrast decoding model to obtain the classification results of normal or abnormal brain anatomical features. The dual-encoder contrast decoding model is based on the above-mentioned dual-encoder contrast method based on the whole-brain voxel level.

[0022] The purpose of this invention is to provide a method for constructing a specific lesion model for identifying the neuropsychiatric symptoms of Alzheimer's disease, comprising:

[0023] Obtain datasets of whole-brain images from patients with the AD disease spectrum and normal whole-brain images;

[0024] The dataset of whole-brain images of patients with the AD disease spectrum and normal whole-brain images is input into a dual-encoder comparison decoding model to extract whole-brain features and classify them for training, thereby obtaining a model for identifying specific lesions of AD neuropsychiatric symptoms; the dual-encoder comparison decoding model is obtained based on the above-mentioned dual-encoder comparison method based on the whole-brain voxel level.

[0025] Optionally, the AD disease spectrum includes one or more of twelve neuropsychiatric symptom phenotypes. The whole brain images of patients with the AD disease spectrum and normal whole brain images are input into a dual encoder comparison decoding model to extract one or more of the normal features of the whole brain and the specific lesion features of the whole brain and classify them for training, so as to obtain a first model for recognizing specific lesions of AD neuropsychiatric symptoms.

[0026] Optionally, the whole-brain images of patients with the AD disease spectrum include phenotype images with neuropsychiatric symptoms and phenotype images without neuropsychiatric symptoms. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual encoder comparison decoding model to extract normal features of the whole brain and specific lesion features of the whole brain and classify them. During the training process, the three-dimensional images of the diseased whole brain are alternately input into the phenotype images with neuropsychiatric symptoms and the phenotype images without neuropsychiatric symptoms for training, to obtain a second model for recognizing specific lesions of AD neuropsychiatric symptoms.

[0027] Optionally, during the training process of images with neuropsychiatric symptoms, similar coding features and differential coding features are updated, while during the training process of images without neuropsychiatric symptoms, similar coding features are updated.

[0028] Optionally, the whole-brain images of patients with the AD disease spectrum include images with phenotypes accompanied by neuropsychiatric symptoms and images without phenotypes accompanied by neuropsychiatric symptoms. The images with phenotypes accompanied by neuropsychiatric symptoms include one or more of twelve neuropsychiatric phenotypes. The whole-brain images of patients with the AD disease spectrum and the normal whole-brain images dataset are input into a dual encoder comparison decoding model to extract normal features of the whole brain and specific lesion features of the whole brain and classify them for training, thereby obtaining a third model for recognizing specific lesions of AD neuropsychiatric symptoms.

[0029] Optionally, the construction of the model for identifying specific lesions of AD neuropsychiatric symptoms further includes a first specific feature, acquiring general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological markers of patients and normal individuals in the AD disease spectrum, extracting features from the general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data to obtain the first specific feature, comparing the first specific feature with the whole-brain features extracted from the dual encoder decoding model for similarity analysis to obtain the specific feature, and classifying based on the specific feature and / or whole-brain features to obtain the model for identifying specific lesions of AD neuropsychiatric symptoms.

[0030] The purpose of this invention is to provide a classification method based on identifying specific lesion models of AD neuropsychiatric symptoms, comprising:

[0031] Acquire whole-brain imaging data of the test subject;

[0032] The whole-brain imaging data is input into the AD neuropsychiatric symptom-specific lesion identification model for classification to obtain the classification results of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion identification model is obtained by the above-described method for constructing the AD neuropsychiatric symptom-specific lesion identification model;

[0033] Optionally, the whole-brain imaging data is input into a first AD neuropsychiatric symptom-specific lesion model for classification to obtain one or more classification results of one or more of the twelve neuropsychiatric symptom phenotypes of the AD disease spectrum, either normal or suffering from AD; the first AD neuropsychiatric symptom-specific lesion model is obtained through the above-described method for constructing the AD neuropsychiatric symptom-specific lesion model.

[0034] Optionally, the whole-brain imaging data is input into a second AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having AD disease spectrum with neuropsychiatric symptom phenotype or having AD disease spectrum without neuropsychiatric symptom phenotype; the second AD neuropsychiatric symptom-specific lesion model is obtained by the above-described method for constructing the AD neuropsychiatric symptom-specific lesion model.

[0035] Optionally, the whole-brain imaging data is input into a third AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having AD disease spectrum with neuropsychiatric symptom phenotype or having AD disease spectrum without neuropsychiatric symptom phenotype; the third AD neuropsychiatric symptom-specific lesion model is obtained by the above-described method for constructing an AD neuropsychiatric symptom-specific lesion model.

[0036] Optionally, the acquisition further includes acquiring general information of the subject, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data; extracting features from the general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data to obtain a first specific feature; inputting the whole brain imaging data and the first specific feature into the AD neuropsychiatric symptom-specific lesion model for classification to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion model is obtained through the above-described method for constructing the AD neuropsychiatric symptom-specific lesion model.

[0037] The purpose of this invention is to provide a computer program product, which includes a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the above-mentioned dual encoder comparison method based on the whole brain voxel level, or to implement the above-mentioned brain feature classification method based on the whole brain voxel dual encoder, or to implement the above-mentioned method for constructing a model for identifying specific lesions of AD neuropsychiatric symptoms, or to implement the above-mentioned classification method based on identifying specific lesions of AD neuropsychiatric symptoms.

[0038] The purpose of this invention is to provide a computer device comprising a memory, a processor, and a computer program or instructions stored in the memory. The computer program or instructions are executed by the processor to implement the above-described dual encoder comparison method based on whole-brain voxel level, or to implement the above-described brain feature classification method based on whole-brain voxel dual encoder, or to implement the above-described method for constructing a model for identifying specific lesions of AD neuropsychiatric symptoms, or to implement the above-described classification method based on identifying specific lesions of AD neuropsychiatric symptoms.

[0039] The purpose of this invention is to provide a computer-readable storage medium storing a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the above-described dual encoder comparison method based on whole-brain voxel level, or to implement the above-described brain feature classification method based on whole-brain voxel dual encoder, or to implement the above-described method for constructing a model for identifying specific lesions of AD neuropsychiatric symptoms, or to implement the above-described classification method based on identifying specific lesions of AD neuropsychiatric symptoms.

[0040] Advantages of this invention:

[0041] 1. This invention proposes a dual encoder contrast decoding framework, which extracts specific anatomical features at the whole brain voxel level and designs multiple rounds of contrast learning to perform global feature comparison and local feature comparison on the normal group and the diseased group respectively. In the local feature comparison, patch images are generated by image subsampling. After contrast learning on the patch images, the encoder is used to extract features, so that the model can improve the recognition of specific change features in the anatomical images of the patient group.

[0042] 2. This invention proposes morphological calculations for three-dimensional brain anatomical images. In the reconstruction of three-dimensional images, there are problems of three-dimensional volume deformation or distortion, which can easily cause errors in the recognition of three-dimensional regions. Therefore, this invention performs morphological calculations on the brain feature map generated after feature compression and reconstruction through an encoder and the real brain image to reduce the error between the reconstructed features and the real image, reduce model illusion, and thus improve the reliability and usability of the model.

[0043] 3. This invention utilizes a dual-encoder contrast decoding framework to accurately separate AD patients with different NPS phenotype-specific anatomical features from other shared features, extracts highly sensitive NPS brain imaging representations, and constructs an NPS-specific brain injury network. This network can effectively distinguish different phenotypes of mental and behavioral abnormalities, forming a tool to quantify the specific categories of NPS in AD patients, further contributing to the targeted nature of clinical treatment.

[0044] 4. This invention targets AD patients with NPS neuropsychiatric symptom phenotypes, including hallucinations, delusions, agitation / aggression, depression, anxiety, euphoria, apathy, disinhibition, irritability, abnormal motor behavior, sleep disorders / nocturnal behavior abnormalities, and appetite / eating behavior disorders. This invention utilizes a dual-encoder contrastive decoding model for feature extraction, while simultaneously employing multimodal data feature fusion using general information, neuropsychological assessment data, and cerebrospinal fluid or plasma pathological marker data. Compared to traditional imaging assessment methods, this avoids the influence of multiple factors such as physician subjective thinking, patient age, and disease stage. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the process of the dual encoder comparison method based on the whole brain voxel level provided in the embodiments of the present invention;

[0047] Figure 2 This is a schematic diagram of a dual encoder comparison system based on the whole-brain voxel level provided in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of a dual encoder comparison device based on the whole brain voxel level provided in an embodiment of the present invention;

[0049] Figure 4 Network structure diagram of the dual encoder contrast decoding model provided in the embodiment of the present invention.

[0050] Figure 5 This is a flowchart of the three-dimensional whole-brain patch image processing for diseased patients provided in an embodiment of the present invention;

[0051] Figure 6 A flowchart of the morphological calculation structure provided in an embodiment of the present invention;

[0052] Figure 7 This is a diagram of the encoder network structure provided in an embodiment of the present invention;

[0053] Figure 8 This is a network structure diagram of a diseased test subject in a dual encoder provided in an embodiment of the present invention;

[0054] Figure 9 The brain regions related to female AD patients provided in the embodiments of the present invention;

[0055] Figure 10 The brain regions related to male AD patients provided in the embodiments of the present invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0057] In some of the processes described in the specification, claims, and accompanying drawings of this invention, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as S101, S102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0058] Figure 1 The schematic diagram of the dual encoder comparison method based on the whole-brain voxel level provided in this embodiment of the invention specifically includes:

[0059] S1: Obtain a 3D whole-brain image dataset;

[0060] In one embodiment, the three-dimensional whole-brain imaging includes one or more of the following: MRI, CT, and X-ray;

[0061] Preferably, the three-dimensional whole-brain image is an MRI, which includes anatomical information of the human brain.

[0062] In one embodiment, the method further includes data preprocessing, wherein the three-dimensional whole-brain image includes structural feature images and perfusion time feature images, the structural feature images and perfusion time feature images are spatially registered and then brain normalized to obtain a processed three-dimensional whole-brain image dataset, and a dual encoder contrast decoding model is constructed using the processed three-dimensional whole-brain image dataset.

[0063] S2: Construct a dual encoder contrast decoding model using the three-dimensional whole-brain image dataset. The dual encoder contrast decoding model is used to compare whole-brain image data for difference recognition and classification.

[0064] The construction process of the dual encoder contrast decoding model is as follows: the three-dimensional whole-brain images are grouped into normal or diseased groups, and any normal three-dimensional whole-brain image is selected as the reference image. The reference image is input into the encoder model to be trained in parallel with other normal or diseased three-dimensional whole-brain images to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the encoder model to be trained and the comparison calculation steps are repeated until the preset stopping condition is reached to obtain the dual encoder contrast decoding model.

[0065] In one embodiment, when the reference image is input to the encoder to be trained in parallel with other normal or diseased 3D whole-brain images, a second comparison is further included. After the reference image is compared with other normal 3D whole-brain images, it is input to the encoder to be trained for the other normal 3D whole-brain images to obtain a first output feature. After the reference image is compared with diseased 3D whole-brain images, it is input to the encoder to be trained for the diseased 3D whole-brain images to obtain a second output feature. The reference image is input to the encoder to be trained for the reference image to obtain a third output feature. The first output feature and the third output feature are compared and calculated, and the second output feature and the third output feature are compared and calculated to obtain a whole-brain differential classification result.

[0066] In one embodiment, the reference image and the diseased three-dimensional whole-brain image are compared to obtain similar features and difference features. The similar features and difference features are then input into the encoder to be trained on the diseased three-dimensional whole-brain image. The encoder to be trained on the diseased three-dimensional whole-brain image encodes the similar features and difference features in parallel to obtain similar coded features and difference coded features. The similar coded features and difference coded features are then fused and decoded to obtain the third output feature.

[0067] In one embodiment, the similar and different features are encoded in the encoder through parallel shared coding layers and specific coding layers to obtain similar coded features and different coded features.

[0068] In one embodiment, the encoder employs any one or more of the following: variational autoencoder, vector quantization variational autoencoder, adversarial autoencoder, sparse autoencoder, conditional variational autoencoder, and Beta-VAE.

[0069] In one embodiment, the encoder is a variational autoencoder. The reference image is input in parallel with other normal or diseased three-dimensional whole-brain images into the variational autoencoder model to be trained to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the variational autoencoder model to be trained and the comparison calculation steps are repeated until a preset stopping condition is reached to obtain a dual encoder comparison decoding model.

[0070] In one embodiment, the encoder includes an encoding layer and a decoding layer. The reference image and other normal or diseased three-dimensional whole-brain images are encoded by feature compression in their respective encoders through the encoding layer, and then reconstructed by the decoding layer to obtain output features. The encoding layer includes a three-dimensional convolution module. The feature is compressed to the latent space through the three-dimensional convolution module to calculate the feature probability distribution parameters and obtain compressed features. The compressed features are then sampled and reconstructed by the three-dimensional convolution module of the decoding layer to obtain output features.

[0071] In one embodiment, the dual encoder contrastive decoding model further includes morphological analysis, which calculates local volume change features by reconstructing the output features from the decoding layer and the input image of the encoder, fuses the output features and the local volume change features to obtain fused features, and compares and calculates the fused features to obtain whole-brain differential classification results.

[0072] Alternatively, the reference image is compared with other normal 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the other normal 3D whole-brain images to obtain a first output feature. The first output feature is then compared with the input image of the encoder to obtain a first local volume change feature. The first output feature and the first local volume change feature are fused to obtain a first fusion feature. The reference image is compared with diseased 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the diseased 3D whole-brain images to obtain a second output feature. The second output feature is then compared with the input image of the encoder to obtain a second local volume change feature. The second output feature and the second local volume change feature are fused to obtain a second fusion feature. The reference image is input into the encoder to be trained on the reference image to obtain a third output feature. The first fusion feature and the third output feature are compared and calculated, and the second fusion feature and the third output feature are compared and calculated to obtain a whole-brain differential classification result.

[0073] In one embodiment, the output features, or the first output features, the second output features, and the third output features, are feature maps, further being whole-brain feature maps (or whole-brain patch feature maps). Morphological calculation is performed by calculating local physical examination change features through local volume changes using a normal three-dimensional whole-brain image or a diseased three-dimensional whole-brain image (or a normal three-dimensional whole-brain patch image or a diseased three-dimensional whole-brain patch image) input by the encoder and the whole-brain feature map reconstructed by the encoder.

[0074] In one embodiment, the comparison calculation performs feature classification by calculating the distance between features to obtain a whole-brain differential classification result.

[0075] In one embodiment, the reference image is input to the encoder to be trained to obtain reference global output features, and other normal or diseased three-dimensional whole-brain images are input to the encoder to be trained to obtain normal or diseased global output features. The reference global output features and the normal or diseased global output features are compared and calculated to obtain the whole-brain differential classification result.

[0076] In one embodiment, the second comparison involves first obtaining a baseline patch image and other normal 3D whole-brain images through image subsampling, selecting any one of the baseline patch images as the second baseline image, and then performing a second comparison calculation on the second baseline image and other baseline patch images or other normal 3D whole-brain patch images before inputting it into the encoder to be trained for the other normal 3D whole-brain images to obtain the first output feature; the second comparison involves first obtaining a baseline patch image and other 3D whole-brain images of the diseased brain through image subsampling, selecting any one of the baseline patch images as the third baseline image, and then performing a second comparison calculation on the third baseline image and other baseline patch images or 3D whole-brain patch images of the diseased brain before inputting it into the encoder to be trained for the diseased 3D whole-brain images to obtain the second output feature; the second comparison involves inputting the baseline image into the encoder to be trained for the baseline image to obtain the third output feature, and then performing a comparison calculation on the first output feature and the third output feature, and a comparison calculation on the second output feature and the third output feature to obtain the whole-brain differential classification result.

[0077] In one embodiment, the second comparison calculation involves calculating the image feature distance between the second reference image and other reference patch images or other normal three-dimensional whole-brain patch images to obtain a first internal classification result, and inputting the first internal classification into the encoder to be trained of other normal three-dimensional whole-brain images to obtain a first output feature; or the third reference image involves calculating the image feature distance between other reference patch images or diseased three-dimensional whole-brain patch images to obtain a second internal classification result, and inputting the second internal classification into the encoder to be trained of diseased three-dimensional whole-brain images to obtain a second output feature.

[0078] In one specific embodiment, the structure of the dual encoder contrast decoding model is as follows: Figure 4 As shown, feature learning is performed on normal and diseased 3D whole-brain images through contrastive learning. The encoder's encoder-decoder structure is used for feature compression and reconstruction, and 3D convolution is used for feature extraction to identify differences in brain anatomy and neurological lesions. This also includes a second contrast calculation, where other normal and diseased 3D whole-brain images are compared with a baseline image before being input into the encoder to be trained. Optionally, in the dual-encoder contrastive decoding model, image subsampling is performed on other normal and diseased 3D whole-brain images compared with the baseline image to obtain corresponding patch images. The second contrast calculation is then performed on these patch images before being input into the encoder to be trained. Taking diseased 3D whole-brain images as an example, its structure is shown in Figure 5.

[0079] Optionally, the structure of the dual-encoder contrast decoding model also includes morphological calculations, such as the morphological calculation structures of the diseased three-dimensional whole-brain image and other normal three-dimensional whole-brain images. Figure 6 As shown.

[0080] In one specific embodiment, the dual-encoder contrastive decoding model uses a variational autoencoder for encoding. Within the variational autoencoder, the decoder and encoder reconstruct and compress features using a 3D convolutional module to obtain a feature map, which has the following characteristics: Figure 7 As shown in Figure A, the upper part represents the reconstruction process of the brain in healthy individuals, while the lower part represents the reconstruction process of the brain in diseased individuals. The diseased brain simultaneously activates both the C (common) and S (special) encoders, while the healthy brain only activates the C encoder, thus forming a contrastive learning process. Figure B describes the model architecture of the VAE's encoder-decoder.

[0081] In one embodiment, the method further includes representation similarity calculation, wherein the output representation of the encoder layer of the diseased three-dimensional whole brain image to be trained is input into the RSA model, any two output representations are selected for dissimilarity calculation or similarity calculation to obtain a representation similarity matrix, and the consistency of any two representation similarity matrices is compared to obtain a representation similarity result.

[0082] Optionally, the output representation includes the output representation of any layer in the coding layer and / or the voxel features and / or similar coding features and / or differential coding features of the ROI region of the diseased three-dimensional whole brain image;

[0083] Optionally, the output representation includes similarity coding features and / or difference coding features.

[0084] Optionally, the representation similarity result represents the relationship between the compared representation similarity matrices.

[0085] This invention provides a brain feature classification method based on a whole-brain voxel dual encoder, comprising:

[0086] Acquire whole-brain imaging data of the test subject;

[0087] The whole-brain imaging data is input into a dual-encoder contrast decoding model to obtain the classification results of normal or abnormal brain anatomical features. The dual-encoder contrast decoding model is based on the above-mentioned dual-encoder contrast method based on the whole-brain voxel level.

[0088] In one embodiment, the method is used for diseases in which changes occur in brain anatomy on imaging, wherein the brain anatomy is a whole-brain anatomical region or a region of particular focus on a whole-brain anatomical image, such as a lesion region.

[0089] This invention provides a method for constructing a specific lesion model for identifying neuropsychiatric symptoms of Alzheimer's disease, comprising:

[0090] Obtain datasets of whole-brain images from patients with the AD disease spectrum and normal whole-brain images;

[0091] The dataset of whole-brain images of patients with the AD disease spectrum and normal whole-brain images is input into the dual encoder comparison decoding model to extract whole-brain features and classify them for training, thereby obtaining a model for identifying specific lesions of AD neuropsychiatric symptoms; the dual encoder comparison decoding model is obtained based on the above-mentioned dual encoder comparison method based on the whole-brain voxel level.

[0092] In one embodiment, the AD disease spectrum includes one or more of twelve neuropsychiatric symptom phenotypes. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual encoder comparison decoding model to extract one or more of the normal features of the whole brain and the specific lesion features of the whole brain and classify them for training, thereby obtaining a first model for recognizing specific lesions of AD neuropsychiatric symptoms.

[0093] In one embodiment, the whole-brain images of patients with the AD disease spectrum include phenotype images with and without neuropsychiatric symptoms. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual-encoder comparison decoding model to extract normal features and specific lesion features of the whole brain and classify them. During the training process, the three-dimensional images of the diseased whole brain are alternately input into the phenotype images with and without neuropsychiatric symptoms for training, thereby obtaining a second model for identifying specific lesions of AD neuropsychiatric symptoms.

[0094] In one embodiment, during the training process of images with neuropsychiatric symptoms, similar coding features and differential coding features are updated, while during the training process of images without neuropsychiatric symptoms, similar coding features are updated.

[0095] In one embodiment, the whole-brain images of patients with the AD disease spectrum include images with and without neuropsychiatric symptoms. The images with neuropsychiatric symptoms include one or more of twelve neuropsychiatric symptom phenotypes. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual encoder comparison decoding model to extract normal features and specific lesion features of the whole brain and classify them to obtain a third model for identifying specific lesions of AD neuropsychiatric symptoms.

[0096] In one embodiment, the construction of the model for identifying specific lesions of AD neuropsychiatric symptoms further includes a first specific feature, which involves acquiring general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data of patients with the AD disease spectrum and normal individuals, extracting features from the general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data to obtain the first specific feature, comparing the first specific feature with the whole-brain features extracted from the dual encoder decoding model for similarity analysis to obtain specific features, and classifying based on the specific features and / or whole-brain features to obtain the model for identifying specific lesions of AD neuropsychiatric symptoms. In one embodiment, the method further includes feature extraction, which involves extracting features from the whole-brain images of patients with the AD disease spectrum and normal whole-brain images to obtain one or more of the following features: gray matter volume, white matter volume, and cortical thickness;

[0097] The features and images are input into a dual-encoder comparison decoding model for feature extraction and classification training, resulting in a model for identifying specific lesions of AD neuropsychiatric symptoms. General information includes age, gender, height, and weight.

[0098] In one embodiment, the method further includes data preprocessing, wherein the whole-brain images of patients with AD disease spectrum and normal whole-brain images include structural feature images and perfusion time feature images. After spatial registration of the structural feature images and perfusion time feature images, brain normalization is performed to obtain processed images of patients with AD disease spectrum and processed normal images. The processed images of patients with AD disease spectrum and processed normal images are then input into a dual encoder comparison decoding model.

[0099] In one specific embodiment, 1. Patients with the AD disease spectrum and normal controls are screened, and patients with the AD disease spectrum are divided into AD-NP group (patients with AD disease spectrum and NPS group), AD-nNPS group (patients with AD disease spectrum and no NPS group), and HC (normal control group) based on the presence or absence of NPS.

[0100] 2. Clinical assessments were conducted on the tested individuals, and they were categorized into 12 types of abnormal mental and behavioral phenotypes based on the results of the Neuropsychiatric Symptom Questionnaire (NPI).

[0101] 3. Perform 3D-T1WI structural magnetic resonance imaging on the head of the test subject to obtain structural feature images of gray matter, white matter, ventricles, etc. of the head of the individual. The spatial registration, brain normalization, and brain atlas coverage methods of structural feature images and perfusion time feature images are used. The CAT toolbox is used to extract features such as gray matter volume, white matter volume, and cortical thickness.

[0102] 4. By integrating contrastive learning, 3D convolutional neural networks and variational autoencoders, a dual-encoder contrastive decoding architecture is constructed to analyze NPI-specific brain image representation models.

[0103] To accurately characterize the anatomical specific features of AD-NPS patients, this invention develops a dual-encoder contrastive decoding framework, optimized by integrating contrastive learning, a 3D convolutional neural network (3D CNN), and a variational autoencoder (VAE). Through strategic architectural adjustments, the dual-encoder contrastive decoding framework reduces the number of parameters by 73% while improving computational efficiency and model performance. This framework processes 3D-T1WI structural grayscale images from AD-NPS patients and healthy controls, eliminating the limitations of traditional case-control methods while isolating disease-specific variants. Furthermore, its voxel-based analysis minimizes information masking caused by brain atlas averaging, thereby achieving more accurate variant localization.

[0104] 5. Extract AD-NPS-specific anatomical features based on the whole brain:

[0105] Based on the AD-NPS-specific variant brain anatomical features extracted using a dual encoder contrastive decoding framework, the specific variant brain regions for each site were calculated using Jacobi determinant.

[0106] The encoder projects the data onto two distinct 16-dimensional latent distributions (a distribution of shared features and a distribution of NPS phenotypic specific features). The decoder takes a 32-dimensional vector (obtained by concatenating the shared features and NPS phenotypic specific features) as input and outputs a reconstructed image of brain volume structure. The decoder uses two deconvolutional layers to reconstruct the brain volume structure image from the features of the latent distributions. The reconstructed feature dimension of the control group is 32-dimensional, consisting of the shared features (a 16-dimensional vector) and a concatenated vector of 16 zero elements. Therefore, the NPS phenotypic specific features can be separated from the shared features.

[0107] Coding distribution:

[0108]

[0109] Where x represents the input image (e.g., T1 MRI) and y represents the category / status label (e.g., HC / AD). This represents the potential characteristics of sharing (regardless of category). Indicates specific (category-related) latent characteristics. This represents the posterior distribution of the synthesized brain output by the encoder. , This indicates that the encoder predicts from x. The mean and standard deviation, , Indicates encoder from predict The mean and standard deviation, where I represents the identity matrix. This represents a Gaussian distribution.

[0110] Decoding and Reconstruction:

[0111]

[0112] in, Represents the decoder / generator, with parameters as follows: , This indicates that under the path with specific factors, Generated / reconstructed images (corresponding to disease phenotypes), This indicates that under the path of despecific factors, by The generated / reconstructed image (corresponding to a healthy phenotype) is used here. Control phenotypic variations; setting it to zero is equivalent to generating "removing lesions / removing offsets".

[0113] Training process:

[0114] a. Loss function:

[0115]

[0116] Where λ represents the control feature decoupling strength, forcing shared features. With special features Orthogonal , Indicates the distribution of training data. Indicates using The error in reconstructing the input, This represents the reconstruction error of the health path. Indicates health imaging. Indicates the Kullback-Leibler divergence. Indicates the prior of latent variables, Indicates the weight of KL. , This indicates the correlation between penalty sharing and the special subspace.

[0117] b. Training strategy:

[0118] AD-NP group: Simultaneously update shared encoders and specific encoders;

[0119] nNPS group: Update only the shared encoder (fixed Zd = 0);

[0120] Alternate input of AD-NP and nNPS samples enables comparative learning.

[0121] c. Morphological analysis:

[0122] Calculate the Jacobian determinant of the deformation field of a real brain (where the encoder is the input image) and a synthetic brain (where the encoder reconstructs the feature map):

[0123]

[0124] Where ϕ represents from Differential homeomorphisms to the real brain are used to quantify local volume changes. Then, it means taking the partial derivative of the nth component of the synthetic brain coordinates with respect to the nth component of the real brain, where ϕn represents the spatial coordinates of the synthetic brain and xn represents the spatial coordinates of the "real" brain.

[0125] 6. Perform characterization similarity analysis between anatomical features and demographic, neuropsychological assessments, and cerebrospinal fluid / plasma metabolomics and / or pathological marker data to obtain NPS phenotype-specific anatomical features;

[0126] Shared Feature RDM:

[0127]

[0128] in, No. The line is the first The shared feature vector of _n_ subjects (or conditions), where N represents the number of subjects or conditions, and F_n is the shared feature vector of _n_ subjects (or conditions). s Indicates the shared feature dimension. The transpose of the shared eigenvectors of the subjects or conditions. The correlation matrix is ​​obtained by performing Pearson (or Spearman) correlation on column A of matrix.

[0129] Specific features RDM:

[0130]

[0131] in, No. The line is the first The specific feature vectors of _n_ subjects (or conditions), where N represents the number of subjects or conditions, and F_n represents the number of conditions. s The dimension of the specific feature is represented. The transpose of the specific eigenvectors of the subject or condition is represented by corr(A), which represents the correlation matrix obtained by performing correlation calculations on columns A of matrix A. Correlation calculations include Pearson or Spearman correlation calculations.

[0132] Behavioral RDM:

[0133]

[0134] in, This represents the score vector of the k-th behavior item (e.g., NPI item k) across all subjects, where the i-th element is the subject's score for that item. , Represents the square Euclidean norm (MSE at the pixel / voxel level). , This represents an N×N outer product, equivalent to a "(global normalized) similarity" matrix;

[0135] Analysis related to Kendall's Tau:

[0136]

[0137]

[0138] in, Indicates the index of the system / subject / brain region / model layer being compared. Indicates the number of observations included in the comparison; in the RDM scenario, only the upper triangular matrix is ​​compared. This indicates that in two vectors (such as the system) On the RDM and reference RDM, for all The count obtained by comparing "item by item": consistent logarithm For any two entries The two vectors satisfy: Inconsistent logarithms Then we have: ; This indicates consistency in disease conditions, and is related to " "The formula is consistent, the denominator This represents the total number of all comparable pairs.

[0139] 7. Because deep learning models have a large number of hyperparameters, they are particularly prone to non-standard parameter selection, so external database validation is necessary.

[0140] This invention provides a classification method based on identifying specific lesion models of AD neuropsychiatric symptoms, including:

[0141] Acquire whole-brain imaging data of the test subject;

[0142] The whole-brain imaging data is input into the AD neuropsychiatric symptom-specific lesion identification model for classification to obtain the classification results of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion identification model is obtained by the above-described method for constructing the AD neuropsychiatric symptom-specific lesion identification model.

[0143] In one embodiment, the whole-brain imaging data is input into a first AD neuropsychiatric symptom-specific lesion model for classification to obtain one or more classification results of one or more of the twelve neuropsychiatric symptom phenotypes of the AD disease spectrum, either normal or suffering from AD; the first AD neuropsychiatric symptom-specific lesion model is obtained by the above-described method for constructing an AD neuropsychiatric symptom-specific lesion model.

[0144] In one embodiment, the whole-brain imaging data is input into a second AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having an AD disease spectrum with neuropsychiatric symptom phenotype or having an AD disease spectrum without neuropsychiatric symptom phenotype; the second AD neuropsychiatric symptom-specific lesion model is obtained by the above-described method for constructing an AD neuropsychiatric symptom-specific lesion model.

[0145] In one embodiment, the whole-brain imaging data is input into a third AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having AD disease spectrum with neuropsychiatric symptom phenotype or having AD disease spectrum without neuropsychiatric symptom phenotype; the third AD neuropsychiatric symptom-specific lesion model is obtained by the above-described method for constructing an AD neuropsychiatric symptom-specific lesion model.

[0146] In one embodiment, the acquisition further includes acquiring general information of the subject, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data; extracting features from the general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data to obtain a first specific feature; inputting the whole-brain imaging data and the first specific feature into an AD neuropsychiatric symptom-specific lesion model for classification to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion model is obtained by the above-described method for constructing an AD neuropsychiatric symptom-specific lesion model.

[0147] In one embodiment, preferably, the first identification of AD neuropsychiatric symptom-specific lesion model, the second identification of AD neuropsychiatric symptom-specific lesion model, and the third identification of AD neuropsychiatric symptom-specific lesion model also include a data preprocessing step and / or a feature processing (feature extraction) step and / or a combination of a first specific feature step and / or a characterization feature calculation with the identification of AD neuropsychiatric symptom-specific lesion model.

[0148] In one specific embodiment, the patient's whole-brain images are input into a model for analyzing specific lesions that identify AD neuropsychiatric symptoms, such as... Figure 8 As shown, the process of analyzing disease populations using the RSA model is illustrated. The reconstructed brain features decouple general and specific features. RSA model representational similarity analysis is a general framework for comparing "representational geometry": First, the multidimensional responses of a model (a certain ROI / voxel in the brain, a certain layer of a deep model, behavioral scale, etc.) under a set of conditions are compressed into pairwise dissimilarity matrices, forming representational dissimilarity matrices (RDMs). Then, the consistency of different RDMs is compared to determine whether they encode the same information structure. Based on the encoded information structure, the relationship between different RDMs is obtained.

[0149] This allows for the alignment and comparison of representations from different sources (brain regions, models, behaviors) without relying on specific decoders or feature labels, thereby assessing whether a model / brain region can explain another system or behavioral pattern.

[0150] Therefore, RSA provides a model-independent evidentiary basis for cross-modal representation alignment, which can be used to locate information sources, assess the interpretability of theoretical / deep models, and establish a verifiable link between brain representations and behavioral differences.

[0151] Specifically, whole-brain images of N patients are input into a model for identifying specific lesions of AD neuropsychiatric symptoms to obtain N whole-brain features (whole-brain features include normal whole-brain features and / or whole-brain specific features). The N whole-brain features are then input into an RSA model to calculate the dissimilarity between any two features, resulting in (N(N-1)) / 2 feature representation difference matrices. The relationship between feature representations is obtained by comparing the feature representation difference matrices, where N is a natural number greater than 1. The feature difference matrices include specific feature difference matrices, shared feature difference matrices, and behavioral feature difference matrices.

[0152] Optionally, the input of the RSA model also includes disease-specific features. The disease-specific features are obtained by extracting features from general patient information, neuropsychological assessments, and cerebrospinal fluid / plasma pathological marker data. The disease-specific features and whole-brain features are then input into the RSA model to calculate the dissimilarity between any two features.

[0153] Optionally, the input of the RSA model also includes identifying the output features of any layer in the AD neuropsychiatric symptom-specific lesion model, and inputting the output features of any layer and the whole brain features into the RSA model to calculate the dissimilarity of any two features.

[0154] Optionally, the whole-brain features are the output features of the encoder in a model that identifies specific lesions of AD neuropsychiatric symptoms.

[0155] In one embodiment, whole-brain images of the subject are acquired and input into a model for identifying AD neuropsychiatric symptoms to obtain classification results and the relationship between different feature representations. The relationship between different feature representations, such as the similarity between gender feature representation and weight feature representation and depression feature representation, indicates that gender feature representation can better represent the difference between depression and non-depression.

[0156] In one specific embodiment, by using a dual encoder to compare the decoding model, we found, based on whole-brain voxel 3D-T1 magnetic resonance imaging, that there are significant gender differences in the correlation between the clinical subtype of NPS and brain regions in male and female AD patients. That is, there are significant gender differences in the clinical subtype of NPS in male and female AD patients, which are significantly related to the anatomical variability of related brain regions.

[0157] Result 1:

[0158] In female Alzheimer's disease (AD) patients, the severity of depression was significantly positively correlated with the right superior temporal lobe, right precuneus, left postcentral gyrus, and paracentral lobule (P < 0.001); however, no brain regions significantly associated with depression were found in male AD patients. Relevant brain regions in female AD patients are as follows: Figure 9 As shown in Table 1, the voxel values, peak intensity, and peak MNI coordinates of different brain regions in this patient are shown in Table 1 below.

[0159] Table 1

[0160]

[0161] Result 2:

[0162] In male AD patients, the severity of agitation was significantly negatively correlated with the degree of atrophy in the lower right temporal lobe, upper temporal pole, middle cingulate gyrus, middle left temporal pole, and parahippocampal gyrus (P < 0.001); while in female AD patients, no brain regions were found to be significantly associated with agitation.

[0163] Relevant brain regions in male AD patients, such as Figure 10As shown in Table 2, the voxel values, peak intensity, and peak MNI coordinates of different brain regions in this patient are shown below. The "major brain regions" in the table refer to the significantly different brain regions obtained through the aforementioned model for identifying specific lesions of AD neuropsychiatric symptoms. Each brain region is volumetric and three-dimensional, equivalent to having a coordinate system XYZ; the peak MNI represents its coordinates. The lower right temporal lobe is a large brain region. [The text abruptly ends here, likely due to an incomplete translation or missing information.] Figure 10 The study revealed several (L, where L is a natural number greater than 1) small brain region clusters in the lower right temporal lobe that exhibited differences (all of these small brain region clusters belong to the lower right temporal lobe). The method described in this application more precisely identifies which specific small brain region clusters show greater differences, resulting in more accurate segmentation. This involves specifying the voxel values, peak intensities, and coordinates of the different differentially expressed brain regions, thus clearly identifying which brain region is different and providing greater precision, rather than simply stating which brain region is different.

[0164] Table 2

[0165]

[0166] In one embodiment, this invention proposes a computational method for obtaining NPS phenotypic-specific anatomical features based on a dual-encoder contrastive decoding framework at the whole-brain voxel level. This represents a technological innovation: traditional research paradigms, where NPS imaging evidence is limited and often concentrated at a single scale or a pre-defined region of interest, are influenced by factors such as researchers' subjective thinking, patient age, and disease stage, making it difficult to accurately determine the extent to which gender differences in NPS are due to anatomical abnormalities. This invention combines contrastive learning, three-dimensional convolutional neural networks, and variational autoencoders to propose a dual-encoder contrastive decoding framework. Based on the whole-brain voxel level, it extracts AD-NPS phenotypic-specific anatomical features, accurately separates different NPS phenotypic-specific anatomical features from other shared features, and extracts highly sensitive NPS brain imaging representations to construct NPS-specific brain injury networks. From local to global, and then to multi-voxel-level anatomical abnormality networks, an interpretable multi-scale feature fusion algorithm for NPS phenotype-specific damage patterns is formed. This algorithm is then combined with other multimodal imaging, gene / spatial transcriptomics, targeted metabolomics, and pathological markers from cerebrospinal fluid or blood to reveal NPS phenotype-specific brain network damage and potential metabolic pathways. This provides an innovative and clinically valuable technical means for further understanding the pathophysiological mechanisms of NPS sex differences and exploring the anatomical basis of other complex brain diseases.

[0167] The present invention also discloses a computer program product or system, including a computer program that, when executed by a processor, implements the above-described brain feature classification method based on a whole-brain voxel dual encoder, or implements the above-described method for constructing a model for identifying specific lesions of AD neuropsychiatric symptoms, or implements the above-described classification method steps based on identifying specific lesions of AD neuropsychiatric symptoms.

[0168] Figure 2 The schematic diagram of the dual encoder comparison system based on the whole-brain voxel level provided in this embodiment of the invention specifically includes:

[0169] Acquisition Unit: Acquire a 3D whole-brain image dataset;

[0170] Construction Unit: A dual-encoder contrast decoding model is constructed using the aforementioned 3D whole-brain image dataset. This model is used to compare whole-brain image data for difference identification and classification. The construction process of the dual-encoder contrast decoding model is as follows: The 3D whole-brain images are grouped into normal or diseased groups. Any normal 3D whole-brain image is selected as the reference image. The reference image is input in parallel with other normal or diseased 3D whole-brain images into the encoder model to be trained to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased 3D whole-brain images, the encoder model to be trained and the comparison calculation steps are repeated until a preset stopping condition is reached, thus obtaining the dual-encoder contrast decoding model.

[0171] This invention provides a brain feature classification system based on a whole-brain voxel dual encoder, comprising:

[0172] Acquisition module: Acquires whole-brain imaging data of the test subject;

[0173] Classification module: The whole brain imaging data is input into the dual encoder contrast decoding model to obtain the classification result of normal or abnormal brain anatomical features. The dual encoder contrast decoding model is based on the above-mentioned dual encoder contrast method based on the whole brain voxel level.

[0174] This invention provides a system for constructing a model of specific lesions for recognizing the neuropsychiatric symptoms of Alzheimer's disease, comprising:

[0175] Acquisition Unit: Acquire datasets of whole-brain images from patients with the AD disease spectrum and normal whole-brain images;

[0176] Training Unit: The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into the dual encoder comparison decoding model for training to extract whole-brain features and classify them, thereby obtaining a model for identifying specific lesions of AD neuropsychiatric symptoms; the dual encoder comparison decoding model is obtained based on the above-mentioned dual encoder comparison method based on the whole-brain voxel level.

[0177] This invention provides a classification system based on a model of specific lesions in the neuropsychiatric symptoms of Alzheimer's disease (AD), comprising:

[0178] Acquisition module: Acquires whole-brain imaging data of the test subject;

[0179] Recognition module: The whole brain imaging data is input into the AD neuropsychiatric symptom-specific lesion recognition model for classification to obtain the classification results of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion recognition model is obtained by the above-described method for constructing the AD neuropsychiatric symptom-specific lesion recognition model.

[0180] Figure 3 An embodiment of the present invention provides a schematic diagram of a computer device, specifically including:

[0181] The system includes a memory and a processor; the memory is used to store program instructions; the processor is used to invoke the program instructions, which, when executed, are the above-described dual encoder comparison method based on whole-brain voxel level, or the above-described brain feature classification method based on whole-brain voxel dual encoder, or the above-described method for constructing a model for identifying specific lesions of AD neuropsychiatric symptoms, or the above-described classification method based on identifying specific lesions of AD neuropsychiatric symptoms.

[0182] The present invention also discloses a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described dual encoder comparison method based on the whole-brain voxel level, or implements the above-described brain feature classification method based on the whole-brain voxel dual encoder, or implements the above-described method for constructing a model for identifying specific lesions of AD neuropsychiatric symptoms, or implements the above-described classification method based on identifying specific lesions of AD neuropsychiatric symptoms.

[0183] The verification results of this verification embodiment show that assigning inherent weights to indications can improve the performance of this method compared to the default settings. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated; the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of this embodiment. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0184] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0185] The computer device provided by the present invention has been described in detail above. For those skilled in the art, there will be changes in the specific implementation and application scope based on the ideas of the embodiments of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dual encoder comparison method based on whole-brain voxel level, characterized in that, include: Obtain a 3D whole-brain imaging dataset; A dual encoder contrast decoding model was constructed using the aforementioned three-dimensional whole-brain image dataset. This dual encoder contrast decoding model is used to compare whole-brain image data for difference recognition and classification. The construction process of the dual encoder contrast decoding model is as follows: the three-dimensional whole-brain images are grouped into normal or diseased groups, and any normal three-dimensional whole-brain image is selected as the reference image. The reference image is input into the encoder model to be trained in parallel with other normal or diseased three-dimensional whole-brain images to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the encoder model to be trained and the comparison calculation steps are repeated until the preset stopping condition is reached to obtain the dual encoder contrast decoding model. When the reference image is input to the encoder to be trained in parallel with other normal or diseased 3D whole-brain images, a second comparison is also included. After the reference image is compared with other normal 3D whole-brain images, it is input to the encoder to be trained for the other normal 3D whole-brain images to obtain a first output feature. After the reference image is compared with diseased 3D whole-brain images, it is input to the encoder to be trained for the diseased 3D whole-brain images to obtain a second output feature. The reference image is input to the encoder to be trained for the reference image to obtain a third output feature. The first output feature and the third output feature are compared and calculated, and the second output feature and the third output feature are compared and calculated to obtain the whole-brain differential classification result.

2. The dual encoder comparison method based on whole-brain voxel level according to claim 1, characterized in that, The reference image and the diseased three-dimensional whole-brain image are compared in a second comparison to obtain similar features and difference features. The similar features and difference features are then input into the encoder to be trained on the diseased three-dimensional whole-brain image. The encoder to be trained on the diseased three-dimensional whole-brain image encodes the similar features and difference features in parallel to obtain similar coded features and difference coded features. The similar coded features and difference coded features are then fused and decoded to obtain the second output feature.

3. The dual encoder comparison method based on whole-brain voxel level according to claim 1 or 2, characterized in that, The encoder may be any one or more of the following: variational autoencoder, vector quantization variational autoencoder, adversarial autoencoder, sparse autoencoder, conditional variational autoencoder, or Beta-VAE.

4. The dual encoder comparison method based on whole-brain voxel level according to claim 1, characterized in that, The encoder is a variational autoencoder. The reference image and other normal or diseased three-dimensional whole-brain images are input in parallel into the variational autoencoder model to be trained to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the variational autoencoder model to be trained and the comparison calculation steps are repeated until a preset stopping condition is reached to obtain a dual encoder comparison decoding model.

5. The dual encoder comparison method based on whole-brain voxel level according to claim 1, characterized in that, The encoder includes an encoding layer and a decoding layer. The reference image and other normal or diseased three-dimensional whole-brain images are encoded by feature compression in their respective encoders through the encoding layer, and then the output features are reconstructed through the decoding layer. The encoding layer includes a three-dimensional convolution module. The feature is compressed to the latent space through the three-dimensional convolution module to calculate the feature probability distribution parameters and obtain compressed features. The compressed features are then sampled and reconstructed through the three-dimensional convolution module of the decoding layer to obtain the output features.

6. The dual encoder comparison method based on whole-brain voxel level according to claim 1, characterized in that, The dual encoder contrast decoding model also includes morphological analysis, which calculates local volume change features by reconstructing the output features from the decoding layer and comparing them with the input image of the encoder. The output features and the local volume change features are fused to obtain fused features, and the fused features are compared to obtain whole-brain differential classification results. Alternatively, the reference image is compared with other normal 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the other normal 3D whole-brain images to obtain a first output feature. The first output feature is then compared with the input image of the encoder to obtain a first local volume change feature. The first output feature and the first local volume change feature are fused to obtain a first fusion feature. The reference image is compared with diseased 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the diseased 3D whole-brain images to obtain a second output feature. The second output feature is then compared with the input image of the encoder to obtain a second local volume change feature. The second output feature and the second local volume change feature are fused to obtain a second fusion feature. The reference image is input into the encoder to be trained on the reference image to obtain a third output feature. The first fusion feature and the third output feature are compared and calculated, and the second fusion feature and the third output feature are compared and calculated to obtain a whole-brain differential classification result.

7. The dual encoder comparison method based on whole-brain voxel level according to claim 1, characterized in that, The comparison calculation classifies features by calculating the distance between features, and obtains the whole-brain differential classification result.

8. The dual encoder comparison method based on whole-brain voxel level according to claim 7, characterized in that, The baseline image is input into the encoder to be trained to obtain baseline global output features. Other normal or diseased three-dimensional whole-brain images are input into the encoder to be trained to obtain normal or diseased global output features. The baseline global output features and the normal or diseased global output features are compared and calculated to obtain the whole-brain differential classification results.

9. The dual encoder comparison method based on whole-brain voxel level according to claim 1, characterized in that, In the second comparison, the reference image and other normal 3D whole-brain images are first sampled to obtain a reference patch image and other normal 3D whole-brain patch images. Any one of the reference patch images is selected as the second reference image. Then, the second reference image is compared with other reference patch images or other normal 3D whole-brain patch images in a second comparison calculation, and then input into the encoder to be trained for the other normal 3D whole-brain images to obtain the first output feature. The reference image and diseased 3D whole-brain images are first sampled to obtain a reference patch image and diseased 3D whole-brain patch images. Any one of the reference patch images is selected as the third reference image. Then, the third reference image is compared with other reference patch images or diseased 3D whole-brain patch images in a second comparison calculation, and then input into the encoder to be trained for the diseased 3D whole-brain images to obtain the second output feature. The reference image is input into the encoder to be trained for the reference image to obtain the third output feature. The first output feature and the third output feature are compared and calculated, and the second output feature and the third output feature are compared and calculated to obtain the whole-brain differential classification result.

10. The dual encoder comparison method based on whole-brain voxel level according to claim 9, characterized in that, The second comparison calculation involves calculating the image feature distance between the second reference image and other reference patch images or other normal three-dimensional whole-brain patch images to obtain the first intra-class classification result, and inputting the first intra-class classification into the encoder to be trained on other normal three-dimensional whole-brain images to obtain the first output feature; Alternatively, the third reference image can be used to calculate the image feature distance with other reference patch images or diseased three-dimensional whole-brain patch images to obtain the second internal classification result. The second internal classification is then input into the encoder to be trained on the diseased three-dimensional whole-brain image to obtain the second output feature.

11. A brain feature classification method based on a whole-brain voxel dual encoder, characterized in that, include: Acquire whole-brain imaging data of the test subject; The whole-brain imaging data is input into a dual-encoder contrast decoding model to obtain a classification result of whether the brain anatomical features are normal or abnormal. The dual-encoder contrast decoding model is obtained based on the dual-encoder contrast method based on the whole-brain voxel level as described in any one of claims 1-10.

12. A method for constructing a specific lesion model for identifying AD neuropsychiatric symptoms, characterized in that, include: Obtain datasets of whole-brain images from patients with the AD disease spectrum and normal whole-brain images; The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into the dual encoder comparison decoding model to extract whole-brain features and classify them for training, thereby obtaining a model for identifying specific lesions of AD neuropsychiatric symptoms; the dual encoder comparison decoding model is obtained based on the dual encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-10.

13. The method for constructing a specific lesion model for identifying AD neuropsychiatric symptoms according to claim 12, characterized in that, The AD disease spectrum includes one or more of twelve neuropsychiatric symptom phenotypes. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual encoder comparison decoding model to extract one or more of the normal features of the whole brain and the specific lesion features of the whole brain and classify them for training, so as to obtain the first model for recognizing specific lesions of AD neuropsychiatric symptoms.

14. The method for constructing a specific lesion model for identifying AD neuropsychiatric symptoms according to claim 12, characterized in that, The whole-brain images of patients with the AD disease spectrum include phenotype images with and without neuropsychiatric symptoms. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual encoder comparison decoding model to extract normal features and specific lesion features of the whole brain and classify them. During the training process, the three-dimensional images of the diseased whole brain are alternately input into the images with and without neuropsychiatric symptoms for training, so as to obtain a second model for recognizing specific lesions of AD neuropsychiatric symptoms.

15. The method for constructing a specific lesion model for identifying AD neuropsychiatric symptoms according to claim 14, characterized in that, During the training process of images with neuropsychiatric symptoms, similar coding features and differential coding features are updated; during the training process of images without neuropsychiatric symptoms, similar coding features are updated.

16. The method for constructing a specific lesion model for identifying AD neuropsychiatric symptoms according to claim 12, characterized in that, The whole-brain images of patients with the AD disease spectrum include images with and without neuropsychiatric symptoms. The images with neuropsychiatric symptoms include one or more of twelve neuropsychiatric symptom phenotypes. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual encoder comparison decoding model to extract normal features and specific lesion features of the whole brain and classify them to obtain a third model for identifying specific lesions of AD neuropsychiatric symptoms.

17. The method for constructing a specific lesion model for identifying AD neuropsychiatric symptoms according to claim 12, characterized in that, The construction of the specific lesion model for identifying AD neuropsychiatric symptoms also includes a first specific feature, which involves acquiring general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data of patients and normal individuals in the AD disease spectrum, extracting features from the general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data to obtain a first disease-specific feature, performing similarity analysis between the first specific feature and the whole-brain features extracted by the dual encoder decoding model to obtain a specific feature, and classifying based on the specific feature and / or whole-brain features to obtain the specific lesion model for identifying AD neuropsychiatric symptoms.

18. A classification method based on identifying specific lesion models of AD neuropsychiatric symptoms, characterized in that, include: Acquire whole-brain imaging data of the test subject; The whole-brain imaging data is input into the AD neuropsychiatric symptom-specific lesion model for classification to obtain the classification results of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 12.

19. The classification method based on identifying specific lesion models of AD neuropsychiatric symptoms according to claim 18, characterized in that, The whole-brain imaging data is input into the first AD neuropsychiatric symptom-specific lesion model for classification to obtain one or more of the twelve neuropsychiatric symptom phenotypes of the AD disease spectrum, which are normal or have AD. The first AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 13.

20. The classification method based on identifying specific lesion models of AD neuropsychiatric symptoms according to claim 18, characterized in that, The whole-brain imaging data is input into the second AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having AD disease spectrum with neuropsychiatric symptom phenotype or having AD disease spectrum without neuropsychiatric symptom phenotype; the second AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 14 or 15.

21. The classification method based on identifying specific lesion models of AD neuropsychiatric symptoms according to claim 18, characterized in that, The whole-brain imaging data is input into a third AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having AD disease spectrum with neuropsychiatric symptom phenotype or having AD disease spectrum without neuropsychiatric symptom phenotype; the third AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 16.

22. The classification method based on identifying specific lesion models of AD neuropsychiatric symptoms according to claim 18, characterized in that, The acquisition also includes acquiring general information of the subject, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data; extracting features from the general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data to obtain a first specific feature; inputting the whole brain imaging data and the first specific feature into the AD neuropsychiatric symptom-specific lesion model for classification to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 17.

23. A computer program product comprising a computer program or instructions, characterized in that, The computer program or instructions are executed by the processor to implement the dual encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-10, or to implement the brain feature classification method based on the whole-brain voxel dual encoder as described in claim 11, or to implement the construction method for identifying specific lesion models of AD neuropsychiatric symptoms as described in any one of claims 12-17, or to implement the classification method based on identifying specific lesion models of AD neuropsychiatric symptoms as described in any one of claims 18-22.

24. A computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, characterized in that, The computer program or instructions are executed by the processor to implement the dual encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-10, or to implement the brain feature classification method based on the whole-brain voxel dual encoder as described in claim 11, or to implement the construction method for identifying specific lesion models of AD neuropsychiatric symptoms as described in any one of claims 12-17, or to implement the classification method based on identifying specific lesion models of AD neuropsychiatric symptoms as described in any one of claims 18-22.

25. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instructions are executed by the processor to implement the dual encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-10, or to implement the brain feature classification method based on the whole-brain voxel dual encoder as described in claim 11, or to implement the construction method for identifying specific lesion models of AD neuropsychiatric symptoms as described in any one of claims 12-17, or to implement the classification method based on identifying specific lesion models of AD neuropsychiatric symptoms as described in any one of claims 18-22.

Images