Whole-brain voxel level-based double-encoder comparison method, device and program product

Abstract

The invention relates to the field of intelligent medical treatment, in particular to a double-encoder comparison method and device based on the whole-brain voxel level and a program product. Comprising the following steps: acquiring a three-dimensional whole-brain image data set; the three-dimensional whole-brain image data set is used for constructing a double-encoder comparison decoding model, and the construction process of the double-encoder comparison decoding model comprises the steps that the three-dimensional whole-brain images are subjected to normal or diseased grouping, any normal three-dimensional whole-brain image is selected as a reference image, and the three-dimensional whole-brain images are subjected to normal or diseased grouping; the reference image and other normal or diseased three-dimensional whole-brain images are parallelly output to a to-be-trained encoder model to obtain output features, and the output features are compared and calculated to obtain a whole-brain difference classification result; and according to the reference image and other normal or diseased three-dimensional whole-brain images, the steps of updating the to-be-trained encoder model and comparing and calculating are repeated until a preset stop condition is met, and a double-encoder comparison decoding model is obtained. The application has good clinical application value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent medical treatment, in particular to a double-encoder comparison method based on whole-brain voxel level, equipment, program product and computer readable storage medium. BACKGROUND

[0002] Alzheimer's disease (AD) disease spectrum neuropsychiatric symptoms (NPS) is a group of common and highly heterogeneous clinical symptoms, including hallucinations, delusions, agitation / aggression, depression, anxiety, euphoria, apathy, disinhibition, irritability, abnormal motor behavior, sleep disturbances / nighttime behavior abnormalities, appetite / feeding behavior disorders, and at least 12 clinical symptoms (phenotypes). NPS runs through the whole course of AD, and more than 90% of AD patients can have at least one NPS. Individuals often show different behavioral symptoms and genetic variations, and often show different symptom (phenotype) combinations, which significantly limits the effect of drug treatment and the development of new drugs, and also seriously affects the quality of life of patients, and brings great burden to society and family. At present, the diagnosis of NPS is severely dependent on clinical scales such as neuropsychiatric questionnaire (NPI, NPI-Q, BEHAVE-AD, etc.), and the evaluation results are greatly influenced by the emotions, culture and familiarity of the caregivers, and the individual differences are obvious and the results are unstable.

[0003] In recent years, the medical and engineering cross-fusion based on multi-modal neuroimaging, which innovatively applies engineering principles and technologies (such as machine learning algorithms, complex network analysis and high-performance computing, etc.) to the medical field, is more conducive to revealing the neural activity patterns of the coordination between brain regions, and promoting the breakthrough of complex neuropsychiatric disease diagnosis and treatment and the development of technology. Understanding the heterogeneity of its neuroanatomy and clarifying the diagnosis of NPS and targeted behavioral intervention may be the key to improving the quality of life of patients, but there is currently no determination of the neuroanatomical variations related to different phenotypes of NPS. Moreover, the current evidence of NPS imaging research is limited and mostly concentrated in the traditional research paradigm of a single scale or pre-set region of interest, which is influenced by the subjective thinking of researchers, patient age, disease stage and other factors, and cannot accurately obtain the individualized differences of NPS to a large extent due to anatomical abnormalities. SUMMARY

[0004] In view of the above problems, the present application provides a double-encoder comparison method based on whole-brain voxel level, which specifically comprises:

[0005] Obtaining a three-dimensional whole-brain image data set;

[0006] A dual-encoder contrastive decoding model is constructed using the three-dimensional whole brain image dataset, and the dual-encoder contrastive decoding model is used for difference recognition classification of contrast whole brain image data.

[0007] The construction process of the dual-encoder contrastive decoding model: grouping the three-dimensional whole brain images into normal or diseased groups, selecting an arbitrary normal three-dimensional whole brain image as a reference image, inputting the reference image and other normal or diseased three-dimensional whole brain images into a to-be-trained encoder model in parallel to obtain output features, and performing contrast calculation on the output features to obtain whole brain difference classification results; according to the reference image and other normal or diseased three-dimensional whole brain images, the steps of repeatedly updating the to-be-trained encoder model and contrast calculation are repeated until a preset stopping condition is reached, and a dual-encoder contrastive decoding model is obtained.

[0008] When the reference image and other normal or diseased three-dimensional whole brain images are input into the to-be-trained encoder, a second contrast is further included, the reference image and other normal three-dimensional whole brain images are subjected to second contrast calculation, and then the first output features are obtained by inputting the reference image and other normal three-dimensional whole brain images into the to-be-trained encoder of other normal three-dimensional whole brain images; the reference image and the diseased three-dimensional whole brain image are subjected to second contrast calculation, and then the second output features are obtained by inputting the reference image and the diseased three-dimensional whole brain image into the to-be-trained encoder of the diseased three-dimensional whole brain image; the reference image is input into the to-be-trained encoder of the reference image to obtain third output features; the first output features and the third output features are subjected to contrast calculation, and the second output features and the third output features are subjected to contrast calculation, to obtain whole brain difference classification results.

[0009] Optionally, the reference image and the diseased three-dimensional whole brain image are subjected to second contrast calculation to obtain similar features and difference features, the similar features and the difference features are input into the to-be-trained encoder of the diseased three-dimensional whole brain image, the to-be-trained encoder of the diseased three-dimensional whole brain image encodes the similar features and the difference features in parallel to obtain similar encoding features and difference encoding features, and the similar encoding features and the difference encoding features are fused and then decoded to obtain third output features.

[0010] The encoder adopts any one or several of the following: variational autoencoder, vector quantization variational autoencoder, adversarial autoencoder, sparse autoencoder, conditional variational autoencoder, and Beta-VAE.

[0011] Optionally, the encoder is a variational autoencoder, the reference image and other normal or diseased three-dimensional whole brain images are input into a to-be-trained variational autoencoder model in parallel to obtain output features, and whole brain difference classification results are obtained by performing contrast calculation on the output features; according to the reference image and other normal or diseased three-dimensional whole brain images, the steps of repeatedly updating the to-be-trained variational autoencoder model and contrast calculation are repeated until a preset stopping condition is reached, and a dual-encoder contrastive decoding model is obtained.

[0012] Optionally, the encoder comprises an encoding layer and a decoding layer, the reference image and other normal or diseased three-dimensional whole brain images are subjected to feature compression coding through the encoding layer and feature reconstruction through the decoding layer in the respective encoder to obtain output features; the encoding layer comprises a three-dimensional convolution module, the feature compression is performed through the three-dimensional convolution module to calculate feature probability distribution parameters in a latent space to obtain compressed features, and the compressed features are subjected to sampling reconstruction of the compressed features through the three-dimensional convolution module of the decoding layer to obtain the output features;

[0013] Optionally, the dual-encoder contrast decoding model further comprises morphological analysis, the output features reconstructed by the decoding layer are subjected to morphological calculation with the input images of the encoder to obtain local volume change features, the output features and the local volume change features are fused to obtain fused features, and the fused features are subjected to contrast calculation to obtain whole brain difference classification results;

[0014] Or the reference image and other normal three-dimensional whole brain images are subjected to second contrast calculation and then input into a to-be-trained encoder of the other normal three-dimensional whole brain images to obtain first output features, the first output features are subjected to morphological calculation with the input images of the encoder to obtain first local volume change features, the first output features and the first local volume change features are fused to obtain first fused features; the reference image and the diseased three-dimensional whole brain images are subjected to second contrast calculation and then input into a to-be-trained encoder of the diseased three-dimensional whole brain images to obtain second output features, the second output features are subjected to morphological calculation with the input images of the encoder to obtain second local volume change features, the second output features and the second local volume change features are fused to obtain second fused features; the reference image is input into a to-be-trained encoder of the reference image to obtain third output features, the first fused features and the third output features are subjected to contrast calculation, and the second fused features and the third output features are subjected to contrast calculation to obtain whole brain difference classification results.

[0015] The contrast calculation is performed by calculating the distance between the features to classify the features and obtain the whole brain difference classification results;

[0016] Optionally, the reference image is input into a to-be-trained encoder to obtain a reference global output feature, the other normal or diseased three-dimensional whole brain images are input into a to-be-trained encoder to obtain normal or diseased global output features, and the reference global output feature and the normal or diseased global output features are subjected to contrast calculation to obtain whole brain difference classification results;

[0017] Optionally, the second comparison, the reference image and other normal three-dimensional whole brain images are first subjected to image subsampling to obtain reference patch images and other normal three-dimensional whole brain patch images, any one of the reference patch images is selected as a second reference image, and the second reference image and other reference patch images or other normal three-dimensional whole brain patch images are subjected to second comparison calculation and then input into a to-be-trained encoder of other normal three-dimensional whole brain images to obtain first output features; the reference image and diseased three-dimensional whole brain images are first subjected to image subsampling to obtain reference patch images and diseased three-dimensional whole brain patch images, any one of the reference patch images is selected as a third reference image, and the third reference image and other reference patch images or diseased three-dimensional whole brain patch images are subjected to second comparison calculation and then input into a to-be-trained encoder of the diseased three-dimensional whole brain images to obtain second output features; the reference image is input into a to-be-trained encoder of the reference image to obtain third output features, and the first output features and the third output features are subjected to comparison calculation, and the second output features and the third output features are subjected to comparison calculation to obtain a whole brain difference classification result.

[0018] Optionally, the second comparison calculation is image feature distance calculation of the second reference image and other reference patch images or other normal three-dimensional whole brain patch images to obtain a first internal classification result, and the first internal classification result is input into a to-be-trained encoder of other normal three-dimensional whole brain images to obtain first output features; or the third reference image and other reference patch images or diseased three-dimensional whole brain patch images are subjected to image feature distance calculation to obtain a second internal classification result, and the second internal classification result is input into a to-be-trained encoder of the diseased three-dimensional whole brain images to obtain second output features.

[0019] The present application aims to provide a brain feature classification method based on a whole brain voxel double-encoder, comprising:

[0020] Obtaining whole brain image data of a to-be-tested subject;

[0021] Inputting the whole brain image data into a double-encoder comparison decoding model to obtain a classification result of normal or abnormal brain anatomical features, wherein the double-encoder comparison decoding model is obtained based on the above-mentioned double-encoder comparison method based on whole brain voxel level.

[0022] The present application aims to provide a method for constructing an AD neuropsychiatric symptom-specific lesion model, comprising:

[0023] Obtaining AD disease spectrum patient whole brain image data and normal whole brain image data set;

[0024] The whole brain image of the AD disease spectrum patient is input into the double encoder contrast decoding model to extract whole brain features and classify the training, and an AD neuropsychiatric symptom specific lesion model is obtained.

[0025] Optionally, the AD disease spectrum includes one or several of twelve neuropsychiatric symptom phenotypes, the whole brain image of the AD disease spectrum patient is input into the double encoder contrast decoding model to extract one or several of whole brain normal features and whole brain specific lesion features and classify the training, and a first AD neuropsychiatric symptom specific lesion model is obtained.

[0026] Optionally, the whole brain image of the AD disease spectrum patient includes images with neuropsychiatric symptom phenotypes and images without neuropsychiatric symptom phenotypes, the whole brain image of the AD disease spectrum patient is input into the double encoder contrast decoding model to extract whole brain normal features and whole brain specific lesion features and classify the training, and a second AD neuropsychiatric symptom specific lesion model is obtained.

[0027] Optionally, the similar encoding features and the difference encoding features are updated during the training of the images with neuropsychiatric symptom phenotypes, and the similar encoding features are updated during the training of the images without neuropsychiatric symptom phenotypes.

[0028] Optionally, the whole brain image of the AD disease spectrum patient includes images with neuropsychiatric symptom phenotypes and images without neuropsychiatric symptom phenotypes, the images with neuropsychiatric symptom phenotypes include one or several of twelve neuropsychiatric symptom phenotypes, the whole brain image of the AD disease spectrum patient is input into the double encoder contrast decoding model to extract whole brain normal features and whole brain specific lesion features and classify the training, and a third AD neuropsychiatric symptom specific lesion model is obtained.

[0029] Optionally, the AD neuropsychiatric symptom specific lesion model construction further includes a first specific feature, general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological markers of the AD disease spectrum patient and normal people are obtained, the general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data are extracted to obtain the first specific feature, the first specific feature and the whole brain features extracted from the double encoder contrast decoding model are analyzed for similarity to obtain specific features, and classification is performed based on the specific features and / or whole brain features to obtain the AD neuropsychiatric symptom specific lesion model.

[0030] The present application aims to provide a classification method based on identifying AD neuropsychiatric symptom specific lesion models, comprising:

[0031] acquiring whole brain image data of a subject to be tested;

[0032] feeding the whole brain image data to an AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom specific lesion model is obtained by the above-mentioned method for constructing an AD neuropsychiatric symptom specific lesion model;

[0033] Optionally, the whole brain image data is fed to a first AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or one or more of twelve neuropsychiatric symptom phenotypes of AD disease spectrum; the first AD neuropsychiatric symptom specific lesion model is obtained by the above-mentioned method for constructing an AD neuropsychiatric symptom specific lesion model;

[0034] Optionally, the whole brain image data is fed to a second AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or AD disease spectrum with neuropsychiatric symptom phenotype or AD disease spectrum without neuropsychiatric symptom phenotype; the second AD neuropsychiatric symptom specific lesion model is obtained by the above-mentioned method for constructing an AD neuropsychiatric symptom specific lesion model;

[0035] Optionally, the whole brain image data is fed to a third AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or AD disease spectrum with neuropsychiatric symptom phenotype or AD disease spectrum without neuropsychiatric symptom phenotype; the third AD neuropsychiatric symptom specific lesion model is obtained by the above-mentioned method for constructing an AD neuropsychiatric symptom specific lesion model;

[0036] Optionally, the acquisition further comprises acquiring general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data of the subject to be tested; the general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data are extracted to obtain first specific features, and the whole brain image data and the first specific features are fed to the AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom specific lesion model is obtained by the above-mentioned method for constructing an AD neuropsychiatric symptom specific lesion model.

[0037] The computer program product of the present application comprises a computer program or instructions, which are executed by a processor to implement the double-encoder comparison method based on the whole brain voxel level, or to implement the brain feature classification method based on the whole brain voxel double-encoder, or to implement the construction method of identifying the AD neuropsychiatric symptom-specific lesion model, or to implement the classification method based on the identified AD neuropsychiatric symptom-specific lesion model.

[0038] The computer equipment of the present application comprises a memory, a processor and a computer program or instructions stored on the memory, which are executed by the processor to implement the double-encoder comparison method based on the whole brain voxel level, or to implement the brain feature classification method based on the whole brain voxel double-encoder, or to implement the construction method of identifying the AD neuropsychiatric symptom-specific lesion model, or to implement the classification method based on the identified AD neuropsychiatric symptom-specific lesion model.

[0039] The computer readable storage medium of the present application stores a computer program or instructions, which are executed by a processor to implement the double-encoder comparison method based on the whole brain voxel level, or to implement the brain feature classification method based on the whole brain voxel double-encoder, or to implement the construction method of identifying the AD neuropsychiatric symptom-specific lesion model, or to implement the classification method based on the identified AD neuropsychiatric symptom-specific lesion model.

[0040] Advantages of the present application:

[0041] 1. The present application proposes a double-encoder comparison decoding framework, extracts specific anatomical features based on the whole brain voxel level, and designs multiple rounds of contrast learning to compare the global features and local features of the normal group and the patient group, respectively. In the local feature comparison, patch images are generated by image subsampling, and feature extraction is performed using the encoder after contrast learning on the patch images, so that the model can improve the recognition of specific change features in the patient group anatomical image.

[0042] 2. For three-dimensional brain anatomical images, the present application proposes morphological calculation. In three-dimensional images, there are problems of three-dimensional volume deformation or distortion in the reconstruction of whole brain images or whole brain patch images, which can easily cause three-dimensional region recognition errors. Therefore, the present application performs morphological calculation on the brain feature map generated after feature compression and reconstruction by the encoder and the real brain image, reduces the error between the reconstructed feature and the real image, reduces the model hallucination, and further improves the reliability and usability of the model.

[0043] 3. The present application uses a dual-encoder contrast decoding framework to accurately separate AD patients with NPS from different phenotypic specific anatomical features and other shared features, extracts NPS highly sensitive brain image features, and constructs NPS specific brain damage networks, which can effectively distinguish different phenotypes of NPS, form a tool for quantifying specific categories of AD patients with NPS, and further help the targetedness of clinical treatment.

[0044] 4. The present application is aimed at AD patients with NPS neuropsychiatric symptoms, wherein the neuropsychiatric symptoms include hallucinations, delusions, agitation / aggression, depression, anxiety, euphoria, apathy, disinhibition, irritability, abnormal motor behavior, sleep disorders / night behavior abnormalities, appetite / feeding behavior disorders, and the present application uses general data, neuropsychological assessment data, cerebrospinal fluid or plasma pathological marker data for multi-modal data feature fusion while extracting features using a dual-encoder contrast decoding model, which avoids the influence of doctor's subjective thinking, patient's age, disease stage and other factors compared with traditional image evaluation methods. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 A dual-encoder contrast method flowchart based on whole brain voxel level is provided for the embodiments of the present application;

[0047] Figure 2 A dual-encoder contrast system schematic diagram based on whole brain voxel level is provided for the embodiments of the present application;

[0048] Figure 3 A dual-encoder contrast device schematic diagram based on whole brain voxel level is provided for the embodiments of the present application;

[0049] Figure 4 A dual-encoder contrast decoding model network structure diagram is provided for the embodiments of the present application

[0050] Figure 5 A three-dimensional whole brain patch image processing flowchart for the embodiments of the present application is provided;

[0051] Figure 6 A morphological calculation structure flowchart is provided for the embodiments of the present application;

[0052] Figure 7 An encoder network structure diagram is provided for the embodiments of the present application;

[0053] Figure 8 A network structure diagram of a double-encoder provided for an embodiment of the present application for a patient to be tested;

[0054] Figure 9 A brain region related to a female AD patient provided for an embodiment of the present application;

[0055] Figure 10 A brain region related to a male AD patient provided for an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to enable persons skilled in the art to better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0057] In some of the processes described in this specification and in the accompanying drawings, multiple operations are described in a specific order. However, it should be clear to those skilled in the art that these operations can be performed in a different order or in parallel, and the serial numbers of the operations, such as S101, S102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this paper are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0058] Figure 1 The double-encoder comparison method based on whole brain voxel level provided by the embodiment of the present application specifically includes:

[0059] S1: Obtain a three-dimensional whole brain image data set;

[0060] In one embodiment, the three-dimensional whole brain image includes one or more of the following: MRI, CT, X-ray;

[0061] Preferably, the three-dimensional whole brain image is MRI, and the MRI includes anatomical information of the human brain.

[0062] In one embodiment, the method further includes data preprocessing, the three-dimensional whole brain image includes a structural feature image and a perfusion time feature image, and after spatial registration of the structural feature image and the perfusion time feature image, brain normalization is performed to obtain a processed three-dimensional whole brain image data set, and the double-encoder comparison decoding model is constructed using the processed three-dimensional whole brain image data set.

[0063] S2: constructing a double-encoder contrast decoding model using the three-dimensional whole brain image dataset, the double-encoder contrast decoding model being used for difference identification classification of contrast whole brain image data;

[0064] The construction process of the double-encoder contrast decoding model: grouping the three-dimensional whole brain images into normal or diseased groups, selecting an arbitrary normal three-dimensional whole brain image as a reference image, inputting the reference image and other normal or diseased three-dimensional whole brain images into a to-be-trained encoder model in parallel to obtain output features, and performing contrast calculation on the output features to obtain whole brain difference classification results; repeating the steps of updating the to-be-trained encoder model and contrast calculation according to the reference image and other normal or diseased three-dimensional whole brain images until a preset stopping condition is reached, and obtaining a double-encoder contrast decoding model.

[0065] In one embodiment, when the reference image and other normal or diseased three-dimensional whole brain images are input into the to-be-trained encoder, a second contrast is further included, the reference image and other normal three-dimensional whole brain images are subjected to second contrast calculation, and then the first output features are obtained by inputting the reference image and other normal three-dimensional whole brain images into the to-be-trained encoder of other normal three-dimensional whole brain images; the reference image and diseased three-dimensional whole brain images are subjected to second contrast calculation, and then the second output features are obtained by inputting the reference image and diseased three-dimensional whole brain images into the to-be-trained encoder of diseased three-dimensional whole brain images; the reference image is input into the to-be-trained encoder of the reference image to obtain third output features, the first output features and the third output features are subjected to contrast calculation, and the second output features and the third output features are subjected to contrast calculation, thereby obtaining whole brain difference classification results.

[0066] In one embodiment, the reference image and the diseased three-dimensional whole brain image are subjected to second contrast calculation to obtain similar features and difference features, the similar features and the difference features are input into the to-be-trained encoder of the diseased three-dimensional whole brain image, the to-be-trained encoder of the diseased three-dimensional whole brain image encodes the similar features and the difference features in parallel to obtain similar encoding features and difference encoding features, and the similar encoding features and the difference encoding features are fused and then decoded to obtain third output features.

[0067] In one embodiment, the similar features and the difference features are encoded in the encoder through parallel shared encoding layers and specific encoding layers to obtain similar encoding features and difference encoding features.

[0068] In one embodiment, the encoder adopts any one or several of the following: variational autoencoder, vector quantization variational autoencoder, adversarial autoencoder, sparse autoencoder, conditional variational autoencoder, and Beta-VAE.

[0069] In an embodiment, the encoder is a variational autoencoder, the reference image and other normal or diseased three-dimensional whole brain images are input into the to-be-trained variational autoencoder model in parallel to obtain output features, and the whole brain difference classification result is obtained by comparing the output features; the steps of updating the to-be-trained variational autoencoder model and comparison calculation are repeated until a preset stopping condition is reached, and a double-encoder comparison decoding model is obtained.

[0070] In an embodiment, the encoder includes an encoding layer and a decoding layer, the reference image and other normal or diseased three-dimensional whole brain images are compressed and encoded by the encoding layer in the respective encoder to obtain output features; the encoding layer includes a three-dimensional convolution module, the feature is compressed to a latent space by the three-dimensional convolution module to calculate the feature probability distribution parameter, obtain the compressed feature, and the compressed feature is reconstructed by sampling the compressed feature through the three-dimensional convolution module of the decoding layer to obtain the output feature.

[0071] In an embodiment, the double-encoder comparison decoding model further includes morphological analysis, the output feature reconstructed by the decoding layer is morphologically calculated with the input image of the encoder to obtain a local volume change feature, the output feature and the local volume change feature are fused to obtain a fusion feature, and the whole brain difference classification result is obtained by comparing the fusion feature.

[0072] Or the reference image and other normal three-dimensional whole brain images are input into the to-be-trained encoder of the other normal three-dimensional whole brain image to obtain a first output feature after the second comparison calculation, the first output feature is morphologically calculated with the input image of the encoder to obtain a first local volume change feature, the first output feature and the first local volume change feature are fused to obtain a first fusion feature; the reference image and the diseased three-dimensional whole brain image are input into the to-be-trained encoder of the diseased three-dimensional whole brain image to obtain a second output feature after the second comparison calculation, the second output feature is morphologically calculated with the input image of the encoder to obtain a second local volume change feature, the second output feature and the second local volume change feature are fused to obtain a second fusion feature; the reference image is input into the to-be-trained encoder of the reference image to obtain a third output feature, the first fusion feature and the third output feature are compared, and the second fusion feature and the third output feature are compared to obtain the whole brain difference classification result.

[0073] In an embodiment, the output feature or the first output feature, the second output feature, the third output feature is a feature map, further a whole brain feature map (or a whole brain patch feature map). The shape calculation is a local volume change calculation between the normal three-dimensional whole brain image or the diseased three-dimensional whole brain image (or the normal three-dimensional whole brain patch image or the diseased three-dimensional whole brain patch image) input by the encoder and the reconstructed whole brain feature map generated by the encoder.

[0074] In an embodiment, the contrast calculation is a feature classification by calculating the distance between the features to obtain a whole brain difference classification result.

[0075] In an embodiment, the reference image is input into the to-be-trained encoder to obtain a reference global output feature, and the other normal or diseased three-dimensional whole brain image is input into the to-be-trained encoder to obtain a normal or diseased global output feature, and the reference global output feature and the normal or diseased global output feature are compared to obtain a whole brain difference classification result.

[0076] In an embodiment, the second contrast, the reference image and the other normal three-dimensional whole brain image are first subjected to image subsampling to obtain a reference patch image and the other normal three-dimensional whole brain patch image, and any one of the reference patch images is selected as a second reference image, and then the second reference image and the other reference patch images or the other normal three-dimensional whole brain patch images are subjected to a second contrast calculation and input into the to-be-trained encoder of the other normal three-dimensional whole brain image to obtain a first output feature; the reference image and the diseased three-dimensional whole brain image are first subjected to image subsampling to obtain a reference patch image and a diseased three-dimensional whole brain patch image, and any one of the reference patch images is selected as a third reference image, and then the third reference image and the other reference patch images or the diseased three-dimensional whole brain patch images are subjected to a second contrast calculation and input into the to-be-trained encoder of the diseased three-dimensional whole brain image to obtain a second output feature; the reference image is input into the to-be-trained encoder of the reference image to obtain a third output feature, and the first output feature and the third output feature are compared, and the second output feature and the third output feature are compared to obtain a whole brain difference classification result.

[0077] In an embodiment, the second contrast calculation is a feature distance calculation between the second reference image and the other reference patch images or the other normal three-dimensional whole brain patch images to obtain a first internal classification result, and the first internal classification result is input into the to-be-trained encoder of the other normal three-dimensional whole brain image to obtain a first output feature; or a feature distance calculation between the third reference image and the other reference patch images or the diseased three-dimensional whole brain patch images to obtain a second internal classification result, and the second internal classification result is input into the to-be-trained encoder of the diseased three-dimensional whole brain image to obtain a second output feature.

[0078] In one specific embodiment, the structure of the dual-encoder contrast decoding model is as shown in Figure 4 The normal three-dimensional whole brain image and the diseased three-dimensional whole brain image are learned through contrast learning, the encoding-decoding structure of the encoder is used for feature compression and reconstruction, and three-dimensional convolution is used for feature extraction to complete the recognition of the differences in brain anatomy neuropathies. The second contrast calculation is also included, and the second contrast calculation is performed on the other normal three-dimensional whole brain image and the diseased three-dimensional whole brain image and the reference image, and then the result is input into the encoder to be trained. Optionally, in the structure of the dual-encoder contrast decoding model, the other normal three-dimensional whole brain image and the diseased three-dimensional whole brain image and the reference image are sub-sampled to obtain corresponding patch images, and the second contrast calculation is performed based on the patch images, and then the result is input into the encoder to be trained. Taking the diseased three-dimensional whole brain image as an example, its structure is as shown in FIG. 5.

[0079] Optionally, the structure of the dual-encoder contrast decoding model also includes morphological calculation, and the morphological calculation structure of the diseased three-dimensional whole brain image and the other normal three-dimensional whole brain image is as shown in Figure 6

[0080] In one specific embodiment, the structure of the dual-encoder contrast decoding model is as shown in Figure 7 The upper part of A figure represents the reconstruction process of the brain of the healthy population, and the lower part is the reconstruction process of the diseased brain. The diseased brain synchronously activates the C (common) and S (special) two encoders, while the brain of the healthy population only activates the C encoder, thereby forming a contrast learning process. B figure describes the model architecture of the encoder-decoder of VAE.

[0081] In one embodiment, the method further includes representation similarity calculation, and the output representation of the encoding layer of the diseased three-dimensional whole brain image to be trained is input into the RSA model to select any two output representations for dissimilarity calculation or similarity calculation to obtain a representation similarity matrix, and the consistency of any two representation similarity matrices is compared to obtain a representation similarity result.

[0082] Optionally, the output representation includes the output representation of any one layer in the encoding layer and / or the ROI region voxel feature of the diseased three-dimensional whole brain image and / or the similar encoding feature and / or the difference encoding feature.

[0083] Optionally, the output representation includes the similar encoding feature and / or the difference encoding feature.

[0084] Optionally, the representation similarity result represents the relationship between the compared representation similarity matrices.

[0085] ​The embodiment of the present application provides a brain feature classification method based on a whole brain voxel double-encoder, comprising:

[0086] Obtaining whole brain image data of a to-be-tested person;

[0087] Inputting the whole brain image data into a double-encoder contrast decoding model to obtain a classification result of brain anatomical features being normal or abnormal, wherein the double-encoder contrast decoding model is obtained based on the double-encoder contrast method based on the whole brain voxel level.

[0088] In one embodiment, the method is used for a disease in which a brain anatomical site has a change on an image, wherein the brain anatomical site is a whole brain anatomical region or a region of interest on a whole brain anatomical image, such as a lesion region.

[0089] The embodiment of the present application provides a construction method of an AD neuropsychiatric symptom-specific lesion model, comprising:

[0090] Obtaining whole brain image data of AD disease spectrum patients and normal whole brain image data sets;

[0091] Inputting the whole brain image data of the AD disease spectrum patients and the normal whole brain image data sets into a double-encoder contrast decoding model to extract whole brain features and perform classification training, so as to obtain the AD neuropsychiatric symptom-specific lesion model; wherein the double-encoder contrast decoding model is obtained based on the double-encoder contrast method based on the whole brain voxel level.

[0092] In one embodiment, the AD disease spectrum includes one or more of twelve neuropsychiatric symptom phenotypes, the whole brain image data of the AD disease spectrum patients and the normal whole brain image data sets are input into the double-encoder contrast decoding model to extract one or more of whole brain normal features and whole brain specific lesion features and perform classification training, so as to obtain a first AD neuropsychiatric symptom-specific lesion model.

[0093] In one embodiment, the whole brain image data of the AD disease spectrum patients include images with neuropsychiatric symptom phenotypes and images without neuropsychiatric symptom phenotypes, the whole brain image data of the AD disease spectrum patients and the normal whole brain image data sets are input into the double-encoder contrast decoding model to extract whole brain normal features and whole brain specific lesion features and perform classification training, in the training process, the whole brain three-dimensional images of the patients are alternately input into the images with neuropsychiatric symptom phenotypes and the images without neuropsychiatric symptom phenotypes to perform training, so as to obtain a second AD neuropsychiatric symptom-specific lesion model.

[0094] In one embodiment, the similar encoding features and the difference encoding features are updated in the training process of the images with neuropsychiatric symptom phenotypes, and the similar encoding features are updated in the training process of the images without neuropsychiatric symptom phenotypes.

[0095] In one embodiment, the AD disease spectrum patient whole brain image includes an image with neuropsychiatric symptom phenotype, an image without neuropsychiatric symptom phenotype, the image with neuropsychiatric symptom phenotype includes one or several of twelve neuropsychiatric symptom phenotypes, and the AD disease spectrum patient whole brain image and normal whole brain image data set are input into a dual encoder contrast decoding model to extract whole brain normal features, whole brain specific lesion features, and classification training, to obtain a third recognition AD neuropsychiatric symptom specific lesion model.

[0096] In one embodiment, the recognition AD neuropsychiatric symptom specific lesion model construction further includes a first specific feature, obtaining general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data of AD disease spectrum patients and normal people, extracting features from the general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data to obtain the first specific feature, performing similarity analysis on the first specific feature and the whole brain features extracted from the dual encoder contrast decoding model to obtain specific features, and classifying based on specific features and / or whole brain features to obtain the recognition AD neuropsychiatric symptom specific lesion model. In one embodiment, the method further includes feature extraction, and the feature extraction is performed on the AD disease spectrum patient whole brain image and the normal whole brain image data to obtain one or several of the following features: brain gray matter volume, white matter volume, and cortical thickness.

[0097] The features and images are input into a dual encoder contrast decoding model for feature extraction and classification training to obtain a recognition AD neuropsychiatric symptom specific lesion model. The general information includes age, gender, height, and weight.

[0098] In one embodiment, the method further includes data preprocessing, and the AD disease spectrum patient whole brain image and the normal whole brain image data include structural feature images and perfusion time feature images. After spatial registration and brain normalization, the structural feature images and the perfusion time feature images are processed to obtain processed AD disease spectrum patient images and processed normal images, and the processed AD disease spectrum patient images and the processed normal images are input into a dual encoder contrast decoding model.

[0099] In one specific embodiment, 1. Screening AD disease spectrum patients and normal controls, and dividing the AD disease spectrum patients into an AD-NP group (AD disease spectrum patient with NPS group) and an AD-nNPS (AD disease spectrum patient without NPS group) according to the presence or absence of NPS, and an HC (normal control group);

[0100] 2. Clinical evaluation of the tested individuals, and classification into 12 kinds of mental and behavioral abnormality phenotypes according to the results of the neuropsychiatric symptom questionnaire (NPI);

[0101] 3. 3D-T1WI structural magnetic resonance imaging is performed on the head of the tested individual to obtain structural characteristic images of the gray matter, white matter, ventricle and the like of the head of the individual, and a method for spatial registration of the structural characteristic images and the perfusion time characteristic images, brain normalization, and covering of a brain atlas, and a CAT toolbox extracts brain gray matter volume, white matter volume, cortical thickness and the like characteristics;

[0102] 4. Integrating contrast learning, three-dimensional convolutional neural network and variational autoencoder, constructing a double-encoder contrast decoding architecture to analyze NPI-specific brain image representation model.

[0103] In order to accurately depict the anatomical specific characteristic representation of AD-NPS patients, the present application develops a double-encoder contrast decoding framework, which is optimized by integrating contrast learning, three-dimensional convolutional neural network (3D CNN) and variational autoencoder (VAE). Through strategic adjustment on the architecture, the double-encoder contrast decoding framework realizes a reduction of 73% in the number of parameters, while improving the computational efficiency and model performance. The framework processes 3D-T1WI structural gray matter images from AD-NPS patients and healthy control groups, which can eliminate the limitations of traditional case-control methods while isolating disease-specific variations. In addition, its voxel-based analysis minimizes the information masking caused by brain atlas averaging, thereby achieving more accurate variation positioning.

[0104] 5. Extracting AD-NPS-specific anatomical features based on the whole brain:

[0105] Based on the AD-NPS-specific anatomical features extracted by the double-encoder contrast decoding framework, the Jacobian determinant is used to calculate the specific variation brain regions based on each site.

[0106] The encoder projects the data onto two different 16-dimensional latent distributions (distribution of shared features and distribution of NPS phenotype-specific features), respectively. The decoder takes a 32-dimensional vector (obtained by concatenating shared features and NPS phenotype-specific features) as input and produces a reconstructed brain volume structure image as output. The decoder uses two deconvolution layers to reconstruct the brain volume structure image from the latent distribution features. After reconstruction, the feature dimension of the control subject is 32-dimensional, which is a vector concatenated by shared features (a 16-dimensional vector) and a 16-element zero vector. Therefore, NPS phenotype-specific features can be separated from shared features.

[0107] Encoding distribution:

[0108]

[0109] Where x represents the input image (such as T1 MRI), y represents the category / state label (such as HC / AD, etc.), represents shared (independent of class) latent features, represents specific (class-dependent) latent features, represents posterior distribution of synthetic brain encoding the encoder output, , represents mean and standard deviation of the prediction from the encoder from x, , represents mean and standard deviation of the prediction from the encoder from x, I denotes the identity matrix, represents a Gaussian distribution.

[0110] Decoding reconstruction:

[0111]

[0112] where, represents the decoder / generator, parameters are , represents the image (corresponding to the diseased phenotype) generated / reconstructed by in the path with specific factors, represents the image (corresponding to the healthy phenotype) generated / reconstructed by in the path without specific factors, here the phenotype-specific changes are controlled by ; zeroing is equivalent to "de-lesion / de-bias" generation.

[0113] Training procedure:

[0114] a. Loss function:

[0115]

[0116] where λ represents the control feature decoupling strength, forcing shared features to be orthogonal to specific features , , represents the training data distribution, represents the error of reconstructing the input with , represents the reconstruction error of the healthy path, represents the healthy image, represents the Kullback-Leibler divergence, represents the latent variable prior, represents the weight of KL, , represents the penalty for the correlation of shared and specific subspaces.

[0117] b. Training strategy:

[0118] AD-NP group: Simultaneously update shared encoders and specific encoders;

[0119] nNPS group: Update only the shared encoder (fixed Zd = 0);

[0120] Alternate input of AD-NP and nNPS samples enables comparative learning.

[0121] c. Morphological analysis:

[0122] Calculate the Jacobian determinant of the deformation field of a real brain (where the encoder is the input image) and a synthetic brain (where the encoder reconstructs the feature map):

[0123]

[0124] Where ϕ represents from Differential homeomorphisms to the real brain are used to quantify local volume changes. Then, it means taking the partial derivative of the nth component of the synthetic brain coordinates with respect to the nth component of the real brain, where ϕn represents the spatial coordinates of the synthetic brain and xn represents the spatial coordinates of the "real" brain.

[0125] 6. Perform characterization similarity analysis between anatomical features and demographic, neuropsychological assessments, and cerebrospinal fluid / plasma metabolomics and / or pathological marker data to obtain NPS phenotype-specific anatomical features;

[0126] Shared Feature RDM:

[0127]

[0128] in, No. The line is the first The shared feature vector of _n_ subjects (or conditions), where N represents the number of subjects or conditions, and F_n is the shared feature vector of _n_ subjects (or conditions). s Indicates the shared feature dimension. The transpose of the shared eigenvectors of the subjects or conditions. The correlation matrix is ​​obtained by performing Pearson (or Spearman) correlation on column A of matrix.

[0129] Specific features RDM:

[0130]

[0131] in, No. The line is the first The specific feature vectors of _n_ subjects (or conditions), where N represents the number of subjects or conditions, and F_n represents the number of conditions. s The dimension of the specific feature is represented. The transpose matrix of the specific feature vector of the subject or condition, corr(A) represents the correlation matrix calculated by correlating the columns of matrix A, and the correlation calculation includes Pearson or Spearman correlation calculation.

[0132] Behavioral RDM:

[0133]

[0134] where, represents the score vector of the kth behavioral item (such as NPI item k) on all subjects, and the ith element is the score of the subject on the item , represents the square Euclidean norm (pixel / voxel level MSE), , represents the N*N outer product, which is equivalent to a " (global normalization) similarity" matrix;

[0135] Kendall's Tau correlation analysis:

[0136]

[0137]

[0138] where, represents the index of the system / subject / brain region / model layer being compared, represents the number of observation items participating in the comparison, and in the RDM scenario, only the upper triangular matrix is compared; represents the count obtained by comparing "item-item" in two vectors (such as the RDM of the system and the reference RDM) on all pairs of "item-item" comparison: consistent count , for any two items , two vectors satisfy: , inconsistent count then: ; represents the consistency in the disease condition, consistent with the formula , and the denominator represents the total number of all comparable pairs.

[0139] 7. Since the deep learning model has a large number of hyperparameters, it is particularly prone to non-standard parameter selection, and external database verification is performed.

[0140] The embodiment of the application provides a classification method based on identification of AD neuropsychiatric symptom-specific lesion models, comprising:

[0141] Obtaining whole brain image data of a to-be-tested person;

[0142] The whole brain image data is input into the AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom specific lesion model is obtained by the method for constructing the AD neuropsychiatric symptom specific lesion model.

[0143] In one embodiment, the whole brain image data is input into the first AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or one or more of twelve neuropsychiatric symptom phenotypes of the AD disease spectrum; the first AD neuropsychiatric symptom specific lesion model is obtained by the method for constructing the AD neuropsychiatric symptom specific lesion model.

[0144] In one embodiment, the whole brain image data is input into the second AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or the AD disease spectrum with neuropsychiatric symptom phenotype or the AD disease spectrum without neuropsychiatric symptom phenotype; the second AD neuropsychiatric symptom specific lesion model is obtained by the method for constructing the AD neuropsychiatric symptom specific lesion model.

[0145] In one embodiment, the whole brain image data is input into the third AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or the AD disease spectrum with neuropsychiatric symptom phenotype or the AD disease spectrum without neuropsychiatric symptom phenotype; the third AD neuropsychiatric symptom specific lesion model is obtained by the method for constructing the AD neuropsychiatric symptom specific lesion model.

[0146] In one embodiment, the obtaining further comprises obtaining general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data of the subject; performing feature extraction on the general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data to obtain first specific features; inputting the whole brain image data and the first specific features into the AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or the AD disease spectrum; the AD neuropsychiatric symptom specific lesion model is obtained by the method for constructing the AD neuropsychiatric symptom specific lesion model.

[0147] In one embodiment, preferably, the first AD neuropsychiatric symptom specific lesion model, the second AD neuropsychiatric symptom specific lesion model and the third AD neuropsychiatric symptom specific lesion model also comprise a data preprocessing step and / or a feature processing (feature extraction) step and / or a combination of first specific features step and / or a characteristic feature calculation of the AD neuropsychiatric symptom specific lesion model.

[0148] In one embodiment, the whole brain images of the patients are input into the AD neuropsychiatric symptom-specific lesion model for analysis, as shown in Figure 8 The process of analyzing the disease population using the RSA model is shown in FIG. 2, where the reconstructed brain features decouple general features and specific features; where RSA model represents representational similarity analysis, which is a general framework for comparing “representational geometry”: the multi-dimensional responses of a model (a ROI / volume of interest in the brain, a layer of a deep model, a behavioral scale, etc.) under a set of conditions are compressed into a dissimilarity matrix, which is then compared to determine whether different RDMs encode the same information structure, and the relationship between different RDMs is determined based on the encoded information structure.

[0149] This allows the alignment and comparison of representations from different sources (brain regions, models, behaviors) without relying on specific decoders or feature labels, thereby evaluating whether a model / brain region can explain another system or behavioral pattern.

[0150] Therefore, RSA provides a model-independent evidence base for cross-modal representation alignment, which can be used to locate information sources, evaluate the explainability of theories / deep models, and establish a verifiable link between brain representations and behavioral differences.

[0151] Specifically, the whole brain images of N patients are input into the AD neuropsychiatric symptom-specific lesion model to obtain N whole brain features (including whole brain normal features and / or whole brain specific features), and the N whole brain features are input into the RSA model for dissimilarity calculation of any two features, to obtain (N(N-1)) / 2 feature representation dissimilarity matrices, and the relationship between the feature representations is determined by comparing the feature representation dissimilarity matrices, where N is a natural number greater than 1; where the feature dissimilarity matrix includes a specific feature dissimilarity matrix, a shared feature dissimilarity matrix, and a behavioral feature dissimilarity matrix.

[0152] Optionally, the input of the RSA model further includes disease-specific features, and the disease-specific features are obtained by extracting features from patient general information, neuropsychological assessment, and cerebrospinal fluid / plasma pathological marker data, and the disease-specific features and the whole brain features are input into the RSA model for dissimilarity calculation of any two features.

[0153] Optionally, the input of the RSA model further comprises output features of any layer in the AD neuropsychiatric symptom-specific lesion model, and the output features of any layer and the whole brain features are input into the RSA model to calculate the dissimilarity of any two features.

[0154] Optionally, the whole brain features are output features of the encoder in the AD neuropsychiatric symptom-specific lesion model.

[0155] In one embodiment, the whole brain image of the subject to be tested is obtained, and the whole brain image is input into the AD neuropsychiatric symptom-specific lesion model to obtain the relationship between the classification results and different feature representations; the relationship between different feature representations, such as the gender feature representation and the body weight feature representation, is compared with the depression feature representation, and the gender feature representation is more similar than the depression feature representation, which indicates that the gender feature representation can better represent the difference between depression and non-depression.

[0156] In one specific embodiment, through the double-encoder contrast decoding model, we find that the correlation between the clinical subtypes of NPS and brain regions in male and female AD patients has obvious gender differences based on the 3D-T1 magnetic resonance imaging of whole brain voxels, that is, the gender differences in the clinical subtypes of NPS in male and female AD patients are obviously related to the obvious anatomical variability of the related brain regions.

[0157] Result 1:

[0158] In female AD patients, the severity of depression is positively correlated with the right superior temporal lobe, right precuneus, left posterior central gyrus, and paracentral lobule (P <0.001); while in male AD patients, no brain region was found to be significantly correlated with depression. The related brain regions in female AD patients are shown in Figure 9 The voxel values, peak intensities, and peak MNI coordinates of different brain regions of the patient are shown in Table 1.

[0159] Table 1

[0160]

[0161] Result 2:

[0162] In male AD patients, the severity of agitation is negatively correlated with the atrophy of the right inferior temporal lobe, superior temporal pole, middle cingulate gyrus, left middle temporal pole, and parahippocampal gyrus (P <0.001); while in female AD patients, no brain region was found to be significantly correlated with agitation.

[0163] The related brain regions in male AD patients are shown in Figure 10The voxel values, peak intensities, and peak MNI coordinates for different brain regions of this patient are shown in Table 2 below. The "main brain regions" in the table are the significantly different brain region areas obtained by identifying the AD neuropsychiatric symptom-specific lesion model described above. Each brain region is volumetric and has a coordinate XYZ, and the peak MNI represents its coordinates, where the right inferior temporal lobe is a large brain region. It can be seen from Figure 10 the right inferior temporal lobe, the method of the present application more accurately locates which part of the brain region has more differences in the small brain region cluster, and the segmentation is more accurate. Here, it is said that the voxel values, peak intensities, and coordinates of different difference brain regions, which can more clearly indicate which part of the brain region has differences and is more accurate. Instead of generally expressing which part of the brain region has differences.

[0164] Table 2

[0165]

[0166] In one embodiment, the present application proposes a calculation method for obtaining NPS phenotype-specific anatomical features based on a double-encoder contrast decoding framework at the whole brain voxel level, which belongs to the technical tracking innovation: NPS imaging research evidence is limited and mostly concentrated in traditional research paradigms at a single scale or in pre-set regions of interest, and is affected by multiple factors such as researchers' subjective thinking, patient age, and disease stage, and cannot accurately obtain the extent to which NPS gender differences are caused by anatomical abnormalities. The present application combines contrast learning, three-dimensional convolutional neural networks, and variational autoencoders to propose a double-encoder contrast decoding framework, extract AD-NPS phenotype-specific anatomical features based on whole brain voxel level, accurately separate NPS different phenotype-specific anatomical features from other shared features, extract NPS highly sensitive brain image representations, and construct NPS-specific brain injury networks. From local to global, to multiple voxel depth development of anatomical abnormality network, form an interpretable multi-scale feature fusion algorithm for NPS phenotype-specific damage mode, and then combine with other multi-modal imaging, gene / space transcriptomics, targeted metabolomics, cerebrospinal fluid or blood pathological markers, etc. to reveal NPS phenotype-specific brain network damage and potential metabolic pathways, and provide an innovative and clinically valuable technical means for further understanding the pathophysiological mechanisms of NPS gender differences and exploring the anatomical basis of other complex brain diseases.

[0167] The embodiment of the present application also provides a computer program product or system, comprising a computer program which, when executed by a processor, implements the brain feature classification method based on the whole brain voxel double encoder, or implements the construction method of the AD neuropsychiatric symptom specific lesion model, or implements the classification method step based on the AD neuropsychiatric symptom specific lesion model.

[0168] Figure 2 The embodiment of the present application provides a double encoder comparison system based on the whole brain voxel level, and specifically comprises:

[0169] The acquisition unit acquires the three-dimensional whole brain image data set.

[0170] The construction unit constructs a double encoder comparison decoding model by using the three-dimensional whole brain image data set, and the double encoder comparison decoding model is used for difference recognition and classification of the whole brain image data.

[0171] The embodiment of the present application provides a brain feature classification system based on the whole brain voxel double encoder, comprising:

[0172] The acquisition module acquires the whole brain image data of the to-be-detected person.

[0173] The classification module inputs the whole brain image data into the double encoder comparison decoding model to obtain the classification result of the brain anatomical features being normal or abnormal, and the double encoder comparison decoding model is obtained based on the double encoder comparison method based on the whole brain voxel level.

[0174] The embodiment of the present application provides a construction system of an AD neuropsychiatric symptom specific lesion model, comprising:

[0175] The acquisition unit acquires the whole brain image data set of the AD disease spectrum patient and the normal whole brain image data set.

[0176] The training unit: the AD disease spectrum patient whole brain image and normal whole brain image data set are input into the double encoder contrast decoding model to extract whole brain features and classify the training, and an AD neuropsychiatric symptom specific lesion model is obtained.

[0177] The embodiment of the present application provides a classification system based on the AD neuropsychiatric symptom specific lesion model, which comprises:

[0178] The acquisition module: acquiring the whole brain image data of the to-be-tested person;

[0179] The recognition module: the whole brain image data is input into the AD neuropsychiatric symptom specific lesion model to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom specific lesion model.

[0180] Figure 3 The embodiment of the present application provides a computer device schematic diagram, which specifically comprises:

[0181] The memory and the processor; the memory is used for storing program instructions; the processor is used for calling program instructions, when the program instructions are executed, the above-mentioned double encoder contrast method based on the whole brain voxel level, or the above-mentioned brain feature classification method based on the whole brain voxel double encoder is executed, or the above-mentioned construction method of the AD neuropsychiatric symptom specific lesion model is executed, or the above-mentioned classification method based on the AD neuropsychiatric symptom specific lesion model is executed.

[0182] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the above-mentioned double encoder contrast method based on the whole brain voxel level, or to realize the above-mentioned brain feature classification method based on the whole brain voxel double encoder, or to realize the above-mentioned construction method of the AD neuropsychiatric symptom specific lesion model, or to realize the above-mentioned classification method based on the AD neuropsychiatric symptom specific lesion model.

[0183] The verification result of the verification embodiment shows that assigning inherent weights to the indications can improve the performance of the method compared with the default setting. It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme. In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of software functional units. Those skilled in the art can understand that all or part of the steps of the various methods in the above embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, which can include read only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0184] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, and the above-mentioned medium storage can be read only memory, magnetic disk or optical disk, etc.

[0185] The computer device provided by the present application has been described in detail above. For those skilled in the art, according to the idea of the embodiment of the present application, there will be changes in specific implementation and application range. In view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A dual encoder comparison method based on whole-brain voxel level, characterized in that, include: Obtain a 3D whole-brain imaging dataset; A dual encoder contrast decoding model was constructed using the aforementioned three-dimensional whole-brain image dataset. This dual encoder contrast decoding model is used to compare whole-brain image data for difference recognition and classification. The construction process of the dual encoder contrast decoding model is as follows: the three-dimensional whole-brain images are grouped into normal or diseased groups, and any normal three-dimensional whole-brain image is selected as the reference image. The reference image is input into the encoder model to be trained in parallel with other normal or diseased three-dimensional whole-brain images to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the encoder model to be trained and the comparison calculation steps are repeated until the preset stopping condition is reached to obtain the dual encoder contrast decoding model.

2. The dual encoder comparison method based on whole-brain voxel level according to claim 1, characterized in that, When the reference image is input to the encoder to be trained in parallel with other normal or diseased 3D whole-brain images, a second comparison is also included. After the reference image is compared with other normal 3D whole-brain images, the second comparison calculation is performed and the reference image is input to the encoder to be trained for the other normal 3D whole-brain images to obtain a first output feature. After the reference image is compared with diseased 3D whole-brain images, the reference image is input to the encoder to be trained for the diseased 3D whole-brain images to obtain a second output feature. The reference image is input to the encoder to be trained for the reference image to obtain a third output feature. The first output feature and the third output feature are compared and calculated, and the second output feature and the third output feature are compared and calculated to obtain the whole-brain differential classification result. Optionally, the reference image and the diseased three-dimensional whole-brain image are compared in a second way to obtain similar features and difference features. The similar features and difference features are then input into the encoder to be trained of the diseased three-dimensional whole-brain image. The encoder to be trained of the diseased three-dimensional whole-brain image encodes the similar features and difference features in parallel to obtain similar coded features and difference coded features. The similar coded features and difference coded features are then fused and decoded to obtain the third output feature.

3. The dual encoder comparison method based on whole-brain voxel level according to claim 1 or 2, characterized in that, The encoder is any one or more of the following: variational autoencoder, vector quantization variational autoencoder, adversarial autoencoder, sparse autoencoder, conditional variational autoencoder, Beta-VAE; Optionally, the encoder is a variational autoencoder. The reference image and other normal or diseased three-dimensional whole-brain images are input in parallel into the variational autoencoder model to be trained to obtain output features. The output features are compared and calculated to obtain the whole-brain difference classification result. Based on the reference image and other normal or diseased three-dimensional whole-brain images, the variational autoencoder model to be trained and the comparison calculation steps are repeated until a preset stopping condition is reached to obtain a dual encoder comparison decoding model. Optionally, the encoder includes an encoding layer and a decoding layer. The reference image and other normal or diseased three-dimensional whole-brain images are encoded by feature compression in their respective encoders through the encoding layer, and then reconstructed by the decoding layer to obtain output features. The encoding layer includes a three-dimensional convolution module. The feature is compressed to the latent space through the three-dimensional convolution module to calculate the feature probability distribution parameters and obtain compressed features. The compressed features are then sampled and reconstructed by the three-dimensional convolution module of the decoding layer to obtain output features. Optionally, the dual encoder contrast decoding model further includes morphological analysis, which calculates local volume change features by reconstructing the output features from the decoding layer and the input image of the encoder, fuses the output features and the local volume change features to obtain fused features, and compares and calculates the fused features to obtain whole-brain differential classification results. Alternatively, the reference image is compared with other normal 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the other normal 3D whole-brain images to obtain a first output feature. The first output feature is then compared with the input image of the encoder to obtain a first local volume change feature. The first output feature and the first local volume change feature are fused to obtain a first fusion feature. The reference image is compared with diseased 3D whole-brain images for a second calculation, and then input into the encoder to be trained on the diseased 3D whole-brain images to obtain a second output feature. The second output feature is then compared with the input image of the encoder to obtain a second local volume change feature. The second output feature and the second local volume change feature are fused to obtain a second fusion feature. The reference image is input into the encoder to be trained on the reference image to obtain a third output feature. The first fusion feature and the third output feature are compared and calculated, and the second fusion feature and the third output feature are compared and calculated to obtain a whole-brain differential classification result.

4. The dual encoder comparison method based on whole-brain voxel level according to claim 1 or 2, characterized in that, The comparison calculation performs feature classification by calculating the distance between features, and obtains the whole brain difference classification result; Optionally, the reference image is input into the encoder to be trained to obtain the reference global output features, and the other normal or diseased three-dimensional whole brain images are input into the encoder to be trained to obtain the normal or diseased global output features. The reference global output features and the normal or diseased global output features are compared and calculated to obtain the whole brain differential classification results. Optionally, in the second comparison, the reference image and other normal 3D whole-brain images are first sampled to obtain a reference patch image and other normal 3D whole-brain patch images. Any one of the reference patch images is selected as the second reference image. Then, the second reference image is compared with other reference patch images or other normal 3D whole-brain patch images in a second comparison calculation, and then input into the encoder to be trained of the other normal 3D whole-brain images to obtain the first output feature. The reference image and diseased 3D whole-brain images are first sampled to obtain a reference patch image and diseased 3D whole-brain patch images. Any one of the reference patch images is selected as the third reference image. Then, the third reference image is compared with other reference patch images or diseased 3D whole-brain patch images in a second comparison calculation, and then input into the encoder to be trained of the diseased 3D whole-brain images to obtain the second output feature. The reference image is input into the encoder to be trained to obtain the third output feature. The first output feature and the third output feature are compared and calculated, and the second output feature and the third output feature are compared and calculated to obtain the whole-brain differential classification result. Optionally, the second comparison calculation is to calculate the image feature distance between the second reference image and other reference patch images or other normal three-dimensional whole brain patch images to obtain the first intra-class classification result, and input the first intra-class classification into the encoder to be trained of other normal three-dimensional whole brain images to obtain the first output feature; Alternatively, the third reference image can be used to calculate the image feature distance with other reference patch images or diseased three-dimensional whole-brain patch images to obtain the second internal classification result. The second internal classification is then input into the encoder to be trained on the diseased three-dimensional whole-brain image to obtain the second output feature.

5. A brain feature classification method based on a whole-brain voxel dual encoder, characterized in that, include: Acquire whole-brain imaging data of the test subject; The whole-brain imaging data is input into a dual-encoder contrast decoding model to obtain a classification result of whether the brain anatomical features are normal or abnormal. The dual-encoder contrast decoding model is obtained based on the dual-encoder contrast method based on the whole-brain voxel level as described in any one of claims 1-5.

6. A method for constructing a specific lesion model for identifying AD neuropsychiatric symptoms, characterized in that, include: Obtain datasets of whole-brain images from patients with the AD disease spectrum and normal whole-brain images; The dataset of whole-brain images of patients with the AD disease spectrum and normal whole-brain images is input into a dual-encoder comparison decoding model to extract whole-brain features and classify them for training, thereby obtaining a model for identifying specific lesions of AD neuropsychiatric symptoms; the dual-encoder comparison decoding model is obtained based on the dual-encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-5. Optionally, the AD disease spectrum includes one or more of twelve neuropsychiatric symptom phenotypes. The whole brain images of patients with the AD disease spectrum and normal whole brain images are input into a dual encoder comparison decoding model to extract one or more of the normal features of the whole brain and the specific lesion features of the whole brain and classify them for training, so as to obtain a first model for recognizing specific lesions of AD neuropsychiatric symptoms. Optionally, the whole-brain images of patients with the AD disease spectrum include phenotype images with neuropsychiatric symptoms and phenotype images without neuropsychiatric symptoms. The whole-brain images of patients with the AD disease spectrum and normal whole-brain images are input into a dual encoder comparison decoding model to extract normal features of the whole brain and specific lesion features of the whole brain and classify them. During the training process, the three-dimensional images of the diseased whole brain are alternately input into the phenotype images with neuropsychiatric symptoms and the phenotype images without neuropsychiatric symptoms for training, to obtain a second model for recognizing specific lesions of AD neuropsychiatric symptoms. Optionally, during the training process of images with neuropsychiatric symptoms, similar coding features and differential coding features are updated, while during the training process of images without neuropsychiatric symptoms, similar coding features are updated. Optionally, the whole-brain images of patients with the AD disease spectrum include images with phenotypes accompanied by neuropsychiatric symptoms and images without phenotypes accompanied by neuropsychiatric symptoms. The images with phenotypes accompanied by neuropsychiatric symptoms include one or more of twelve neuropsychiatric phenotypes. The whole-brain images of patients with the AD disease spectrum and the normal whole-brain images dataset are input into a dual encoder comparison decoding model to extract normal features of the whole brain and specific lesion features of the whole brain and classify them for training, thereby obtaining a third model for recognizing specific lesions of AD neuropsychiatric symptoms. Optionally, the construction of the model for identifying specific lesions of AD neuropsychiatric symptoms further includes a first specific feature, acquiring general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data of patients and normal individuals in the AD disease spectrum, extracting features from the general information, neuropsychological assessment data and / or cerebrospinal fluid or plasma pathological marker data to obtain a first disease-specific feature, performing similarity analysis between the first specific feature and the whole-brain features extracted by the dual encoder decoding model to obtain a specific feature, and classifying based on the specific feature and / or whole-brain features to obtain a model for identifying specific lesions of AD neuropsychiatric symptoms.

7. A classification method based on identifying specific lesion models of AD neuropsychiatric symptoms, characterized in that, include: Acquire whole-brain imaging data of the test subject; The whole-brain imaging data is input into the AD neuropsychiatric symptom-specific lesion identification model for classification to obtain classification results of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion identification model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion identification model as described in claim 6; Optionally, the whole-brain imaging data is input into a first AD neuropsychiatric symptom-specific lesion model for classification to obtain one or more classification results of one or more of the twelve neuropsychiatric symptom phenotypes of the AD disease spectrum, either normal or suffering from AD; the first AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 6; Optionally, the whole-brain imaging data is input into a second AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having an AD disease spectrum with neuropsychiatric symptom phenotype or having an AD disease spectrum without neuropsychiatric symptom phenotype; the second AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 6; Optionally, the whole-brain imaging data is input into a third AD neuropsychiatric symptom-specific lesion model for classification to obtain classification results of normal or having AD disease spectrum with neuropsychiatric symptom phenotype or having AD disease spectrum without neuropsychiatric symptom phenotype; the third AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 6; Optionally, the acquisition further includes acquiring general information of the subject, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data; extracting features from the general information, neuropsychological assessment data, and / or cerebrospinal fluid or plasma pathological marker data to obtain a first specific feature; inputting the whole brain imaging data and the first specific feature into the AD neuropsychiatric symptom-specific lesion model for classification to obtain a classification result of normal or AD disease spectrum; the AD neuropsychiatric symptom-specific lesion model is obtained by the construction method of the AD neuropsychiatric symptom-specific lesion model as described in claim 6.

8. A computer program product comprising a computer program or instructions, characterized in that, The computer program or instructions are executed by the processor to implement the dual encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-4, or to implement the brain feature classification method based on the whole-brain voxel dual encoder as described in claim 5, or to implement the construction method for identifying specific lesion models of AD neuropsychiatric symptoms as described in claim 6, or to implement the classification method based on identifying specific lesion models of AD neuropsychiatric symptoms as described in claim 7.

9. A computer device comprising a memory, a processor, and a computer program or instructions stored in the memory, characterized in that, The computer program or instructions are executed by the processor to implement the dual encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-4, or to implement the brain feature classification method based on the whole-brain voxel dual encoder as described in claim 5, or to implement the construction method for identifying specific lesion models of AD neuropsychiatric symptoms as described in claim 6, or to implement the classification method based on identifying specific lesion models of AD neuropsychiatric symptoms as described in claim 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instructions are executed by the processor to implement the dual encoder comparison method based on the whole-brain voxel level as described in any one of claims 1-4, or to implement the brain feature classification method based on the whole-brain voxel dual encoder as described in claim 5, or to implement the construction method for identifying specific lesion models of AD neuropsychiatric symptoms as described in claim 6, or to implement the classification method based on identifying specific lesion models of AD neuropsychiatric symptoms as described in claim 7.

Images