A deep learning-based method for synthesizing transmembrane state of pet images

Abstract

The present invention discloses a PET image transmembrane state synthesis method based on deep learning, which is suitable for dopamine transporter (DAT) imaging and improves the diagnostic accuracy of Parkinson's disease through advanced image processing technology. 11 It consists of two core parts: C-CFT generation model and PD classification model. 11 The C-CFT generation model is responsible for converting the actual scanned 18 F-FDG PET images were converted into 11 C‑CFT PET images, while the PD classification model utilizes real 18 F‑FDG PET images and synthesis 11 The multimodal information output prediction results of C-CFT PET images. The advantages of the present invention are that 11 Advantages of C-CFT PET in the assessment of dopaminergic system by synthesizing 11 C-CFT PET images directly reflect the DAT level in the striatum of PD patients, providing an intuitive visual basis for the model's diagnostic results, helping doctors understand and trust the diagnostic decisions of the AI ​​model, and effectively overcoming the 11 The resource limitations of C-CFT PET imaging make it possible to implement accurate automated PD diagnosis technology in a wider range of medical settings.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to a method for synthesizing transmembrane states of PET images based on deep learning. Background Art

[0002] Parkinson's disease (PD) is a common progressive neurodegenerative disorder characterized by the gradual degeneration of dopaminergic neurons in the substantia nigra pars compacta. This neuronal degeneration primarily affects the part of the brain that controls movement, leading to a series of typical motor symptoms such as resting tremor, muscle rigidity, bradykinesia, and postural balance disorders. In addition, Parkinson's disease may also be accompanied by non-motor symptoms, including cognitive impairment, affective disorders, sleep disorders, and autonomic dysfunction. Diagnosis of Parkinson's disease is particularly difficult in the early stages because its symptoms are similar to those of other neurodegenerative diseases and early symptoms may be mild or atypical. Currently, clinical diagnosis is usually made after symptoms of movement disorders appear, and the confirmation process often takes time, resulting in delayed diagnosis and treatment. Therefore, the development of early diagnostic tools is crucial to improving the prognosis of Parkinson's disease patients.

[0003] Positron emission tomography (PET) is a nuclear medicine imaging technique that provides information about the internal chemical processes and physiological functions of an organism by injecting radioactive tracers into the patient's body and circulating through the blood to specific organs or tissues. Among them, dopamine transporter (DAT) imaging can be achieved by using specific radioactive tracers (such as 11 C-CFT) reflects the functional status of dopaminergic neurons, thus becoming an important tool for the diagnosis of Parkinson's disease. Numerous studies have shown that 11 In C-CFT PET imaging, the DAT level in the striatum of the healthy control (NC) group was normal, while that in PD patients was significantly reduced. 11 Normal DAT levels shown by C-CFT PET imaging can be used as an absolute exclusion criterion for Parkinson's disease.

[0004] However, most hospitals or medical institutions have difficulty obtaining 11 C-CFT tracer, only fluorodeoxyglucose ( 18 F-FDG) is used as a tracer in PET imaging for PD diagnosis. 18 F-FDG is the most widely used radiotracer in the field of neuroimaging and can indirectly reflect the activity of nerve cells by evaluating the level of glucose metabolism in the brain. 18In F-FDG PET imaging, the glucose metabolism level of the basal ganglia region of PD patients is often increased, which is related to compensatory metabolic changes caused by the loss of dopaminergic neurons. Nevertheless, due to the inability to directly assess the functional status of dopaminergic neurons, 18 F-FDG PET imaging is difficult to distinguish PD from other neurodegenerative diseases, especially in the early stages of the disease, where metabolic changes are not significant enough to make differential diagnosis particularly difficult.

[0005] Patent document (CN114862774A) discloses a PET image cross-modality reconstruction method based on deep learning. The method constrains and modifies the generated image by constructing a neural network, thereby obtaining a high-quality AV45-PET brain scan image. This method is only suitable for observing the cortical function of Alzheimer's disease and is not suitable for subcortical dopaminergic function examination. SUMMARY

[0006] The present application provides a PET image cross-modality synthesis method based on deep learning, which solves the problem that most hospitals or medical institutions are difficult to obtain 11 C-CFT tracer, using fluorodeoxyglucose (F-FDG) as a tracer for PET imaging for PD diagnosis, 18 F-FDG PET imaging is difficult to distinguish PD from other neurodegenerative diseases. 8 F-FDG PET imaging is difficult to distinguish PD from other neurodegenerative diseases.

[0007] To solve the above technical problems, the present application discloses the following technical solutions:

[0008] The present application provides a PET image cross-modality synthesis method based on deep learning, which includes an image acquisition step, a first model construction step, a second model construction step, a third model construction step, a fourth model construction step, a fifth model construction step, a first training step, a second training step, and an input step.

[0009] The image acquisition step is used to acquire two or more first PET brain scan images and two or more second PET brain scan images, the first PET brain scan images being first tracer PET brain scan images of a subject, and the second PET brain scan images being second tracer PET brain scan images of a subject; the first model in the first model construction step adopts a 3D U-Net network structure, the 3D U-Net network structure comprising an encoder, a bottleneck layer, a decoder, and a skip connection, the encoder comprising two or more residual modules and a down-sampling layer, the bottleneck layer comprising two or more residual modules, the structure of the decoder being symmetrical to that of the encoder, and the skip connection being used to cascade the encoder features and the decoder features of different scales; the second model in the second model construction step adopts a 3D ResNet-18 network structure, the 3D ResNet-18 network structure being composed of a feature extractor and a fully connected layer, the feature extractor comprising two or more residual modules and a convolutional layer; the third model in the third model construction step is constructed based on the training mode of Cycle GAN to optimize the first model; the fourth model in the fourth model construction step is constructed based on the training mode of Patch GAN and a projection discriminator to optimize the first model; the fifth model in the fifth model construction step is constructed based on the training mode of Patch GAN and a projection discriminator to optimize the third model; the first training step is used to build a first cycle path and a second cycle path, the first PET brain scan images being generated into first images through the first cycle path, the first images and the first PET brain scan images being subjected to first similarity constraints based on the cycle consistency loss of an l1 loss function, the second PET brain scan images being generated into second images through the second cycle path, and the second images and the second PET brain scan images being subjected to second similarity constraints based on the cycle consistency loss of an l1 loss function; the second training step is to input the first PET brain scan images into the trained first model to generate third images, to construct image pairs by cascading the first PET brain scan images and the third images, to input the image pairs into the second model, to output prediction results, to use real disease labels as supervision information, to perform third similarity constraints on the prediction results and the real disease labels through a cross-entropy loss, and to classify the prediction results; and the input step is to input any first PET brain scan image into the trained first model and the trained second model to output classification results.

[0010] Further, the first training step comprises a first image conversion step, a first image reconstruction step, a fourth similarity constraint step, and a first similarity constraint step.

[0011] the first image conversion step is converting the first PET brain scan image into a first synthetic PET brain scan image by inputting the first PET brain scan image into the first model; the first image reconstruction step is generating the first image by inputting the first synthetic PET brain scan image into the third model, forming the first cycle path; the fourth similarity constraint step is performing fourth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image based on the adversarial loss of the hinge loss function by the fourth model; the first similarity constraint step is performing first similarity constraint on the first PET brain scan image and the first image based on the cycle consistency loss of the l1 loss function.

[0012] Further, the first training step further comprises a second image conversion step, a second image reconstruction step, a fifth similarity constraint step and a second similarity constraint step.

[0013] the second image conversion step is converting the second PET brain scan image into a second synthetic PET brain scan image by inputting the second PET brain scan image into the third model; the second image reconstruction step is reconstructing the second image by inputting the second synthetic PET brain scan image into the first model, forming the second cycle path; the fifth similarity constraint step is performing fifth similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image based on the adversarial loss of the hinge loss function by the fifth model; the second similarity constraint step is performing second similarity constraint on the second PET brain scan image and the second image based on the cycle consistency loss of the l1 loss function.

[0014] Further, in the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, the first training step further comprises a first image conversion step and a sixth similarity constraint step.

[0015] the first image conversion step is converting the first PET brain scan image into a first synthetic PET brain scan image by inputting the first PET brain scan image into the first model; the sixth similarity constraint step is performing sixth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image based on the reconstruction loss of the l1 loss function.

[0016] Further, in the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, the first training step further comprises a second image conversion step and a seventh similarity constraint step.

[0017] The second image conversion step is to input the second PET brain scan image into the third model to convert into a second synthetic PET brain scan image; and the seventh similarity constraint step is to perform seventh similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image based on a reconstruction loss of an l1 loss function.

[0018] Further, in the inputting step, when the first PET brain scan image is input into the trained first model, the inputting step specifically includes a downsampling step, a first feature extraction step, and an upsampling step.

[0019] The downsampling step is to downsample the input first PET brain scan image to 1 / 8 of the original size by the encoder to obtain a first feature map; the first feature extraction step is to extract first image features of the first feature map by the bottleneck layer to obtain a second feature map; and the upsampling step is to upsample the second feature map to the original size by the decoder to obtain the third image.

[0020] Further, in the inputting step, when the first PET brain scan image and the third image are input into the trained second model, the inputting step specifically includes an image pair construction step, a projection step, a second feature extraction step, a feature vector acquisition step, and a mapping step.

[0021] The image pair construction step is to construct the first PET brain scan image and the third image into an image pair in a cascading manner; the projection step is to input the image pair into the feature extractor, and the feature extractor projects the image pair by the convolution layer; the second feature extraction step is to extract second image features of the image pair after reducing the data dimension by the residual module; the feature vector acquisition step is to obtain a feature vector with a dimension of 512 from the second image features based on a global average pooling method; and the mapping step is to perform nonlinear mapping on the feature vector by the full connection layer to output a two-dimensional vector, and the two-dimensional vector is the prediction result.

[0022] Further, the first model, the third model, the fourth model, and the fifth model all apply spectral normalization, and an Adam optimizer is used to update the weights of the first model, the weights of the third model, the weights of the fourth model, and the weights of the fifth model; and the residual modules of the first model all use LeakyReLU activation functions and instance normalization.

[0023] Further, the second model adopts a cosine annealing learning rate updating algorithm and an Adam optimizer to update the weights of the second model; the convolutional layer and the residual module of the second model both use a ReLU activation function and batch normalization.

[0024] Compared with the prior art, the present application has at least the following technical effects:

[0025] The present application provides a PET image transmembrane state synthesis method based on deep learning, which converts 18 F-FDG PET images into 11 C-CFT PET images, takes advantage of 11 C-CFT PET in the evaluation of the dopaminergic system, significantly improves the accuracy of the AI model in PD diagnosis, and can better help doctors determine whether the subject is sick; the synthesized 11 C-CFT PET images directly reflect the DAT level of the striatal region of PD patients, providing intuitive visual evidence for model diagnosis results, which helps doctors understand and trust the AI model's diagnosis decision; relying on common 18 F-FDG tracers, effectively overcoming the resource limitations of 11 C-CFT PET imaging, so that precise PD automatic diagnosis technology can be implemented in a wider medical environment, especially for hospitals or medical institutions that are difficult to obtain 11 C-CFT tracers, which helps doctors in hospitals or medical institutions that are difficult to obtain 11 C-CFT tracers to diagnose PD. BRIEF DESCRIPTION OF DRAWINGS

[0026] The technical solutions of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0027] Figure 1 A schematic diagram of a PET image transmembrane state synthesis method based on deep learning according to an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the first training step and the second training step according to an embodiment of the present application;

[0029] Figure 3 A flowchart of a PET image transmembrane state synthesis method based on deep learning according to an embodiment of the present application;

[0030] Figure 4 A flowchart of the first training step according to an embodiment of the present application; Figure 1

[0031] Figure 5 A flowchart of the first training step according to an embodiment of the present application​Figure 2 ;

[0032] Figure 6 This is the process of the first training step described in the embodiment of the present invention Figure 3 ;

[0033] Figure 7 This is the process of the first training step described in the embodiment of the present invention Figure 4 ;

[0034] Figure 8 The process of inputting steps described in the embodiment of the present invention Figure 1 ;

[0035] Figure 9 The process of inputting steps described in the embodiment of the present invention Figure 2 . DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0037] like Figure 1 - Figure 3 As shown, the present invention provides a PET image transmembrane state synthesis method based on deep learning, which is suitable for dopamine transporter (DAT) imaging and improves the diagnostic accuracy of Parkinson's disease through advanced image processing technology. 11 The C-CFT generation model and the PD classification model are the two core parts. 11 The C-CFT generation model is responsible for converting the actual scan 18 F-FDG PET images were converted into 11 C-CFTPET images, while the PD classification model uses real 18 F-FDG PET images and synthesis 11 The multimodal information of C-CFT PET images is used to improve the accuracy of PD judgment by the artificial intelligence model, which can better help doctors analyze whether a subject is sick. The present invention includes step S1) an image acquisition step, step S2) a first model construction step, step S3) a second model construction step, step S4) a third model construction step, step S5) a fourth model construction step, step S6) a fifth model construction step, step S7) a first training step, step S8) a second training step, and step S9) an input step.

[0038] Step S1) an image acquisition step of acquiring two or more first PET brain scan images and two or more second PET brain scan images, the first PET brain scan images being first tracer PET brain scan images of a subject, and the second PET brain scan images being second tracer PET brain scan images of the subject.

[0039] In this embodiment, the first PET brain scan images are actually scanned 18 F-FDG PET images, and the second PET brain scan images are actually scanned 11 C-CFT PET images, each of the actually scanned F-FDG PET images of a subject has a corresponding actually scanned 18 C-CFT PET image of the subject, the first tracer is 11 F-FDG tracer, and the second tracer is 18 C-CFT tracer. 11

[0040] Step S2) a first model construction step, the first model adopts a 3D U-Net network structure, the 3D U-Net network structure includes an encoder, a bottleneck layer, a decoder, and a skip connection, the encoder includes two or more residual modules and a down-sampling layer, the bottleneck layer includes two or more residual modules, the structure of the decoder is symmetrical to the structure of the encoder, and the skip connection is used to cascade the encoder features and the decoder features of different scales to promote information flow.

[0041] In this embodiment, the first model is a 11 C-CFT generation model, which can convert the actually scanned 18 F-FDG PET images into corresponding synthetic 11 C-CFT PET images.

[0042] Step S3) a second model construction step, the second model adopts a 3D ResNet-18 network structure, the 3D ResNet-18 network structure is composed of a feature extractor and a fully connected layer, and the feature extractor includes two or more residual modules and a convolutional layer.

[0043] In this embodiment, the second model is a PD classification model, which can integrate the actually scanned 18 F-FDG PET images and the synthetic 11 ​The multi-modal information of the C-CFT PET image is processed and analyzed, and the output result is divided into two categories of healthy and diseased, so as to improve the accuracy of the artificial intelligence model in diagnosing PD and better help doctors to judge whether the subject is diseased.

[0044] Step S4) third model construction step, the first model is optimized based on the training mode of Cycle GAN, and a third model with the same structure as the first model is constructed.

[0045] In this embodiment, the third model is 18 F-FDG generation model, which can convert the actually scanned 11 C-CFT PET image into a synthetic 18 F-FDG PET image.

[0046] Step S5) fourth model construction step, the first model is optimized based on the training mode of Patch GAN and the projection discriminator, and a fourth model is constructed.

[0047] In this embodiment, the fourth model is 11 C-CFT discriminant model, which aligns the data distribution of the synthetic 11 C-CFT PET image content with the actually scanned 11 C-CFT PET image.

[0048] Step S6) fifth model construction step, the third model is optimized based on the training mode of Patch GAN and the projection discriminator, and a fifth model is constructed.

[0049] In this embodiment, the fifth model is 18 F-FDG discriminant model, which aligns the data distribution of the synthetic 18 F-FDG PET image content with the actually scanned 18 F-FDG PET image.

[0050] Step S7) first training step, a first cycle path and a second cycle path are built, the first PET brain scan image is generated into a first image through the first cycle path, the first image and the first PET brain scan image are subjected to first similarity constraint based on the cycle consistency loss of the l1 loss function, the second PET brain scan image is generated into a second image through the second cycle path, and the second image and the second PET brain scan image are subjected to second similarity constraint based on the cycle consistency loss of the l1 loss function.

[0051] In this embodiment, the first cycle path is a positive cycle path 18 F-FDG→ 11C-CFTv 18 F-FDG), the second loop path is a reverse loop path 11 C-CFT→ 18 F-FDG→ 11 C-CFT), the first loop path refers to the actual scanned 18 F-FDG PET image is converted into a synthetic 11 C-CFT PET image, the synthetic 11 C-CFT PET image is converted into a synthetic 18 F-FDG PET image, the actual scanned 18 F-FDG PET image is converted into a synthetic 18 F-FDG PET image can be understood as an image reconstruction process, the second loop path refers to the actual scanned 11 C-CFT PET image is converted into a synthetic 18 F-FDG PET image, the synthetic 18 F-FDG PET image is converted into a synthetic 11 C-CFT PET image, the actual scanned 11 C-CFT PET image is converted into a synthetic 11 C-CFT PET image can also be understood as an image reconstruction process.

[0052] The loop consistency loss refers to the consistency between the head and tail two As of each loop path A-->B-->A, that is, the positive loop path 18 F-FDG→ 11 C-CFT→ 18 F-FDG) head and tail two 18 F-FDG between the head and tail two 11 C-CFT→ 18 F-FDG→ 11 C-CFT) head and tail two 11 C-CFT between the head and tail two.

[0053] As Figure 4 shown, the step S7) first training step includes a step S711) first image conversion step, a step S712) first image reconstruction step, a step S713) fourth similarity constraint step and a step S714) first similarity constraint step.

[0054] The step S711) first image conversion step inputs the first PET brain scan image into the first model to convert into a first synthetic PET brain scan image, that is, the actual scanned 18F-FDG PET image is input into the first model to convert into the first composite 11 C-CFT PET image.

[0055] Step S712) a first image reconstruction step, inputting the first composite PET brain scan image into the third model to generate a first image, i.e., converting the first composite 11 C-CFT PET image into the first composite 18 F-FDG PET image, forming the first circulation path, i.e., the positive circulation path 18 F-FDG→ 11 C-CFT→ 18 F-FDG).

[0056] Step S713) a fourth similarity constraint step, performing fourth similarity constraint on the first composite PET brain scan image and the second PET brain scan image based on the adversarial loss of the fourth model, i.e., performing fourth similarity constraint on the first composite 11 C-CFT PET image and the actually scanned 11 C-CFT PET image, improving the accuracy of the first composite 11 C-CFT PET image.

[0057] In this embodiment, how to perform fourth similarity constraint on the first composite 11 C-CFT PET image and the actually scanned 11 C-CFT PET image is a calculation process that can be easily implemented by those skilled in the art, and therefore the calculation process of the fourth similarity constraint is not described in this embodiment. 11 C-CFT PET image and the actually scanned 11 C-CFT PET image is a calculation process that can be easily implemented by those skilled in the art, and therefore the calculation process of the fourth similarity constraint is not described in this embodiment.

[0058] Step S714) a first similarity constraint step, performing first similarity constraint on the first PET brain scan image and the first image based on the cycle consistency loss of the l1 loss function, to improve the accuracy of the first image.

[0059] In this embodiment, how to perform first similarity constraint on the actually scanned 18 F-FDG PET image and the first composite 18 F-FDG PET image is a calculation process that can be easily implemented by those skilled in the art, and therefore the calculation process of the fourth similarity constraint is not described in this embodiment. 18 F-FDG PET image and the first composite 18The first similarity constraint on the F-FDG PET image is a calculation process that can be easily implemented by those skilled in the art, and therefore the calculation process of the first similarity constraint is not described in this embodiment.

[0060] As shown in FIG. 7, the first training step S7) further comprises a second image conversion step S721), a second image reconstruction step S722), a fifth similarity constraint step S723), and a second similarity constraint step S724). Figure 5

[0061] The second image conversion step S721) inputs the second PET brain scan image into the third model to convert the second PET brain scan image into a second synthetic PET brain scan image, i.e., the actual scanned C-CFT PET image is converted into a second synthetic C-CFT PET image. 11 The second image conversion step S721) inputs the second PET brain scan image into the third model to convert the second PET brain scan image into a second synthetic PET brain scan image, i.e., the actual scanned C-CFT PET image is converted into a second synthetic C-CFT PET image. 18 The second image conversion step S721) inputs the second PET brain scan image into the third model to convert the second PET brain scan image into a second synthetic PET brain scan image, i.e., the actual scanned C-CFT PET image is converted into a second synthetic C-CFT PET image.

[0062] The second image reconstruction step S722) inputs the second synthetic PET brain scan image into the first model to reconstruct the second image, i.e., the second synthetic C-CFT PET image is converted into a second synthetic F-FDG PET image. 18 The second image reconstruction step S722) inputs the second synthetic PET brain scan image into the first model to reconstruct the second image, i.e., the second synthetic C-CFT PET image is converted into a second synthetic F-FDG PET image. 18 The second image reconstruction step S722) inputs the second synthetic PET brain scan image into the first model to reconstruct the second image, i.e., the second synthetic C-CFT PET image is converted into a second synthetic F-FDG PET image. 11 C-CFT→ 18 F-FDG→ 11 C-CFT).

[0063] The fifth similarity constraint step S723) performs a fifth similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image based on the adversarial loss of the hinge loss function through the fifth model, thereby improving the accuracy of the second synthetic PET brain scan image.

[0064] In this embodiment, how to perform the fourth similarity constraint on the second synthetic F-FDG PET image and the actual scanned F-FDG PET image is a calculation process that can be easily implemented by those skilled in the art, and therefore the calculation process of the fifth similarity constraint is not described in this embodiment. 18 The second synthetic F-FDG PET image and the actual scanned F-FDG PET image. 18 The second synthetic F-FDG PET image and the actual scanned F-FDG PET image. 18 The second synthetic F-FDG PET image and the actual scanned F-FDG PET image. 18 The second synthetic F-FDG PET image and the actual scanned F-FDG PET image.

[0065] ​Step S724) a second similarity constraint step, based on a cycle-consistency loss of an l1 loss function, performing a second similarity constraint on the second PET brain scan image and the second image, to improve the accuracy of the second image.

[0066] In this embodiment, the cycle-consistency loss of the l1 loss function is known to be consistent with the actual scan of the subject of the similarity constraint 11 C-CFT PET image and the second synthetic 11 C-CFT PET image, how to perform a first similarity constraint on the actual scan of the subject of the similarity constraint 11 C-CFT PET image and the first synthetic 11 C-CFT PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the second similarity constraint is not described in this embodiment.

[0067] As shown in Figure 6 the first training step, when the first PET brain scan image and the second PET brain scan image come from the same subject, i.e., the actual scan of the subject of the similarity constraint 18 F-FDG PET image and the actual scan of the subject of the similarity constraint 11 C-CFT PET image come from the same patient, the first image conversion step S711) and the sixth similarity constraint step S715) are included.

[0068] Step S711) a first image conversion step, inputting the first PET brain scan image into the first model to convert into a first synthetic PET brain scan image, i.e., inputting the actual scan of the subject of the similarity constraint 18 F-FDG PET image into the first model to convert into a first synthetic 11 C-CFT PET image.

[0069] Step S715) a sixth similarity constraint step, based on a reconstruction loss of an l1 loss function, performing a sixth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image.

[0070] In this embodiment, the reconstruction loss of the l1 loss function is known to be consistent with the actual scan of the subject of the similarity constraint 11 C-CFT PET image and the first synthetic 11 C-CFT PET image, how to perform a first similarity constraint on the actual scan of the subject of the similarity constraint 11 C-CFT PET image and the first synthetic 11The first similarity constraint on the C-CFT PET image is a computational process that can be easily implemented by those skilled in the art, so the calculation process of the sixth similarity constraint is not described in this embodiment.

[0071] The reconstruction loss means that if a subject has a pair of A and B, then for the cycle path 1 (A-->B-->A) and the cycle path 2 (B-->A-->B), the similarity constraint is performed directly between the input A of path 1 and the intermediate synthesis A of path 2, and between the input B of path 2 and the intermediate synthesis B of path 1, that is, the similarity constraint is performed between the input of the positive cycle path 18 F-FDG→ 11 C-CFT→ 18 F-FDG) and the input of the inverse cycle path 11 C-CFT→ 11 C-CFT→ 18 F-FDG→ 11 C-CFT) and the input of the inverse cycle path 11 F-FDG→ 18 C-CFT→ 11 F-FDG→ 18 F-FDG) and the input of the inverse cycle path 18 C-CFT→ 11 C-CFT→ 18 F-FDGv 11 C-CFT) and the input of the inverse cycle path 18 F-FDG.

[0072] As Figure 7 shown, in the first training step, when the first PET brain scan image and the second PET brain scan image come from the same subject, it further includes a second image conversion step S721) and a seventh similarity constraint step S725).

[0073] The second image conversion step S721) inputs the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image, that is, the actually scanned 11 C-CFT PET image is input into the third model to convert it into a second synthetic 18 F-FDG PET image.

[0074] The seventh similarity constraint step is a reconstruction loss based on an l1 loss function, which performs a seventh similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image.

[0075] In this embodiment, the reconstruction loss of the l1 loss function is known to be similar to the actual scan of the object with the similarity constraint 18 F-FDG PET image and the second synthesis 18 F-FDG PET image, how to classify the actual scan of the object with the first similarity constraint 18 F-FDG PET image and the second synthesis 18 The first similarity constraint of the F-FDG PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the seventh similarity constraint is not described in this embodiment.

[0076] The first model, the third model, the fourth model and the fifth model all apply spectral normalization, and the weights of the first model, the weights of the third model, the weights of the fourth model and the weights of the fifth model are updated using the Adam optimizer.

[0077] Step S8) Second training step, input the first PET brain scan image into the trained first model to generate a third image, construct the first PET brain scan image and the third image into an image pair in a cascading manner, input the image pair into the second model, output a prediction result, and use the real disease label as supervision information, perform a third similarity constraint on the prediction result and the real disease label through cross-entropy loss, and classify the prediction result.

[0078] In this embodiment, after the first model is trained, the first PET brain scan image is input into the trained first model to generate a third image, and the third image is a final synthesis 11 C-CFT PET image, and the actual scan of the object 18 F-FDG PET image and the final synthesis 11 C-CFT PET image into an image pair, input the image pair into the second model, output a prediction result, and use the real disease label as supervision information, perform a third similarity constraint on the prediction result and the real disease label through cross-entropy loss, and classify the prediction result, which can be classified into healthy and sick categories, and doctors can analyze and judge whether the patient has the risk of Parkinson's disease through the prediction result of the sick category.

[0079] Step S9) Input step, input any first PET brain scan image into the trained first model and the trained second model, and output a classification result.

[0080] As Figure 8As shown, in step S9), when the first PET brain scan image is input into the trained first model, it specifically includes step S91), a downsampling step, step S92), a first feature extraction step, and step S93), an upsampling step.

[0081] Step S91) a downsampling step, downsampling the input first PET brain scan image to 1 / 8 of the original size through the encoder to obtain a first feature map;

[0082] Step S92) a first feature extraction step, extracting first image features of the first feature map through the bottleneck layer to obtain a second feature map;

[0083] Step S93) Upsampling step: upsampling the second feature map to the original size through the decoder to obtain the third image.

[0084] The residual modules of the first model all use the LeakyReLU activation function and instance normalization.

[0085] like Figure 9 As shown, in step S9), when the first PET brain scan image and the third image are input into the trained second model, it specifically includes step S94) an image pair construction step, step S95) a projection step, step S96) a second feature extraction step, step S97) a feature vector acquisition step, and step S98) a mapping step.

[0086] Step S94) An image pair construction step, constructing the first PET brain scan image and the third image into an image pair by cascading.

[0087] Step S95) A projection step is performed, wherein the image pair is input to the feature extractor, and the feature extractor projects the image pair through the convolution layer to reduce the data dimension of the image pair.

[0088] Step S96) A second feature extraction step is performed, in which the residual module is used to extract the second image feature of the image pair after reducing the data dimension.

[0089] Step S97) a feature vector acquisition step, based on a global average pooling method, obtaining a feature vector with a dimension of 512 through the second image feature;

[0090] Step S98) mapping step, performing a non-linear mapping on the feature vector through the full connection layer, outputting a two-dimensional vector, the two-dimensional vector being the prediction result, the first dimension of the two-dimensional vector representing the classification probability of the healthy control (NC), and the second dimension representing the classification probability of PD, the sum of the classification probability of the healthy control (NC) and the classification probability of Parkinson's disease (PD) being 1.

[0091] Using the real disease label as supervision information, the prediction result output by the second model is subjected to a third similarity constraint through cross-entropy loss and the real disease label, and the prediction result is classified into a healthy class and a sick class, thereby assisting doctors in medical diagnosis, and doctors can analyze whether the subject has Parkinson's disease through the prediction result of the sick class.

[0092] The second model uses a cosine annealing learning rate update algorithm and an Adam optimizer to update the weights of the second model; the convolutional layer and the residual module of the second model both use a ReLU activation function and batch normalization.

[0093] The present application provides a PET image transmembrane state synthesis method based on deep learning, which converts 18 F-FDG PET images into 11 C-CFT PET images, takes advantage of 11 C-CFT PET in the evaluation of the dopaminergic system, significantly improves the accuracy of the AI model in PD diagnosis, and can better help doctors determine whether the subject is sick; the synthesized 11 C-CFT PET images directly reflect the DAT level of the striatal region of PD patients, providing an intuitive visual basis for model diagnosis results, which helps doctors understand and trust the AI model's diagnosis decision; relying on common 18 F-FDG tracers, effectively overcoming the resource limitations of 11 C-CFT PET imaging, enabling precise PD automatic diagnosis technology to be implemented in a wider medical environment, especially for hospitals or medical institutions that have difficulty obtaining 11 C-CFT tracers, which helps doctors in hospitals or medical institutions that have difficulty obtaining 11 C-CFT tracers to diagnose PD.

[0094] The above describes in detail the PET image transmembrane state synthesis method based on deep learning provided by the embodiment of the present application, and the principle and implementation manner of the present application are described by applying specific examples; the above embodiment is only used to help understand the technical solutions and core ideas of the present application; those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified or some technical features can be replaced by equivalents; and the modification or replacement does not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A deep learning based PET image transmembrane state synthesis method, characterized in that, Comprising the following steps: an image acquisition step of acquiring two or more first PET brain scan images and two or more second PET brain scan images, the first PET brain scan images being first tracer PET brain scan images of a subject, the second PET brain scan images being second tracer PET brain scan images of a subject, the first PET brain scan images being actual scan 18 F-FDG PET images, the second PET brain scan images being actual scan 11 C-CFT PET images; A first model construction step, the first model adopts a 3D U-Net network structure, the 3D U-Net network structure comprises an encoder, a bottleneck layer, a decoder and a skip connection, the encoder comprises two or more residual modules and a down-sampling layer, the bottleneck layer comprises two or more residual modules, the structure of the decoder is symmetrical with the structure of the encoder, and the skip connection is used to cascade the encoder features and the decoder features of different scales; A second model construction step, the second model adopts a 3D ResNet-18 network structure, the 3D ResNet-18 network structure is composed of a feature extractor and a fully connected layer, and the feature extractor comprises two or more residual modules and a convolutional layer; A third model construction step, based on the training mode of Cycle GAN, optimizes the first model, constructs a third model with the same structure as the first model, and the third model can convert the actual scanned 11 C-CFT PET image into a synthetic 18 F-FDG PET image; A fourth model is constructed based on the training method of Patch GAN and the projection discriminator optimization of the first model, so that the fourth model can make the synthesized 11 C-CFT PET image content aligns with the actual scanned 11 C-CFT PET image data distribution aligns; A fifth model constructing step, based on the training manner of the Patch GAN and the projection discriminator optimization of the third model, constructs a fifth model, and the fifth model makes the synthesized 18 F-FDG PET image content align with the actual scanned 18 F-FDG PET image data distribution; a first training step, a first recurrent path and a second recurrent path are built, the first PET brain scan image is passed through the first recurrent path to generate a first image, the first image and the first PET brain scan image are based on a first similarity constraint by a cycle consistency loss of a loss function, the second PET brain scan image is passed through the second recurrent path to generate a second image, the second image and the second PET brain scan image are based on a second similarity constraint by a cycle consistency loss of a loss function; The first loop path refers to the actual scan of 18 F-FDG PET images into synthetic 11 C-CFT PET images, the synthetic 11 C-CFT PET images being converted back into synthetic 18 F-FDG PET images; the second loop path refers to the actual scan of 11 C-CFT PET images into synthetic 18 F-FDG PET images, the synthetic 18 F-FDG PET images being converted back into synthetic 11 C-CFT PET images; A second training step, inputting the first PET brain scan image into the trained first model to generate a third image, constructing an image pair by cascading the first PET brain scan image and the third image, inputting the image pair into the second model, outputting a prediction result, and using a real disease label as supervision information, performing a third similarity constraint on the prediction result and the real disease label through cross-entropy loss, and classifying the prediction result; And An input step of inputting any first PET brain scan image into the trained first model and the trained second model to output a classification result.

2. The deep learning based PET image transmural synthesis method of claim 1, wherein, The first training step comprises the following steps: A first image conversion step of inputting the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image; A first image reconstruction step of inputting the first synthetic PET brain scan image into the third model to generate a first image, forming the first cycle path; A fourth similarity constraint step of performing a fourth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image based on the adversarial loss of the hinge loss function through the fourth model; and a first similarity constraint step, based on a cycle consistency loss of the loss function, imposing a first similarity constraint on the first PET brain scan image and the first image. 3.The deep learning based PET image transmural synthesis method of claim 1, wherein, The first training step further comprises the following steps: A second image conversion step of inputting the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image; A second image reconstruction step of inputting the second synthetic PET brain scan image into the first model to reconstruct it into the second image, forming the second cycle path; A fifth similarity constraint step of performing a fifth similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image based on the adversarial loss of the hinge loss function through the fifth model; And a second similarity constraint step, based on a cycle consistency loss of the loss function, imposing a second similarity constraint on the second PET brain scan image and the second image. 4.The deep learning based PET image transmural synthesis method of claim 1, wherein, In the first training step, when the first PET brain scan image and the second PET brain scan image come from the same subject, comprising the following steps: A first image conversion step of inputting the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image; and A second image conversion step of inputting the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image; and a sixth similarity constraint step, based on a reconstruction loss of a loss function, a sixth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image.

5. The deep learning based PET image transmural synthesis method of claim 1, wherein, In the first training step, when the first PET brain scan image and the second PET brain scan image come from the same subject, further comprising the following steps: A second image conversion step of inputting the second PET brain scan image into the third model to convert into a second synthetic PET brain scan image; and a seventh similarity constraint step, based on a reconstruction loss of a loss function, a seventh similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image.

6. The deep learning based PET image transmural synthesis method of claim 1, wherein, In the input step, when the first PET brain scan image is input into the trained first model, specifically comprising the following steps: A downsampling step in which the encoder downsamples the input first PET brain scan image to 1 / 8 of the original size to obtain a first feature map; A first feature extraction step of extracting first image features of the first feature map through the bottleneck layer to obtain a second feature map; And An upsampling step of upsampling the second feature map to the original size through the decoder to obtain the third image.

7. The deep learning based PET image transmural synthesis method of claim 1, wherein, In the input step, when the first PET brain scan image and the third image are input into the trained second model, specifically comprising the following steps: An image pair construction step of constructing the first PET brain scan image and the third image into an image pair through cascading; A projection step of inputting the image pair into the feature extractor, which projects the image pair through the convolution layer; A second feature extraction step of extracting second image features of the image pair after reducing the data dimension through the residual module; A feature vector acquisition step of obtaining a feature vector with a dimension of 512 through the second image features based on the global average pooling method; and A mapping step of performing nonlinear mapping on the feature vector through the fully connected layer to output a two-dimensional vector, which is the prediction result.

8. The deep learning-based PET image transmembrane state synthesis method of claim 1, wherein The first model, the third model, the fourth model, and the fifth model all apply spectral normalization, and the Adam optimizer is used to update the weights of the first model, the weights of the third model, the weights of the fourth model, and the weights of the fifth model; The residual modules of the first model all use LeakyReLU activation functions and instance normalization.

9. The deep learning-based PET image transmembrane state synthesis method of claim 1, wherein The second model uses the cosine annealing learning rate update algorithm and the Adam optimizer to update the weights of the second model; The convolution layer and the residual module of the second model both use ReLU activation functions and batch normalization.

Images