PET image transmembrane state synthesis method based on deep learning

Abstract

The invention discloses a PET (Positron Emission Tomography) image transmembrane state synthesis method based on deep learning, which is suitable for DAT (Dopamine Transporter) imaging, improves the diagnosis accuracy of Parkinson's disease through an advanced image processing technology, and consists of two core parts, namely a 11C-CFT generation model and a PD (Parkinson's Disease) classification model. Wherein the < 11 > C-CFT generation model is responsible for converting an actually scanned < 18 > F-FDG PET image into a < 11 > C-CFT PET image through a cross-modal synthesis technology, and the PD classification model outputs a prediction result by utilizing multi-modal information of the real < 18 > F-FDG PET image and the synthesized < 11 > C-CFT PET image. The method has the advantages that the DAT level of a striatum area of a PD patient is directly reflected through the synthesized 11C-CFT PET image by utilizing the advantages of 11C-CFT PET in dopamine energy system evaluation, an intuitive visual basis is provided for a model diagnosis result, doctors are helped to understand and trust diagnosis decisions of an AI model, resource limitation of 11C-CFT PET imaging is effectively overcome, and the method is suitable for popularization and application. Therefore, the accurate PD automatic diagnosis technology can be implemented in a wider medical environment.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to a PET image transmembrane state synthesis method based on deep learning. Background Art

[0002] Parkinson's disease (PD) is a common progressive neurodegenerative disease characterized by the gradual degeneration of dopaminergic neurons in the substantia nigra pars compacta. This neuronal degeneration mainly affects the part of the brain that controls movement, causing a series of typical motor symptoms, such as tremor at rest, muscle rigidity, bradykinesia, and postural balance disorders. In addition, Parkinson's disease may also be accompanied by non-motor symptoms, including cognitive impairment, affective disorders, sleep disorders, and autonomic dysfunction. The diagnosis of Parkinson's disease is particularly difficult in the early stages because its symptoms are similar to those of other neurodegenerative diseases, and the early symptoms may be mild or atypical. At present, clinical diagnosis is usually made after the onset of symptoms of movement disorders, and the diagnosis process often takes time, resulting in delayed diagnosis and delayed treatment. Therefore, the development of early diagnostic tools is crucial to improving the prognosis of patients with Parkinson's disease.

[0003] Positron emission tomography (PET) is a nuclear medicine imaging technique that provides information about the internal chemical processes and physiological functions of an organism by injecting radioactive tracers into the patient's body and circulating through the blood to specific organs or tissues. Among them, dopamine transporter (DAT) imaging can be performed by specific radioactive tracers (such as 11 C-CFT) reflects the functional state of dopaminergic neurons, thus becoming an important tool for diagnosing Parkinson's disease. 11 In C-CFT PET imaging, the DAT level in the striatum of the healthy control (NC) group was normal, while that of PD patients was significantly reduced. 11 Normal DAT levels shown by C-CFT PET imaging can be used as an absolute exclusion criterion for Parkinson's disease.

[0004] However, it is difficult for most hospitals or medical institutions to obtain 11 C-CFT tracer can only be used with fluorodeoxyglucose ( 18 F-FDG) is used as a tracer in PET imaging for PD diagnosis. 18 F-FDG is the most widely used radioactive tracer in the field of neuroimaging, which can indirectly reflect the activity of nerve cells by evaluating the level of glucose metabolism in the brain. 18In F-FDG PET imaging, glucose metabolism levels in the basal ganglia of PD patients are often increased, which is related to compensatory metabolic changes caused by dopaminergic neuron loss. However, since the functional status of dopaminergic neurons cannot be directly assessed, 18 F-FDG PET imaging has difficulty distinguishing PD from other neurodegenerative diseases, especially in the early stages of the disease when metabolic changes are not significant enough, making differential diagnosis particularly difficult.

[0005] The patent document (CN114862774A) discloses a cross-modal reconstruction method of PET images based on deep learning. The method constructs a neural network to constrain and modify the generated images, thereby obtaining high-quality AV45-PET brain scan images. This method is only suitable for observing the cortical function of Alzheimer's disease and is not suitable for subcortical dopaminergic function examination. Summary of the invention

[0006] The present invention provides a deep learning-based PET image transmembrane state synthesis method to solve the problem that most hospitals or medical institutions are difficult to obtain 11 C-CFT tracer, using fluorodeoxyglucose ( 18 When F-FDG is used as a tracer in PET imaging for PD diagnosis, 8 F-FDG PET imaging has the problem of difficulty distinguishing PD from other neurodegenerative diseases.

[0007] In order to solve the above technical problems, the present invention discloses the following technical solutions:

[0008] The present invention provides a PET image transmembrane state synthesis method based on deep learning, comprising an image acquisition step, a first model construction step, a second model construction step, a third model construction step, a fourth model construction step, a fifth model construction step, a first training step, a second training step and an input step.

[0009] The image acquisition step is used to acquire more than two first PET brain scan images and more than two second PET brain scan images, wherein the first PET brain scan image is a first tracer PET brain scan image of a subject, and the second PET brain scan image is a second tracer PET brain scan image of a subject; in the first model construction step, the first model adopts a 3D U-Net network structure, wherein the 3D U-Net network structure includes an encoder, a bottleneck layer, a decoder, and a jump connection, wherein the encoder includes more than two residual modules and a downsampling layer, wherein the bottleneck layer includes more than two residual modules, wherein the structure of the decoder is symmetrical to that of the encoder, and the jump connection is used to cascade encoder features and decoder features of different scales; in the second model construction step, the second model adopts a 3D ResNet-18 network structure, wherein the 3D ResNet-18 network structure is composed of a feature extractor and a fully connected layer, wherein the feature extractor includes more than two residual modules and a convolutional layer; the third model construction step is to optimize the first model based on the training method of Cycle GAN, and construct a third model with the same structure as the first model; the fourth model construction step is to optimize the first model based on the training method of Patch GAN, and construct a third model with the same structure as the first model; The training method of GAN and the projection discriminator optimize the first model to construct a fourth model; the fifth model construction step is to optimize the third model based on the training method of Patch GAN and the projection discriminator to construct a fifth model; the first training step is used to build a first loop path and a second loop path, the first PET brain scan image generates a first image through the first loop path, the first image and the first PET brain scan image are based on l 1 The cycle consistency loss of the loss function performs a first similarity constraint, the second PET brain scan image passes through the second cycle path to generate a second image, and the second image and the second PET brain scan image are based on l 1 The cycle consistency loss of the loss function is used to perform a second similarity constraint; the second training step is to input the first PET brain scan image into the trained first model to generate a third image, construct the first PET brain scan image and the third image into an image pair in a cascade manner, input the image pair into the second model, output the prediction result, and use the real disease label as supervision information, perform a third similarity constraint on the prediction result and the real disease label through cross entropy loss, and classify the prediction result; the input step is to input any of the first PET brain scan images into the trained first model and the trained second model, and output the classification result.

[0010] Furthermore, the first training step includes a first image conversion step, a first image reconstruction step, a fourth similarity constraint step and a first similarity constraint step.

[0011] The first image conversion step is to input the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image; the first image reconstruction step is to input the first synthetic PET brain scan image into the third model to generate a first image, forming the first loop path; the fourth similarity constraint step is to perform a fourth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image based on the adversarial loss of the hinge loss function through the fourth model; the first similarity constraint step is based on l 1 The cycle consistency loss of the loss function imposes a first similarity constraint on the first PET brain scan image and the first image.

[0012] Furthermore, the first training step also includes a second image conversion step, a second image reconstruction step, a fifth similarity constraint step and a second similarity constraint step.

[0013] The second image conversion step is to input the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image; the second image reconstruction step is to input the second synthetic PET brain scan image into the first model to reconstruct it into the second image, forming the second loop path; the fifth similarity constraint step is to perform a fifth similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image based on the adversarial loss of the hinge loss function through the fifth model; the second similarity constraint step is based on l 1 The cycle consistency loss of the loss function performs a second similarity constraint on the second PET brain scan image and the second image.

[0014] Furthermore, in the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, a first image conversion step and a sixth similarity constraint step are included.

[0015] The first image conversion step is to input the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image; the sixth similarity constraint step is based on l 1 The reconstruction loss of the loss function performs a sixth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image.

[0016] Furthermore, in the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, a second image conversion step and a seventh similarity constraint step are also included.

[0017] The second image conversion step is to input the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image; the seventh similarity constraint step is based on l 1 The reconstruction loss of the loss function performs a seventh similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image.

[0018] Furthermore, in the input step, when the first PET brain scan image is input into the trained first model, it specifically includes a downsampling step, a first feature extraction step and an upsampling step.

[0019] The downsampling step is to downsample the input first PET brain scan image to 1 / 8 of the original size through the encoder to obtain a first feature map; the first feature extraction step is to extract the first image feature of the first feature map through the bottleneck layer to obtain a second feature map; the upsampling step is to upsample the second feature map to the original size through the decoder to obtain the third image.

[0020] Furthermore, in the input step, when the first PET brain scan image and the third image are input into the trained second model, it specifically includes an image pair construction step, a projection step, a second feature extraction step, a feature vector acquisition step and a mapping step.

[0021] The image pair construction step is to construct the first PET brain scan image and the third image into an image pair by cascading; the projection step is to input the image pair to the feature extractor, and the feature extractor projects the image pair through the convolution layer; the second feature extraction step is to extract the second image feature of the image pair after reducing the data dimension through the residual module; the feature vector acquisition step is based on the global average pooling method, and a feature vector with a dimension of 512 is obtained through the second image feature; the mapping step is to perform nonlinear mapping on the feature vector through the fully connected layer, and output a two-dimensional vector, which is the prediction result.

[0022] Furthermore, spectral normalization is applied to the first model, the third model, the fourth model and the fifth model, and the Adam optimizer is used to update the weights of the first model, the third model, the fourth model and the fifth model; the residual modules of the first model all use the LeakyReLU activation function and instance normalization.

[0023] Furthermore, the second model uses a cosine annealing learning rate update algorithm and an Adam optimizer to update the weights of the second model; the convolutional layer and the residual module of the second model both use a ReLU activation function and batch normalization.

[0024] Compared with the prior art, the present invention has at least the following technical effects:

[0025] The present invention provides a PET image transmembrane state synthesis method based on deep learning. 18 Conversion of F-FDG PET images to 11 C-CFT PET images, using 11 The advantages of C-CFT PET in the evaluation of the dopaminergic system significantly improve the accuracy of the artificial intelligence model in diagnosing PD, and can better help doctors determine whether the subject is sick; 11 C-CFT PET images directly reflect the DAT level in the striatum of PD patients, providing an intuitive visual basis for the model's diagnostic results, helping doctors understand and trust the diagnostic decisions of the AI ​​model; based on common 18 F-FDG tracer effectively overcomes 11 The resource limitations of C-CFT PET imaging make it possible to implement accurate automated PD diagnosis technology in a wider range of medical settings, especially for patients with limited access to 11 C-CFT tracer is helpful for hospitals or medical institutions that are difficult to obtain 11 Doctors in hospitals or medical institutions using C-CFT tracers go to diagnose PD. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0027] Figure 1 A schematic diagram of a deep learning-based PET image transmembrane state synthesis method according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of the first training step and the second training step in an embodiment of the present invention;

[0029] Figure 3This is a flow chart of a method for transmembrane synthesis of PET images based on deep learning according to an embodiment of the present invention;

[0030] Figure 4 The process of the first training step described in the embodiment of the present invention is Figure 1 ;

[0031] Figure 5 The process of the first training step described in the embodiment of the present invention is Figure 2 ;

[0032] Figure 6 The process of the first training step described in the embodiment of the present invention is Figure 3 ;

[0033] Figure 7 The process of the first training step described in the embodiment of the present invention is Figure 4 ;

[0034] Figure 8 The process of inputting steps described in the embodiment of the present invention is as follows Figure 1 ;

[0035] Fig. 9 The process of inputting steps described in the embodiment of the present invention is as follows Figure 2 . DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0037] like Figure 1 - Figure 3 As shown, the present invention provides a PET image transmembrane state synthesis method based on deep learning, which is suitable for dopamine transporter (DAT) imaging, and improves the diagnostic accuracy of Parkinson's disease through advanced image processing technology. 11 The C-CFT generation model and the PD classification model are the two core parts. 11 The C-CFT generation model is responsible for converting the actual scan 18 F-FDG PET images were converted into 11 C-CFTPET images, while the PD classification model uses the real 18 F-FDG PET images and synthesis 11The multimodal information of C-CFT PET images is used to improve the accuracy of PD judgment by the artificial intelligence model, which can better help doctors analyze whether the subject is sick. The present invention includes step S1) image acquisition step, step S2) first model construction step, step S3) second model construction step, step S4) third model construction step, step S5) fourth model construction step, step S6) fifth model construction step, step S7) first training step, step S8) second training step and step S9) input step.

[0038] Step S1) an image acquisition step, acquiring two or more first PET brain scan images and two or more second PET brain scan images, wherein the first PET brain scan image is a first tracer PET brain scan image of a subject, and the second PET brain scan image is a second tracer PET brain scan image of a subject.

[0039] In this embodiment, the first PET brain scan image is the actual scanned 18 F-FDG PET image, the second PET brain scan image is the actual scan 11 C-CFT PET images, actual scans of each subject 18 F-FDG PET images have corresponding actual scans 11 C-CFT PET image, the first tracer is 18 F-FDG tracer, the second tracer is 11 C-CFT tracer.

[0040] Step S2) A first model construction step, wherein the first model adopts a 3D U-Net network structure, wherein the 3D U-Net network structure includes an encoder, a bottleneck layer, a decoder, and a jump connection, wherein the encoder includes more than two residual modules and a downsampling layer, and the bottleneck layer includes more than two residual modules. The structure of the decoder is symmetrical to that of the encoder, and the jump connection is used to cascade encoder features and decoder features of different scales to promote information flow.

[0041] In this embodiment, the first model is 11 C-CFT generates models that can convert the actual scanned 18 Conversion of F-FDG PET images to their synthetic counterparts 11 C-CFT PET image.

[0042] Step S3) A second model construction step, wherein the second model adopts a 3D ResNet-18 network structure, and the 3DResNet-18 network structure consists of a feature extractor and a fully connected layer, and the feature extractor includes more than two residual modules and convolutional layers.

[0043] In this embodiment, the second model is a PD classification model that can integrate the actual scan 18 F-FDG PET images and synthesis 11 The multimodal information of C-CFT PET images is processed and analyzed, and the output results are divided into two categories: healthy and diseased. This improves the accuracy of the artificial intelligence model in PD diagnosis and can better help doctors determine whether the subjects are sick.

[0044] Step S4) A third model construction step is performed, wherein the first model is optimized based on the training method of Cycle GAN, and a third model having the same structure as the first model is constructed.

[0045] In this embodiment, the third model is 18 F-FDG generates a model that can convert the actual scanned 11 C-CFT PET image conversion to synthesis 18 F-FDG PET images.

[0046] Step S5) A fourth model construction step is performed, in which the first model is optimized based on the training method of Patch GAN and the projection discriminator to construct a fourth model.

[0047] In this embodiment, the fourth model is 11 C-CFT discriminant model, making synthesis 11 C-CFT PET image content and actual scan 11 Data distribution alignment of C-CFT PET images.

[0048] Step S6) A fifth model construction step is performed, in which the third model is optimized based on the training method of Patch GAN and the projection discriminator to construct a fifth model.

[0049] In this embodiment, the fifth model is 18 F-FDG discrimination model, making the synthesis 18 F-FDG PET image content and actual scan 18 Data distribution alignment of F-FDG PET images.

[0050] Step S7) A first training step is to build a first loop path and a second loop path, wherein the first PET brain scan image generates a first image through the first loop path, and the first image and the first PET brain scan image are based on l 1 The cycle consistency loss of the loss function performs a first similarity constraint, the second PET brain scan image passes through the second cycle path to generate a second image, and the second image and the second PET brain scan image are based on l 1 The cycle consistency loss of the loss function performs the second similarity constraint.

[0051] In this embodiment, the first circulation path is a positive circulation path ( 18 F-FDG→ 11 C-CFTv 18 F-FDG), the second circulation path is a reverse circulation path ( 11 C-CFT→ 18 F-FDG→ 11 C-CFT), the first cycle path refers to the actual scanning 18 Conversion of F-FDG PET images into synthetic 11 C-CFT PET image, the synthesis 11 C-CFT PET images are then converted to synthetic 18 F-FDG PET images, from actual scans 18 Conversion of F-FDG PET images to synthetic 18 F-FDG PET imaging can be understood as an image reconstruction process. The second cycle path refers to the actual scanning process. 11 C-CFT PET images converted to synthetic 18 F-FDG PET image, the synthesis 18 F-FDG PET images are then converted to synthetic 11 C-CFT PET images, from actual scans 11 C-CFT PET image conversion to synthesis 11 C-CFT PET images can also be understood as an image reconstruction process.

[0052] Cycle consistency loss means that the two A at the beginning and end of each cycle path A-->B-->A must be consistent, that is, the positive cycle path ( 18 F-FDG→ 11 C-CFT→ 18 F-FDG) head and tail 18 The F-FDG should be consistent with each other, and the reverse circulation path ( 11 C-CFT→ 18 F-FDG→11 C-CFT) head and tail 11 Maintain consistency between C-CFTs.

[0053] like Figure 4 As shown, step S7) the first training step includes step S711) a first image conversion step, step S712) a first image reconstruction step, step S713) a fourth similarity constraint step and step S714) a first similarity constraint step.

[0054] Step S711) A first image conversion step, inputting the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image, that is, the actual scanned 18 The F-FDG PET image is input to the first model and converted into the first synthetic 11 C-CFT PET image.

[0055] Step S712) A first image reconstruction step, inputting the first synthesized PET brain scan image into the third model to generate a first image, that is, the first synthesized 11 The C-CFT PET image is input to the third model and converted into the first synthesis 18 F-FDG PET images, forming the first circulation path, i.e., the positive circulation path ( 18 F-FDG→ 11 C-CFT→ 18 F-FDG).

[0056] Step S713) A fourth similarity constraint step is performed on the first synthetic PET brain scan image and the second PET brain scan image through the fourth model based on the adversarial loss of the hinge loss function, that is, the first synthetic 11 C-CFT PET images and actual scans 11 C-CFT PET images are subjected to the fourth similarity constraint to improve the first synthesis 11 Accuracy of C-CFT PET images.

[0057] In this embodiment, the adversarial loss of the hinge loss function is known and the objects with similarity constraints are first synthesized 11 C-CFT PET images and actual scans 11 How to synthesize the first C-CFT PET image when it is known 11 C-CFTPET images and actual scans 11 Performing the fourth similarity constraint on the C-CFT PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the fourth similarity constraint is not described again in this embodiment.

[0058] Step S714) The first similarity constraint step is based on l 1 The cycle consistency loss of the loss function performs a first similarity constraint on the first PET brain scan image and the first image to improve the accuracy of the first image.

[0059] In this embodiment, 1 The cycle consistency loss of the loss function is known to be the actual scan of the object with similarity constraints 18 F-FDG PET images and first synthesis 18 How to compare the actual scanned F-FDG PET images when they are known? 18 F-FDG PET images and first synthesis 18 Performing the first similarity constraint on the F-FDG PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the first similarity constraint is not described again in this embodiment.

[0060] like Figure 5 As shown, step S7) the first training step also includes step S721) a second image conversion step, step S722) a second image reconstruction step, step S723) a fifth similarity constraint step and step S724) a second similarity constraint step.

[0061] Step S721) A second image conversion step, inputting the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image, that is, the actual scanned 11 C-CFT PET images are input to the third model and converted into the second synthesis 18 F-FDG PET images.

[0062] Step S722) A second image reconstruction step, inputting the second synthesized PET brain scan image into the first model to reconstruct the second image, that is, the second synthesized 18 F-FDG PET images are input to the first model and converted into the second synthesis 18 F-FDG PET images form the second circulation pathway, namely the reverse circulation pathway ( 11 C-CFT→ 18 F-FDG→ 11 C-CFT).

[0063] Step S723) A fifth similarity constraint step is performed on the second synthetic PET brain scan image and the first PET brain scan image through the fifth model based on the adversarial loss of the hinge loss function to improve the accuracy of the second synthetic PET brain scan image.

[0064] In this embodiment, the adversarial loss of the hinge loss function is known and the object with similarity constraint is synthesized 18 F-FDG PET images and actual scans 18 How to synthesize the second F-FDG PET image when it is known 18 F-FDGPET images and actual scans 18 Performing the fourth similarity constraint on the F-FDG PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the fifth similarity constraint is not described again in this embodiment.

[0065] Step S724) The second similarity constraint step is based on l 1 The cycle consistency loss of the loss function performs a second similarity constraint on the second PET brain scan image and the second image to improve the accuracy of the second image.

[0066] In this embodiment, 1 The cycle consistency loss of the loss function is known to be the actual scan of the object with similarity constraints 11 C-CFT PET images and second synthesis 11 How to compare the actual scanned image when the C-CFT PET image is known? 11 C-CFT PET images and second synthesis 11 Performing the first similarity constraint on the C-CFT PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the second similarity constraint is not described again in this embodiment.

[0067] like Figure 6 As shown, in the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, that is, the actual scan 18 F-FDG PET images and actual scans 11 When the C-CFT PET images are from the same patient, the method includes step S711) a first image conversion step and step S715) a sixth similarity constraint step.

[0068] Step S711) A first image conversion step, inputting the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image, that is, the actual scanned 18 The F-FDG PET image is input to the first model and converted into the first synthetic 11 C-CFT PET image.

[0069] Step S715) The sixth similarity constraint step is based on l 1 The reconstruction loss of the loss function performs a sixth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image.

[0070] In this embodiment, 1 The reconstruction loss of the loss function is known to be the actual scan of the object with similarity constraints 11 C-CFT PET images and first synthesis 11 How to compare the actual scanned image when the C-CFT PET image is known? 11 C-CFTPET images and first synthesis 11 Performing the first similarity constraint on the C-CFT PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the sixth similarity constraint is not described again in this embodiment.

[0071] Reconstruction loss means that if a subject has paired A and B, then for cycle path 1 (A-->B-->A) and cycle path 2 (B-->A-->B), similarity constraints are directly applied to the input A of path 1 and the intermediate synthesis A of path 2, and similarity constraints are applied between the input B of path 2 and the intermediate synthesis B of path 1, that is, for the positive cycle path ( 18 F-FDG→ 11 C-CFT→ 18 F-FDG 11 C-CFT and reverse cycle path ( 11 C-CFT→ 18 F-FDG→ 11 Input in C-CFT 11 C-CFT performs similarity constraints on positive cycle paths ( 18 F-FDG→ 11 C-CFT→ 18 Input in F-FDG 18 F-FDG and reverse circulation pathway ( 11 C-CFT→ 18 F-FDGv 11 Input in C-CFT 18 F-FDG performs similarity constraints.

[0072] like Figure 7 As shown, in the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, it also includes step S721) a second image conversion step and step S725) a seventh similarity constraint step.

[0073] Step S721) A second image conversion step, inputting the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image, that is, the actual scanned 11 C-CFT PET images are input to the third model and converted into the second synthesis 18 F-FDG PET images.

[0074] The seventh similarity constraint step is based on l 1 The reconstruction loss of the loss function performs a seventh similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image.

[0075] In this embodiment, 1 The reconstruction loss of the loss function is known to be the actual scan of the object with similarity constraints 18 F-FDGPET images and second synthesis 18 How to compare the actual scanned F-FDG PET images when they are known? 18 F-FDGPET images and second synthesis 18 Performing the first similarity constraint on the F-FDG PET image is a calculation process that can be easily implemented by those skilled in the art, so the calculation process of the seventh similarity constraint is not described again in this embodiment.

[0076] The first model, the third model, the fourth model and the fifth model all apply spectral normalization, and use the Adam optimizer to update the weights of the first model, the third model, the fourth model and the fifth model.

[0077] Step S8) In the second training step, the first PET brain scan image is input into the trained first model to generate a third image, the first PET brain scan image and the third image are constructed into an image pair by cascading, the image pair is input into the second model, the prediction result is output, and the real disease label is used as supervision information, and a third similarity constraint is performed on the predicted result and the real disease label through cross entropy loss to classify the predicted result.

[0078] In this embodiment, after the first model is trained, the first PET brain scan image is input into the trained first model to generate a third image, and the third image is the final synthesis image. 11 C-CFT PET image, and then the actual scan 18 The final synthesis of F-FDG PET images and the 11The C-CFT PET images are constructed into image pairs, which are input into the second model to output prediction results. The real disease labels are used as supervisory information, and the prediction results are classified by a third similarity constraint between the prediction results and the real disease labels through cross entropy loss. The prediction results can be divided into healthy and diseased categories. Doctors can analyze and determine whether patients are at risk of Parkinson's disease through the prediction results of the diseased category.

[0079] Step S9) An input step is performed to input any of the first PET brain scan images into the trained first model and the trained second model, and output a classification result.

[0080] like Figure 8 As shown, in step S9), when the first PET brain scan image is input into the trained first model, it specifically includes step S91), a down-sampling step, step S92), a first feature extraction step and step S93), an up-sampling step.

[0081] Step S91) a downsampling step, downsampling the input first PET brain scan image to 1 / 8 of the original size through the encoder to obtain a first feature map;

[0082] Step S92) a first feature extraction step, extracting the first image feature of the first feature map through the bottleneck layer to obtain a second feature map;

[0083] Step S93) Upsampling step: upsampling the second feature map to the original size through the decoder to obtain the third image.

[0084] The residual modules of the first model all use LeakyReLU activation function and instance normalization.

[0085] like Fig. 9 As shown, in step S9), when the first PET brain scan image and the third image are input into the trained second model, it specifically includes step S94), an image pair construction step, step S95), a projection step, step S96), a second feature extraction step, step S97), a feature vector acquisition step and step S98), a mapping step.

[0086] Step S94) An image pair construction step, in which the first PET brain scan image and the third image are constructed into an image pair by cascading.

[0087] Step S95) A projection step is performed, wherein the image pair is input to the feature extractor, and the feature extractor projects the image pair through the convolution layer to reduce the data dimension of the image pair.

[0088] Step S96) A second feature extraction step is to extract the second image feature of the image pair after reducing the data dimension through the residual module.

[0089] Step S97) a feature vector acquisition step, based on a global average pooling method, a feature vector with a dimension of 512 is obtained through the second image feature;

[0090] Step S98) A mapping step is performed, in which the feature vector is nonlinearly mapped through the fully connected layer to output a two-dimensional vector, where the two-dimensional vector is the prediction result. The numbers in the first dimension of the two-dimensional vector represent the classification probability of a healthy control (NC), and the numbers in the second dimension represent the classification probability of PD. The sum of the classification probability of a healthy control (NC) and the classification probability of Parkinson's disease (PD) is 1.

[0091] By using the real disease label as supervisory information and performing a third similarity constraint on the prediction result output by the second model and the real disease label through cross entropy loss, the prediction result is classified. The prediction result can be divided into healthy class and diseased class, thereby assisting doctors in medical diagnosis. Doctors can analyze whether the subject has Parkinson's disease through the prediction result of the diseased class.

[0092] The second model uses a cosine annealing learning rate update algorithm and an Adam optimizer to update the weights of the second model; the convolutional layer and the residual module of the second model both use a ReLU activation function and batch normalization.

[0093] This application provides a PET image transmembrane state synthesis method based on deep learning. 18 Conversion of F-FDG PET images to 11 C-CFT PET images, using 11 The advantages of C-CFT PET in the evaluation of the dopaminergic system significantly improve the accuracy of the artificial intelligence model in diagnosing PD, and can better help doctors determine whether the subject is sick; 11 C-CFT PET images directly reflect the DAT level in the striatum of PD patients, providing an intuitive visual basis for the model's diagnostic results, helping doctors understand and trust the diagnostic decisions of the AI ​​model; based on common 18 F-FDG tracer effectively overcomes 11 The resource limitations of C-CFT PET imaging make it possible to implement accurate automated PD diagnosis technology in a wider range of medical settings, especially for patients with limited access to 11 C-CFT tracer is helpful for hospitals or medical institutions that are difficult to obtain 11 Doctors in hospitals or medical institutions using C-CFT tracers go to diagnose PD.

[0094] The above is a detailed introduction to a deep learning-based transmembrane synthesis method for PET images provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the technical solutions and core ideas of the present invention. Ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A PET image transmembrane state synthesis method based on deep learning, characterized in that: The steps include: an image acquisition step, acquiring two or more first PET brain scan images and two or more second PET brain scan images, wherein the first PET brain scan images are first tracer PET brain scan images of a subject, and the second PET brain scan images are second tracer PET brain scan images of a subject; A first model construction step, wherein the first model adopts a 3D U-Net network structure, wherein the 3D U-Net network structure includes an encoder, a bottleneck layer, a decoder, and a jump connection, wherein the encoder includes more than two residual modules and a downsampling layer, wherein the bottleneck layer includes more than two residual modules, wherein the structure of the decoder is symmetrical to that of the encoder, and the jump connection is used to cascade encoder features and decoder features of different scales; A second model building step, wherein the second model adopts a 3D ResNet-18 network structure, wherein the 3D ResNet-18 network structure is composed of a feature extractor and a fully connected layer, wherein the feature extractor includes more than two residual modules and convolutional layers; A third model construction step, optimizing the first model based on a Cycle GAN training method, and constructing a third model with the same structure as the first model; A fourth model construction step is to optimize the first model based on the training method of Patch GAN and the projection discriminator to construct a fourth model; A fifth model construction step, optimizing the third model based on the training method of Patch GAN and the projection discriminator to construct a fifth model; In a first training step, a first loop path and a second loop path are constructed, wherein the first PET brain scan image generates a first image through the first loop path, and the first image and the first PET brain scan image are based on The cycle consistency loss of the loss function performs a first similarity constraint, the second PET brain scan image passes through the second cycle path to generate a second image, the second image and the second PET brain scan image are based on The cycle consistency loss of the loss function performs the second similarity constraint; a second training step, inputting the first PET brain scan image into the trained first model to generate a third image, constructing the first PET brain scan image and the third image into an image pair in a cascade manner, inputting the image pair into the second model, outputting a prediction result, and using the true disease label as supervision information, performing a third similarity constraint on the prediction result and the true disease label through a cross entropy loss, and classifying the prediction result; as well as The input step is to input any of the first PET brain scan images into the trained first model and the trained second model, and output the classification result.

2. The method for synthesizing PET images across membrane states based on deep learning according to claim 1, characterized in that: The first training step comprises the following steps: a first image conversion step, inputting the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image; a first image reconstruction step, inputting the first synthetic PET brain scan image into the third model to generate a first image, forming the first cyclic path; a fourth similarity constraint step, performing a fourth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image based on an adversarial loss of a hinge loss function through the fourth model; and The first similarity constraint step is based on The cycle consistency loss of the loss function imposes a first similarity constraint on the first PET brain scan image and the first image.

3. The method for transmembrane synthesis of PET images based on deep learning according to claim 1, characterized in that: The first training step also includes the following steps: a second image conversion step, inputting the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image; a second image reconstruction step, inputting the second synthetic PET brain scan image into the first model to reconstruct the second image, thereby forming a second cyclic path; a fifth similarity constraint step, performing a fifth similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image based on an adversarial loss of a hinge loss function through the fifth model; as well as The second similarity constraint step is based on The cycle consistency loss of the loss function performs a second similarity constraint on the second PET brain scan image and the second image.

4. The method for synthesizing PET images across membrane states based on deep learning according to claim 1, characterized in that: In the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, the following steps are included: a first image conversion step, inputting the first PET brain scan image into the first model to convert it into a first synthetic PET brain scan image; and The sixth similarity constraint step is based on The reconstruction loss of the loss function performs a sixth similarity constraint on the first synthetic PET brain scan image and the second PET brain scan image.

5. The method for synthesizing PET images across membrane states based on deep learning according to claim 1, characterized in that: In the first training step, when the first PET brain scan image and the second PET brain scan image are from the same subject, the following steps are also included: a second image conversion step, inputting the second PET brain scan image into the third model to convert it into a second synthetic PET brain scan image; and The seventh similarity constraint step is based on The reconstruction loss of the loss function performs a seventh similarity constraint on the second synthetic PET brain scan image and the first PET brain scan image.

6. The method for synthesizing PET images across membrane states based on deep learning according to claim 1, characterized in that: In the input step, when the first PET brain scan image is input into the trained first model, the following steps are specifically included: A downsampling step, wherein the encoder downsamples the input first PET brain scan image to 1 / 8 of the original size to obtain a first feature map; A first feature extraction step, extracting a first image feature of the first feature map through the bottleneck layer to obtain a second feature map; as well as The upsampling step upsamples the second feature map to the original size through the decoder to obtain the third image.

7. The method for transmembrane synthesis of PET images based on deep learning according to claim 1, characterized in that: In the input step, when the first PET brain scan image and the third image are input into the trained second model, the following steps are specifically included: an image pair construction step of constructing the first PET brain scan image and the third image into an image pair by cascading; A projection step, inputting the image pair to the feature extractor, and the feature extractor projects the image pair through the convolutional layer; A second feature extraction step, extracting a second image feature of the image pair after reducing the data dimension through the residual module; A feature vector acquisition step, based on a global average pooling method, obtaining a feature vector with a dimension of 512 through the second image feature; and In the mapping step, the fully connected layer performs nonlinear mapping on the feature vector and outputs a two-dimensional vector, which is the prediction result.

8. The method for synthesizing PET images across membrane states based on deep learning according to claim 1, characterized in that: The first model, the third model, the fourth model and the fifth model all apply spectral normalization, and use the Adam optimizer to update the weights of the first model, the third model, the fourth model and the fifth model; The residual modules of the first model all use LeakyReLU activation function and instance normalization.

9. The method for synthesizing PET images across membrane states based on deep learning according to claim 1, characterized in that: The second model uses a cosine annealing learning rate update algorithm and an Adam optimizer to update the weight of the second model; The convolutional layers and residual modules of the second model both use ReLU activation function and batch normalization.

Images