Metabolic abnormality detection method and device based on cross-modal generation, terminal, medium and program product

Abstract

The invention provides a metabolic abnormality detection method and device based on cross-modal generation, a terminal, a medium and a program product, and the method comprises the steps: carrying out the detection of the metabolic abnormality based on an obtained real 18F-FDG positron emission tomography image of the head of a subject through a trained dominant cross-modal image conversion model, a pseudo normal 18F-FDG positron emission tomography image of the subject is reconstructed; according to the real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image obtained through reconstruction, obtaining a residual image through a mode of calculating the intensity difference between the real 18F-FDG positron emission tomography image and the pseudo-normal 18F-FDG positron emission tomography image; and on the basis of the residual image, performing metabolic abnormality detection on the subject by using a set abnormality evaluation index. The head 18F-FDG positron emission tomography image in the three-dimensional space is processed by using the trained dominant cross-modal image conversion model, the residual image is obtained, and the abnormal metabolism mode of the brain of the subject is visually displayed, so that a doctor is assisted to carry out efficient and accurate diagnosis work.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to a metabolic abnormality detection method, device, terminal, medium and program product based on cross-modal generation. Background Art

[0002] Positron emission tomography (PET), as an excellent imaging method, is particularly prominent in the diagnosis and treatment of neurological diseases, especially neurodegenerative diseases. Take Parkinson's disease (PD) as an example. This disease is a common neurodegenerative disease that mainly affects the body's motor ability and is extremely harmful. This disease usually has a long incubation period and is often difficult to diagnose clinically, which makes it very easy for patients to miss the best time for treatment. Therefore, accurate diagnosis of Parkinson's is crucial for timely and correct therapeutic intervention. Using PET technology to image the dopamine transporter (DAT), for example 11 C-CFT PET imaging can reveal the functional changes of dopaminergic neurons in the caudate nucleus and putamen regions of the brain, and can accurately identify Parkinson's patients with high specificity. It has been successfully used in clinical practice.

[0003] However, due to equipment and technology limitations, most hospitals are currently unable to implement 11 C-CFT PET imaging, while 18 F-fluorodeoxyglucose ( 18 F-FDG is easy to obtain, so most hospitals can routinely perform 18 F-FDG PET imaging. 18 F-FDG PET imaging is a relatively low-cost and difficult technique, but it lacks specificity for diagnosing Parkinson's disease. 18 F-FDG PET images also have difficulty distinguishing subtle differences between Parkinson's patients and normal controls. 18 F-FDG PET for PD diagnosis is not only extremely challenging for doctors, but also limits computer-aided diagnosis (CAD) based on deep learning.

[0004] To address the above challenges, a feasible solution is to capture 18F-FDG PET images can be used to detect and amplify metabolic abnormalities associated with diseases, and to display abnormal areas in a more intuitive way, thereby assisting clinicians in making efficient polarity diagnoses or enhancing the accuracy and interpretability of CAD systems. Technically, this can be achieved using the idea of ​​anomaly detection. 18 Starting from the F-FDG PET image, several transformations are performed to obtain a reconstruction of the input image. Assuming that the above transformations are designed for normal controls (e.g., image data collected from normal people rather than diseased people), then the image data from patients 18 F-FDG PET will gradually lose disease-related information during the transformation, and there will be a large residual between the reconstructed image and the original input image. On the contrary, if the input image is from a normal control rather than a patient, the reconstructed image will be highly similar to the input image, that is, the residual is small. 18 The residual between the F-FDGPET image and its corresponding reconstructed image can be used to determine whether the input image is related to neurodegenerative diseases such as Parkinson's disease, that is, whether there is an "abnormality".

[0005] In the above process, the core challenge is to construct a transformation for the image and ensure that it is designed for "normal" (non-patient) images. However, there are two difficulties in completing this task: (1) Difficulty in optimizing the latent space: Generative models such as autoencoders (AEs) and generative adversarial networks (GANs) are commonly used. They need to learn a compact latent space that only represents the distribution of normal data. When optimizing this compact latent space, it is difficult to intuitively understand and judge what it encodes, and whether the encoded latent space is compact enough and only contains normal data distribution. (2) High requirements for paired data: Cross-domain transformation can be used for image reconstruction, but this method requires paired data (such as multimodal PET images from the same person) for training. In medical scenarios, multimodal PET images (such as PET images from the same person) are collected from the same normal person. 18 F-FDG PET and 11 C-CFT PET) is extremely expensive and faces challenges in ethical approval such as radiation dose. Summary of the invention

[0006] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a metabolic abnormality detection method, device, terminal, medium and program product based on cross-modal generation, which is used to solve the problems of difficulty in latent space optimization and high requirements for paired data when using existing imaging methods for diagnosis.

[0007] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a metabolic abnormality detection method based on cross-modal generation, including: obtaining a real 18F-FDG positron emission tomography image of the subject's head; based on the obtained real 18F-FDG positron emission tomography image of the head, using a trained explicit cross-modal image conversion model, reconstructing a pseudo-normal 18F-FDG positron emission tomography image of the subject; according to the real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image, obtaining a residual image by calculating the intensity difference between the two; based on the residual image, using a set abnormality evaluation index, performing metabolic abnormality detection on the subject.

[0008] In some embodiments of the first aspect of the present application, the method of training the explicit cross-modal image conversion model includes: using the obtained real normal 18F-FDG image dataset and the unpaired real normal 11C-CFT positron emission tomography image dataset to train the CycleGAN network to obtain an initial explicit cross-modal image conversion model; based on the real normal 18F-FDG image dataset, by simulating abnormal metabolism of the real normal 18F-FDG image, a simulated metabolic abnormal 18F-FDG image dataset is obtained; using the real normal 18F-FDG image dataset and the simulated metabolic abnormal 18F-FDG image dataset, the initial explicit cross-modal image conversion model is jointly trained to obtain the final explicit cross-modal image conversion model.

[0009] In some embodiments of the first aspect of the present application, the explicit cross-modal image conversion model includes: a CFT image generator and a FDG image generator.

[0010] In some embodiments of the first aspect of the present application, based on the obtained real 18F-FDG positron emission tomography image of the head, a pseudo-normal 18F-FDG positron emission tomography image of the subject is reconstructed using a trained explicit cross-modal image conversion model, including: based on the obtained real 18F-FDG positron emission tomography image of the head, a pseudo-normal 11C-CFT positron emission tomography image is generated using a CFT image generator; and using an FDG image generator to perform reverse generation mapping on the generated pseudo-normal 11C-CFT positron emission tomography image to reconstruct the pseudo-normal 18F-FDG positron emission tomography image of the subject.

[0011] In some embodiments of the first aspect of the present application, the specific process of obtaining a simulated metabolic abnormality 18F-FDG image data set includes: for each image in a real normal 18F-FDG image data set, randomly selecting a volume block in the image; downsampling the volume block and filling it with Poisson noise to obtain a noise map; upsampling the noise map to the size of the volume block and performing a circular shift, and then adding it to the volume block to obtain the corresponding simulated metabolic abnormality 18F-FDG image; outputting the simulated metabolic abnormality 18F-FDG images corresponding to each image in the real normal 18F-FDG image data set as a simulated metabolic abnormality 18F-FDG image data set.

[0012] In some embodiments of the first aspect of the present application, the abnormality assessment indicators include: the highest metabolic abnormality, the lowest metabolic abnormality, the abnormal metabolic range and the average metabolic change; wherein the maximum value of the measured residual image is output as the highest metabolic abnormality; the minimum value of the measured residual image is output as the lowest metabolic abnormality; the difference between the maximum and minimum values ​​of the measured residual image is output as the abnormal metabolic range; and the average value of the cyclic consistency loss between the measured real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image is output as the average metabolic change.

[0013] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a metabolic abnormality detection device based on cross-modal generation, including: an acquisition module, used to obtain a real 18F-FDG positron emission tomography image of the subject's head; a reconstruction module, connected to the acquisition module, used to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject based on the obtained real 18F-FDG positron emission tomography image of the head using a trained explicit cross-modal image conversion model; a detection module, connected to the reconstruction module, used to obtain a residual image based on the real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image by calculating the intensity difference between the two; based on the residual image, the subject is subjected to metabolic abnormality detection using a set abnormality evaluation index.

[0014] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the metabolic anomaly detection method based on cross-modal generation is implemented.

[0015] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides a computer program product, which includes a computer program code. When the computer program code runs on a computer, the computer implements the metabolic anomaly detection method based on cross-modal generation.

[0016] To achieve the above-mentioned purpose and other related purposes, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the metabolic anomaly detection method based on cross-modal generation.

[0017] As described above, the metabolic anomaly detection method, device, terminal, medium and program product based on cross-modal generation of the present application have the following beneficial effects:

[0018] (1) This application alleviates the phenomenon of identity mapping by adopting an explicit cross-modal image conversion model, so that abnormal metabolism can be effectively detected;

[0019] (2) The explicit cross-modal image conversion model can efficiently generate unpaired 3D medical images;

[0020] (3) The abnormal evaluation indicators set comprehensively show the metabolic changes in different brain regions, which helps to conduct more accurate metabolic abnormality detection;

[0021] (4) The residual image can intuitively display the abnormal metabolic pattern of the subject’s head, enhancing the visual interpretability and anomaly detection capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Shown is a flow chart of a metabolic abnormality detection method based on cross-modal generation in one embodiment of the present application.

[0023] Figure 2 Shown is a schematic diagram of the process of training an explicit cross-modal image conversion model in one embodiment of the present application.

[0024] Figure 3 Shown is a schematic block diagram of a metabolic abnormality detection device based on cross-modal generation in one embodiment of the present application.

[0025] Figure 4 Shown is a schematic diagram of the structure of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0026] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0027] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0028] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0029] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0030] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:

[0031] <1> PET (Positron Emission Tomography): PET images are a type of nuclear medicine imaging technique that uses radioactive tracers to detect biochemical processes in the body.

[0032] <2> 18 F-FDG(18 F-fluorodeoxyglucose): 18 F-FDG is a commonly used radiotracer widely used in positron emission tomography (PET) imaging.

[0033] <3> 11 C-CFT: 11 C-CFT is a radioactive tracer used primarily in positron emission tomography imaging. It is an imaging agent for dopamine transporter (DAT) and is used to study the function and integrity of the dopamine system.

[0034] The present application provides a metabolic abnormality detection method, device, terminal, medium and program product based on cross-modal generation, the method comprising: based on the obtained real 18F-FDG positron emission tomography image of the subject's head, using a trained explicit cross-modal image conversion model to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject; according to the real 18F-FDG positron emission tomography image of the head and the reconstructed pseudo-normal 18F-FDG positron emission tomography image, a residual image is obtained by calculating the intensity difference between the two; based on the residual image, using the set abnormal evaluation index, metabolic abnormality detection is performed on the subject. The present application uses a trained explicit cross-modal image conversion model to process the head 18F-FDG positron emission tomography image in three-dimensional space to obtain a residual image, which intuitively shows the metabolic abnormality pattern of the subject's brain, thereby assisting doctors in carrying out efficient and accurate diagnosis.

[0035] To facilitate understanding of the embodiments of the present application, first Figure 1 Detailed description. Figure 1 A schematic flow chart of a metabolic anomaly detection method based on cross-modal generation in an embodiment of the present invention is shown. The metabolic anomaly detection method based on cross-modal generation in this embodiment mainly includes the following steps:

[0036] Step S11: Obtain a real 18F-FDG positron emission tomography image of the subject's head.

[0037] In one embodiment, the 18F positron emission tomography image of the subject's head is 18 F-FDG PET images, 18 F-FDG PET produces three-dimensional images.

[0038] Step S12: Based on the obtained real 18F-FDG positron emission tomography image of the head, a pseudo-normal 18F-FDG positron emission tomography image of the subject is reconstructed using the trained explicit cross-modal image conversion model.

[0039] In one embodiment, if Figure 2 As shown, the method of training the cross-modal image conversion model includes: using the obtained real normal 18F-FDG image dataset and the unpaired real normal 11C-CFT positron emission tomography image dataset to train the CycleGAN network to obtain an initial explicit cross-modal image conversion model; based on the real normal 18F-FDG image dataset, by simulating abnormal metabolism of the real normal 18F-FDG image, a simulated metabolic abnormal 18F-FDG image dataset is obtained; using the real normal 18F-FDG image dataset and the simulated metabolic abnormal 18F-FDG image dataset, the initial explicit cross-modal image conversion model is jointly trained to obtain the final explicit cross-modal image conversion model.

[0040] It should be understood that in actual clinical scenarios, the normal control group (NC) is paired 18 F-FDG PET and 11 The difficulty of C-CFT PET images requires the training of CFT image generators and FDG image generators based on unpaired images. The CycleGAN framework can help achieve the generation of unpaired 3D medical images. CycleGAN (Cycle Consistency Generative Adversarial Network) is a model for image-to-image translation that can learn how to transform from one image domain (modality) to another without paired training samples. The core idea of ​​CycleGAN is to use cycle consistency loss to ensure the reversibility of the conversion process, that is, the image from domain A to domain B and back to domain A should be consistent with the original image. The architecture of CycleGAN contains two generators (G and F) and two discriminators (Dx and Dy). Generator G is responsible for converting images from domain X to domain Y, while generator F performs the opposite operation, converting images from domain Y back to domain X. Discriminators Dx and Dy are used to determine whether the image belongs to the real image of domain X and domain Y, respectively. This architecture allows the model to be trained without paired training samples because the model can learn how to convert images to each other through cycle consistency loss. Joint training refers to using the real normal 18F-FDG image dataset and the simulated metabolic abnormality 18F-FDG image dataset to jointly train the initial explicit cross-modal image conversion model.

[0041] It should be noted that the cross-modal image conversion model of this embodiment is trained using a dual-path training method. The first path "single-category cross-modal conversion" prompts the generator to construct a clear normal (non-patient) data distribution through cross-modal mapping, while the second path "abnormal metabolism suppression" uses the abnormal metabolism simulation mechanism to enhance the generator's ability to model only normal metabolism rather than (patient) abnormal metabolism.

[0042] In one embodiment, the specific process of simulating abnormal metabolism to obtain a simulated metabolic abnormal 18F-FDG image data set includes: for each image in a real normal 18F-FDG image data set, randomly selecting a volume block in the image; downsampling the volume block and filling it with Poisson noise to obtain a noise map; upsampling the noise map to the size of the volume block, performing a circular shift, and then adding it to the position of the volume block to obtain a corresponding simulated metabolic abnormal 18F-FDG image; outputting the simulated metabolic abnormal 18F-FDG images corresponding to each image in the real normal 18F-FDG image data set as a simulated metabolic abnormal 18F-FDG image data set.

[0043] In one embodiment, the cross-modal image conversion model includes: a CFT image generator and a FDG image generator.

[0044] Specifically, the core concept of abnormal metabolic suppression is to increase the difficulty of cyclic modal conversion by simulating metabolic abnormalities in the input image, so that the model can learn and memorize normal metabolic patterns, thereby obtaining sensitive perception of brain metabolism. According to clinical prior knowledge, Poisson noise is the main source of noise in PET imaging. The method of this embodiment increases the unpredictability of Poisson noise, thereby preventing the model from easily identifying abnormal patterns.

[0045] For each image in the real normal 18F-FDG image dataset (the images in the real normal 18F-FDG image dataset are all real normal 18 F-FDG PET image), first select a volume patch on the image. A volume patch refers to a local area or cube in three-dimensional image data. Then, since lower resolution noise is more helpful for abnormality detection than pixel-level noise, the selected volume patch is downsampled and filled with Poisson noise to generate a noise map. Finally, the noise map is upsampled to the size of the selected volume patch, and the upsampled noise image is circularly shifted and then the circularly shifted noise map is added to the selected volume patch to complete the abnormal simulation of the image and obtain the corresponding simulated metabolic abnormality 18F-FDG image.

[0046] It should be understood that Poisson noise is a common type of noise in image processing. It usually appears during the image acquisition process, especially in low light conditions. The characteristic of Poisson noise is that the intensity of the noise is proportional to the signal intensity of the image. It is caused by the randomness of light quanta, so it has a granular visual effect. The generation of Poisson noise is related to the Poisson distribution. Cyclic shift is a simple image processing operation that moves the pixels of an image in a certain direction (usually horizontally or vertically) according to a certain step size. The image after cyclic shift will change visually, but the content remains consistent.

[0047] It should be noted that in abnormal metabolism suppression. The simulated abnormal metabolism is added to the real normal 18F-FDG positron emission tomography image to form a pseudo-abnormal 18F-FDG positron emission tomography image. The CFT image generator learns to recover the abnormal 11C-CFT positron emission tomography image containing complete normal dopamine levels from the pseudo-abnormal 18F-FDG positron emission tomography image. Then the FDG image generator converts the abnormal 11C-CFT positron emission tomography image containing complete normal dopamine levels into an abnormal 18F-FDG positron emission tomography image. The above training setting can also be applied to the reverse cycle modal conversion process.

[0048] In one embodiment, based on the obtained 18F-FDG positron emission tomography image of the head, a trained explicit cross-modal image conversion model is used to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject, including: based on the obtained real 18F-FDG positron emission tomography image of the head, a CFT image generator is used to generate a pseudo-normal 11C-CFT positron emission tomography image; using the FDG image generator, reversely generate mapping on the generated pseudo-normal 11C-CFT positron emission tomography image to reconstruct the pseudo-normal 18F-FDG positron emission tomography image of the subject.

[0049] It should be noted that the 11C-CFT positron emission tomography image is 11 C-CFT PET image.

[0050] Specifically, a 18F-FDG positron emission tomography image of the subject's head is obtained, and quality control and preprocessing are performed. Then, the CFT image generator is used as an encoder, and the CFT image generator maps the subject's real head 18F-FDG positron emission tomography image to the synthetic 11C-CFT positron emission tomography image space to obtain a synthetic pseudo-normal 11C-CFT positron emission tomography image with normal dopamine level characteristics. The synthetic pseudo-normal 11C-CFT positron emission tomography image is used to represent the normal dopamine level and provide a normal benchmark for subsequent metabolic abnormality detection. Then, the FDG image generator is used as a decoder, and the FDG image generator reconstructs the pseudo-normal 18F-FDG positron emission tomography image of the head that is closest to the normal (non-disease) state of the subject from the synthetic pseudo-normal 11C-CFT positron emission tomography image through reverse reconstruction.

[0051] The CFT image generator and the FDG generator both use the same 3D U-Net architecture to ensure that the intermediate generated pseudo-normal 11C-CFT PET images can capture normal metabolic features. A 3D fully convolutional PatchGAN discriminator is used to distinguish between real normal 11C-CFT PET images and synthetic pseudo-normal 11C-CFT PET images, so as to match the metabolic features of the synthetic 11C-CFT PET images with the real data distribution in an adversarial environment. The standard adversarial loss is used as the objective function for optimizing the generator and the discriminator. In addition to the training of the forward loop modality conversion process mentioned above, the same training setting is also implemented in the reverse loop modality conversion process, that is, the conversion from 11C-CFT PET images to 18F-FDG PET images and then to 11C-CFT PET images, thereby enhancing the generator's ability in cross-modality conversion and normal metabolic modeling.

[0052] It should be understood that 3D U-Net is a deep learning network for 3D image segmentation, which is derived from the classic U-Net architecture but is specifically optimized for 3D data (such as medical images). The PatchGAN discriminator is a special type of discriminator that does not make a true or false judgment on the entire image, but divides the image into multiple patches (Patches) and makes an independent judgment on each patch. The output of this discriminator is a probability matrix corresponding to the image patch, where each element represents the probability that the corresponding patch is a real image. In this way, PatchGAN is able to pay more attention to the local details and texture information of the image, thereby producing higher quality results in image generation tasks.

[0053] Step S13: According to the real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image, a residual image is obtained by calculating the intensity difference between the two; based on the residual image, metabolic abnormality detection is performed on the subject using the set abnormality evaluation index.

[0054] In one embodiment, when the subject's head is normal (i.e., not diseased), the subject's head real 18F-FDG positron emission tomography image is input into the trained cross-modal image conversion model to reconstruct a pseudo-normal 18F-FDG positron emission tomography image, which has a small difference in pixel intensity from the real 18F-FDG positron emission tomography image; when the subject's head is abnormal (i.e., diseased), the subject's head 18F-FDG positron emission tomography image is input into the trained cross-modal image conversion model to reconstruct a pseudo-normal 18F-FDG positron emission tomography image. After the conversion model, a pseudo-normal 18F-FDG positron emission tomography image is obtained. The pixel intensity difference between the pseudo-normal 18F-FDG positron emission tomography image and the head 18F-FDG positron emission tomography image is large. Therefore, the residual image obtained based on the pseudo-normal 18F-FDG positron emission tomography image and the head 18F-FDG positron emission tomography image can intuitively show the abnormal metabolic area and degree of abnormality in the subject's head, providing doctors with clear visual indications of abnormal metabolic patterns.

[0055] In one embodiment, the set abnormality assessment indicators include: the highest metabolic abnormality, the lowest metabolic abnormality, the abnormal metabolic range and the average metabolic change; wherein the maximum value of the measured residual image is output as the highest metabolic abnormality; the minimum value of the measured residual image is output as the lowest metabolic abnormality; the difference between the maximum and minimum values ​​of the measured residual image is output as the abnormal metabolic range; and the average value of the cyclic consistency loss between the measured real 18F-FDG-PET image of the head and the pseudo-normal 18F-FDG-PET image is output as the average metabolic change.

[0056] Specifically, the real 18F-FDG positron emission tomography image of the head is compared with the pseudo-normal 18F-FDG positron emission tomography image, and the pixel intensity difference between the two is calculated to obtain the residual image. Highest metabolic abnormality (HMA): The maximum value of the residual image is measured to indicate the peak value of abnormal metabolic increase. Lowest metabolic abnormality (LMA): The minimum value of the residual image is measured to indicate the valley value of abnormal metabolic decrease. Abnormal metabolic range (AMR): The difference between the maximum and minimum values ​​in the residual image is measured to indicate the range of abnormal metabolic changes. Average metabolic change (AMC): The average value of the cyclic consistency loss between the real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image is measured to indicate the average change of metabolic abnormality.

[0057] It should be understood that Cycle Consistency Loss is a loss function used in Generative Adversarial Networks (GANs), especially in the task of unpaired image-to-image translation. The purpose of this loss function is to ensure that the mapping between the two domains is reversible and bijective.

[0058] It should also be understood that a positive value of the residual image (indicated by red) indicates an increase in metabolic abnormality, and a negative value of the residual image (indicated by blue) indicates a decrease in metabolic abnormality. Although the visual difference can intuitively present abnormal metabolism, the use of the set abnormal evaluation index can quantitatively evaluate the subject.

[0059] In one embodiment, the visual detection ability of the present invention and other unsupervised anomaly detection methods in identifying PD metabolic abnormalities in 18F-FDG positron emission tomography images is compared. It can be concluded that although AE and U-Net can reconstruct 18F-FDG positron emission tomography images of normal individuals well, they are not effective in detecting PD metabolic abnormalities. Although CycleGAN and the present invention can both detect high metabolic areas, the present invention is superior to CycleGAN in the accuracy of depicting low metabolic areas. In addition, the present invention shows a stronger ability in modeling the normal metabolism of 18F-FDG PET and the normal dopamine level pattern of 11C-CFT PET, especially in identifying abnormal metabolism in 11C-CFT positron emission tomography images of real PD patients. And, the quantitative performance is compared according to four pre-set abnormality scoring indicators. It is observed that the present invention shows the best performance in all abnormality scores. When the residual image generated by the present invention is used as an additional input, the performance of three common classifiers (ResNet, DenseNet and SEResNet) is significantly improved. This indicates that the present invention can effectively assist supervised CAD systems in making more accurate PD diagnosis and provide credible visual interpretation by directly highlighting abnormal metabolic areas in 18F-FDG positron emission tomography images.

[0060] Figure 3 is a schematic block diagram of a metabolic anomaly detection device based on cross-modal generation provided in an embodiment of the present application. Figure 3 As shown, the metabolic anomaly detection device 3 based on cross-modal generation includes:

[0061] An acquisition module 31 is used to obtain a real 18F-FDG positron emission tomography image of the subject's head;

[0062] A reconstruction module 32, connected to the acquisition module 31, is used to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject based on the acquired real 18F-FDG positron emission tomography image of the head and using a trained explicit cross-modal image conversion model;

[0063] The detection module 33 is connected to the reconstruction module 32, and is used to obtain a residual image based on the head 18F-FDG positron emission tomography image and the pseudo-normal 18F-FDG positron emission tomography image by calculating the intensity difference between the two; based on the residual image, the metabolic abnormality detection of the subject is performed using the set abnormality evaluation index.

[0064] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0065] It should also be understood that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0066] In one embodiment, the method of training the explicit cross-modal image conversion model includes: using the obtained real normal 18F-FDG image data set and the unpaired real normal 11C-CFT positron emission tomography image data set to train the CycleGAN network to obtain an initial explicit cross-modal image conversion model; based on the real normal 18F-FDG image data set, by simulating abnormal metabolism of the real normal 18F-FDG image, to obtain a simulated metabolic abnormal 18F-FDG image data set; using the real normal 18F-FDG image data set and the simulated metabolic abnormal 18F-FDG image data set, the initial explicit cross-modal image conversion model is jointly trained to obtain a final explicit cross-modal image conversion model.

[0067] In one embodiment, the explicit cross-modal image conversion model includes: a CFT image generator and a FDG image generator.

[0068] In one embodiment, based on the obtained real 18F-FDG positron emission tomography image of the head, a trained explicit cross-modal image conversion model is used to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject, including: based on the obtained real 18F-FDG positron emission tomography image of the head, a CFT image generator is used to generate a pseudo-normal 11C-CFT positron emission tomography image; using the FDG image generator, reversely generate mapping on the generated pseudo-normal 11C-CFT positron emission tomography image to reconstruct the pseudo-normal 18F-FDG positron emission tomography image of the subject.

[0069] In one embodiment, the specific process of obtaining a simulated metabolic abnormality 18F-FDG image data set includes: for each image in a real normal 18F-FDG image data set, randomly selecting a volume block in the image; downsampling the volume block and filling it with Poisson noise to obtain a noise map; upsampling the noise map to the size of the volume block and performing a circular shift, and then adding it to the volume block to obtain a corresponding simulated metabolic abnormality 18F-FDG image; outputting the simulated metabolic abnormality 18F-FDG images corresponding to each image in the obtained real normal FDG image data set as a simulated metabolic abnormality 18F-FDG image data set.

[0070] In one embodiment, the abnormality assessment indicators include: the highest metabolic abnormality, the lowest metabolic abnormality, the abnormal metabolic range and the average metabolic change; wherein the maximum value of the measured residual image is output as the highest metabolic abnormality; the minimum value of the measured residual image is output as the lowest metabolic abnormality; the difference between the maximum and minimum values ​​of the measured residual image is output as the abnormal metabolic range; and the average value of the cyclic consistency loss between the measured real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image is output as the average metabolic change.

[0071] Figure 4 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 4 As shown, the electronic terminal includes: at least one processor 401, a memory 402, at least one network interface 403 and a user interface 405. The various components in the device are coupled together through a bus system 404. It can be understood that the bus system 404 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 404 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 4 In the specification, various buses are labeled as bus systems.

[0072] The user interface 405 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0073] It is understood that the memory 402 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0074] The memory 402 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 400. Examples of these data include: any executable program for operating on the electronic terminal 400, such as an operating system 4021 and an application 4022; the operating system 4021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 4022 may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The metabolic anomaly detection method based on cross-modal generation provided in the embodiment of the present invention may be included in the application 4022.

[0075] The method disclosed in the above embodiment of the present invention can be applied to the processor 401, or implemented by the processor 401. The processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 401 or the instruction in the form of software. The above processor 401 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 401 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 401 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0076] In an exemplary embodiment, the electronic terminal 400 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0077] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, the computer executes Figure 1 The metabolic abnormality detection method based on cross-modality generation in the illustrated embodiment.

[0078] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figure 1 The metabolic abnormality detection method based on cross-modality generation in the illustrated embodiment.

[0079] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0080] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0081] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0082] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0084] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0085] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrations. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs), etc.).

[0086] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0087] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0088] In summary, the present application provides a method, device, terminal, medium and program product for detecting metabolic abnormalities based on cross-modal generation, the method comprising: based on the obtained real 18F-FDG positron emission tomography image of the subject's head, using the trained explicit cross-modal image conversion model to reconstruct the subject's pseudo-normal 18F-FDG positron emission tomography image; according to the real 18F-FDG positron emission tomography image of the head and the reconstructed pseudo-normal 18F-FDG positron emission tomography image, by calculating the intensity difference between the two, a residual image is obtained; based on the residual image, using the set abnormal evaluation index, the subject is subjected to metabolic abnormality detection. The present application uses the trained explicit cross-modal image conversion model to process the head 18F-FDG positron emission tomography image in three-dimensional space to obtain a residual image, which intuitively shows the metabolic abnormality pattern of the subject's brain, thereby assisting doctors in carrying out efficient and accurate diagnosis. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0089] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A metabolic anomaly detection method based on cross-modal generation, characterized in that: include: Obtaining real 18F-FDG positron emission tomography images of the subject's head; Based on the obtained real 18F-FDG positron emission tomography image of the head, a pseudo-normal 18F-FDG positron emission tomography image of the subject is reconstructed using the trained explicit cross-modal image conversion model; According to the real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image, a residual image is obtained by calculating the intensity difference between the two; based on the residual image, metabolic abnormality detection is performed on the subject using the set abnormality evaluation index.

2. The metabolic anomaly detection method based on cross-modal generation according to claim 1, characterized in that: Ways to train an explicit cross-modal image translation model include: The CycleGAN network is trained using the obtained real normal 18F-FDG image dataset and the unpaired real normal 11C-CFT positron emission tomography image dataset to obtain the initial explicit cross-modal image conversion model; Based on the real normal 18F-FDG image data set, a simulated metabolic abnormality 18F-FDG image data set is obtained by performing abnormal metabolic simulation on the real normal 18F-FDG image; The initial explicit cross-modal image conversion model is jointly trained using the real normal 18F-FDG image dataset and the simulated metabolic abnormality 18F-FDG image dataset to obtain a final explicit cross-modal image conversion model.

3. The metabolic anomaly detection method based on cross-modal generation according to claim 2, characterized in that: The explicit cross-modal image conversion model includes: a CFT image generator and a FDG image generator.

4. The metabolic anomaly detection method based on cross-modal generation according to claim 3, characterized in that: Based on the obtained real 18F-FDG positron emission tomography image of the head, a pseudo-normal 18F-FDG positron emission tomography image of the subject is reconstructed using the trained explicit cross-modal image conversion model, including: Based on the real 18F-FDG positron emission tomography image of the head, a pseudo-normal 11C-CFT positron emission tomography image is generated using a CFT image generator; The generated pseudo-normal 11C-CFT positron emission tomography image is reversely mapped by using an FDG image generator to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject.

5. The metabolic anomaly detection method based on cross-modal generation according to claim 2, characterized in that: The specific process of obtaining the simulated metabolic abnormality 18F-FDG image dataset includes: For each image in the real normal 18F-FDG image data set, a volume block is randomly selected in the image; the volume block is downsampled and filled with Poisson noise to obtain a noise map; the noise map is upsampled to the size of the volume block and circularly shifted, and then added to the volume block to obtain a corresponding simulated metabolic abnormality 18F-FDG image; The simulated metabolic abnormality 18F-FDG images corresponding to each image in the real normal 18F-FDG image data set are output as the simulated metabolic abnormality 18F-FDG image data set.

6. The metabolic anomaly detection method based on cross-modal generation according to claim 1, characterized in that: The abnormality assessment indicators include: the highest metabolic abnormality, the lowest metabolic abnormality, the abnormal metabolic range, and the average metabolic change; Among them, the maximum value of the measured residual image is output as the highest metabolic abnormality; the minimum value of the measured residual image is output as the lowest metabolic abnormality; the difference between the maximum and minimum values ​​of the measured residual image is output as the abnormal metabolic range; the average value of the cyclic consistency loss between the measured real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image is output as the average metabolic change.

7. A metabolic anomaly detection device based on cross-modal generation, characterized in that: include: An acquisition module, used for obtaining a real 18F-FDG positron emission tomography image of the subject's head; A reconstruction module, connected to the acquisition module, for reconstructing a pseudo-normal 18F-FDG positron emission tomography image of the subject based on the acquired real 18F-FDG positron emission tomography image of the head and using a trained explicit cross-modal image conversion model; The detection module is connected to the reconstruction module and is used to obtain a residual image by calculating the intensity difference between the real 18F-FDG positron emission tomography image of the head and the pseudo-normal 18F-FDG positron emission tomography image; based on the residual image, the metabolic abnormality detection of the subject is performed using the set abnormality evaluation index.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that The computer program product includes a computer program code, and when the computer program code is executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 6.

10. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 6.

Images