Metabolic abnormality detection method and device based on cross-modal generation, terminal, medium and program product

Abstract

The application provides a metabolic abnormality detection method and device based on cross-modal generation, a terminal, a medium and a program product. The method comprises the following steps: based on the obtained head real 18F-FDG positron emission tomography image of a subject, a trained explicit cross-modal image conversion model is used to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject; according to the head real 18F-FDG positron emission tomography image and the reconstructed pseudo-normal 18F-FDG positron emission tomography image, a residual image is obtained by calculating the intensity difference between the two; and based on the residual image, a metabolic abnormality of the subject is detected by using a set abnormality evaluation index. The application uses the trained explicit cross-modal image conversion model to process the head 18F-FDG positron emission tomography image in three-dimensional space, obtains a residual image, and directly shows the metabolic abnormality mode of the brain of the subject, thereby assisting doctors to carry out efficient and accurate diagnosis work.

Description

Technical Field

[0001] This application relates to the field of medical imaging technology, and in particular to a method, device, terminal, medium, and program product for detecting metabolic abnormalities based on cross-modal generation. Background Technology

[0002] Positron emission tomography (PET) is a superior imaging technique, particularly prominent in the diagnosis and treatment of neurological diseases, especially neurodegenerative diseases. Parkinson's disease (PD) is a common neurodegenerative disorder that primarily affects motor function and poses a significant threat to health. This disease often has a long latency period, making diagnosis difficult clinically, which can lead to patients missing the optimal treatment window. Therefore, accurate diagnosis of Parkinson's is crucial for timely and appropriate treatment intervention. Imaging dopamine transporters (DAT) using PET technology, for example... 11 C-CFT PET imaging can reveal functional changes in dopaminergic neurons in the caudate nucleus and putamen regions of the brain, enabling precise identification of Parkinson's patients with high specificity, and has been successfully applied in clinical practice.

[0003] However, due to limitations in equipment and technology, most hospitals are currently unable to implement it. 11 C-CFT PET imaging, and 18 F-fluorodeoxyglucose ( 18 F-FDG is relatively easy to obtain, so many hospitals can routinely perform human FDG testing. 18 F-FDG PET imaging. Although 18 F-FDG PET imaging technology is relatively easy and inexpensive, but it lacks specificity for diagnosing Parkinson's disease. Even experienced nuclear medicine physicians, through... 18 F-FDG PET images also struggle to distinguish subtle differences between Parkinson's patients and normal controls. Therefore, based on 18 Using F-FDG PET for PD diagnosis presents a significant challenge not only for doctors but also limits deep learning-based computer-aided diagnosis (CAD).

[0004] To address these challenges, one feasible solution is to capture... 18F-FDG PET images magnify and highlight disease-related metabolic abnormalities, visually revealing abnormal areas to assist clinicians in efficient polarity diagnosis or enhance the accuracy and interpretability of CAD systems. Technically, this can be achieved using the concept of anomaly detection. 18 Starting with F-FDG PET images, several transformations can be applied to reconstruct the input image. Assuming these transformations are designed for normal controls (e.g., image data collected from healthy individuals rather than diseased populations), then images from patients... 18 F-FDG PET gradually loses disease-related information during the transformation process, resulting in a significant residual between the reconstructed image and the original input image. Conversely, if the input image is from a normal control rather than a patient, the reconstructed image will be highly similar to the input image, with a smaller residual. By comparing different inputs... 18 The residual between the F-FDGPET image and its corresponding reconstructed image can be used to determine whether the input image is related to neurodegenerative diseases such as Parkinson's disease, i.e., whether there is an "abnormality".

[0005] The core challenge in the above process is to construct a transformation for the image and ensure that it is designed for "normal" (non-patient) images. However, there are two difficulties in completing this task: (1) Difficulty in latent space optimization: Generative models such as autoencoders (AEs) and generative adversarial networks (GANs) are commonly used. They need to learn a compact latent space that only represents the distribution of normal data. When optimizing this compact latent space, it is difficult to intuitively understand and judge what it encodes, and whether the latent space obtained by encoding is compact enough and contains only the distribution of normal data. (2) High requirements for paired data: Cross-domain transformation can be used for image reconstruction, but this method requires paired data (e.g., multimodal PET images from the same person) for training. In medical scenarios, multimodal PET images (e.g., from the same normal person) are collected. 18 F-FDG PET and 11 C-CFT PET is extremely expensive and faces ethical approval challenges such as radiation dosage. Summary of the Invention

[0006] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, terminal, medium and program product for detecting metabolic abnormalities based on cross-modal generation, in order to solve the problems of difficulty in latent space optimization and high requirements for paired data when using existing imaging methods for diagnosis.

[0007] To achieve the above and other related objectives, a first aspect of this application provides a method for detecting metabolic abnormalities based on cross-modal generation, comprising: obtaining a real 18F-FDG positron emission tomography (PET) image of a subject's head; reconstructing a pseudo-normal 18F-FDG PET image of the subject using a trained dominant cross-modal image conversion model based on the obtained real 18F-FDG PET image of the head; obtaining a residual image by calculating the intensity difference between the real 18F-FDG PET image of the head and the pseudo-normal 18F-FDG PET image; and detecting metabolic abnormalities in the subject based on the residual image using a set abnormality assessment index.

[0008] In some embodiments of the first aspect of this application, the method of training the dominant cross-modal image conversion model includes: training a CycleGAN network using an obtained real normal 18F-FDG image dataset and an unpaired real normal 11C-CFT positron emission tomography image dataset to obtain an initial dominant cross-modal image conversion model; obtaining a simulated metabolically abnormal 18F-FDG image dataset by performing abnormal metabolic simulation on the real normal 18F-FDG images based on the real normal 18F-FDG image dataset; and jointly training the initial dominant cross-modal image conversion model using the real normal 18F-FDG image dataset and the simulated metabolically abnormal 18F-FDG image dataset to obtain the final dominant cross-modal image conversion model.

[0009] In some embodiments of the first aspect of this application, the explicit cross-modal image conversion model includes: a CFT image generator and an FDG image generator.

[0010] In some embodiments of the first aspect of this application, based on the obtained real 18F-FDG positron emission tomography (PET) images of the head, a pseudo-normal 18F-FDG positron emission tomography image of the subject is reconstructed using a trained dominant cross-modal image transformation model, including: generating a pseudo-normal 11C-CFT positron emission tomography image based on the obtained real 18F-FDG positron emission tomography image of the head using a CFT image generator; and performing a reverse generation mapping on the generated pseudo-normal 11C-CFT positron emission tomography image using an FDG image generator to reconstruct the pseudo-normal 18F-FDG positron emission tomography image of the subject.

[0011] In some embodiments of the first aspect of this application, the specific process of obtaining a simulated metabolic abnormality 18F-FDG image dataset includes: for each image in the real normal 18F-FDG image dataset, randomly selecting a volume block in the image; downsampling the volume block and filling it with Poisson noise to obtain a noise map; upsampling the noise map to the size of the volume block and performing a cyclic shift, then adding it to the volume block to obtain the corresponding simulated metabolic abnormality 18F-FDG image; and outputting the simulated metabolic abnormality 18F-FDG images corresponding to each image in the real normal 18F-FDG image dataset as a simulated metabolic abnormality 18F-FDG image dataset.

[0012] In some embodiments of the first aspect of this application, the anomaly assessment metrics include: maximum metabolic anomaly, minimum metabolic anomaly, abnormal metabolic range, and average metabolic change; wherein, the maximum value of the measured residual image is output as the maximum metabolic anomaly; the minimum value of the measured residual image is output as the minimum metabolic anomaly; the difference between the maximum and minimum values ​​of the measured residual image is output as the abnormal metabolic range; and the average value of the cycle consistency loss between the measured true 18F-FDG positron emission tomography (PET) images of the head and pseudo-normal 18F-FDG PET images is output as the average metabolic change.

[0013] To achieve the above and other related objectives, a second aspect of this application provides a metabolic abnormality detection device based on cross-modal generation, comprising: an acquisition module for acquiring a real 18F-FDG positron emission tomography (PET) image of a subject's head; a reconstruction module connected to the acquisition module for reconstructing a pseudo-normal 18F-FDG PET image of the subject based on the acquired real 18F-FDG PET image of the head using a trained dominant cross-modal image conversion model; a detection module connected to the reconstruction module for obtaining a residual image by calculating the intensity difference between the real 18F-FDG PET image of the head and the pseudo-normal 18F-FDG PET image; and detecting metabolic abnormalities in the subject based on the residual image using a set abnormality assessment index.

[0014] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the metabolic anomaly detection method based on cross-modal generation.

[0015] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, causes the computer to implement the metabolic abnormality detection method based on cross-modal generation.

[0016] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the metabolic abnormality detection method based on cross-modal generation.

[0017] As described above, the metabolic abnormality detection method, device, terminal, medium, and program product based on cross-modal generation of this application have the following beneficial effects:

[0018] (1) This application alleviates the phenomenon of identity mapping by adopting an explicit cross-modal image transformation model, so that abnormal metabolism can be effectively detected;

[0019] (2) The explicit cross-modal image transformation model can efficiently generate three-dimensional medical images in unpaired cases;

[0020] (3) The abnormal assessment indicators set up comprehensively show the metabolic changes in different brain regions, which helps to detect metabolic abnormalities more accurately.

[0021] (4) Residual images can visually show abnormal metabolic patterns in the subject’s head, enhancing visual interpretability and abnormality detection capabilities. Attached Figure Description

[0022] Figure 1 The diagram shown is a flowchart of a metabolic anomaly detection method based on cross-modal generation in one embodiment of this application.

[0023] Figure 2 The diagram shown is a flowchart illustrating the process of training an explicit cross-modal image conversion model in one embodiment of this application.

[0024] Figure 3 The diagram shown is a schematic block diagram of a metabolic abnormality detection device based on cross-modal generation in one embodiment of this application.

[0025] Figure 4 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0027] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0028] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0029] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0030] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0031] <1> PET (Positron Emission Tomography): PET imaging is a nuclear medicine imaging technique that uses radioactive tracers to detect biochemical processes in the body.

[0032] <2> 18 F-FDG(18 F-fluorodeoxyglucose): 18 F-FDG is a commonly used radioactive tracer widely used in positron emission tomography (PET) imaging.

[0033] <3> 11 C-CFT: 11 C-CFT is a radioactive tracer primarily used in positron emission tomography (PET) imaging. It is an imaging agent for dopamine transporters (DAT) used to study the function and integrity of the dopamine system.

[0034] This application provides a method, device, terminal, medium, and program product for detecting metabolic abnormalities based on cross-modal generation. The method includes: reconstructing a pseudo-normal 18F-FDG positron emission tomography (PET) image of the subject using a trained dominant cross-modal image conversion model based on a real 18F-FDG positron emission tomography image of the subject's head; obtaining a residual image by calculating the intensity difference between the real 18F-FDG positron emission tomography image and the reconstructed pseudo-normal 18F-FDG positron emission tomography image; and detecting metabolic abnormalities in the subject based on the residual image using a set abnormality assessment index. This application utilizes a trained dominant cross-modal image conversion model to process 18F-FDG positron emission tomography images of the head in three-dimensional space to obtain residual images, which intuitively display the metabolic abnormality patterns in the subject's brain, thereby assisting doctors in carrying out efficient and accurate diagnostic work.

[0035] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 This document illustrates a flowchart of a metabolic anomaly detection method based on cross-modal generation, as described in an embodiment of the present invention. The method mainly includes the following steps:

[0036] Step S11: Obtain real 18F-FDG positron emission tomography images of the subject's head.

[0037] In one specific embodiment, the 18F positron emission tomography (PET) image of the subject's head is a... 18 F-FDG PET image, 18 F-FDG PET is a three-dimensional image.

[0038] Step S12: Based on the obtained real 18F-FDG positron emission tomography (PET) images of the head, the pseudo-normal 18F-FDG positron emission tomography images of the subject are reconstructed using a trained dominant cross-modal image conversion model.

[0039] In one embodiment, such as Figure 2 As shown, the method for training the cross-modal image conversion model includes: using the obtained real normal 18F-FDG image dataset and unpaired real normal 11C-CFT positron emission tomography image dataset, training the CycleGAN network to obtain an initial dominant cross-modal image conversion model; based on the real normal 18F-FDG image dataset, obtaining a simulated metabolically abnormal 18F-FDG image dataset by performing abnormal metabolism simulation on the real normal 18F-FDG images; and using the real normal 18F-FDG image dataset and the simulated metabolically abnormal 18F-FDG image dataset, jointly training the initial dominant cross-modal image conversion model to obtain the final dominant cross-modal image conversion model.

[0040] It should be understood that, given the difficulty of obtaining a matched normal control (NC) in a real clinical setting... 18 F-FDG PET and 11 The complexity of C-CFT PET images necessitates training both the CFT and FDG image generators using unpaired images. The CycleGAN framework can help generate unpaired 3D medical images. CycleGAN (Cyclic Consistency Generative Adversarial Network) is a model for image-to-image translation that learns to transform from one image domain (modality) to another without paired training samples. The core idea of ​​CycleGAN is to use cycle consistency loss to ensure the reversibility of the transformation process; that is, the image transformed from domain A to domain B and back to domain A should remain consistent with the original image. The CycleGAN architecture consists of two generators (G and F) and two discriminators (Dx and Dy). Generator G is responsible for transforming the image from domain X to domain Y, while generator F performs the reverse operation, transforming the image from domain Y back to domain X. Discriminators Dx and Dy are used to determine whether an image belongs to the true domain X and domain Y, respectively. This architecture allows the model to be trained without paired training samples because the model can learn how to transform images to each other through cycle consistency loss. Joint training refers to training an initial dominant cross-modal image conversion model using both real normal 18F-FDG image datasets and simulated metabolically abnormal 18F-FDG image datasets.

[0041] It should be noted that the cross-modal image conversion model in this embodiment is trained using a dual-path training method. The first path, "single-class cross-modal conversion," prompts the generator to construct a clear normal (non-patient) data distribution through cross-modal mapping, while the second path, "abnormal metabolic suppression," utilizes an abnormal metabolic simulation mechanism to enhance the generator's ability to model only normal metabolism and not (patient) abnormal metabolism.

[0042] In one embodiment, the specific process of simulating abnormal metabolism to obtain a simulated metabolic abnormality 18F-FDG image dataset includes: for each image in the real normal 18F-FDG image dataset, randomly selecting a volume block in the image; downsampling the volume block and filling it with Poisson noise to obtain a noise map; upsampling the noise map to the size of the volume block, performing a cyclic shift, and then adding it to the position of the volume block to obtain the corresponding simulated metabolic abnormality 18F-FDG image; and outputting the simulated metabolic abnormality 18F-FDG images corresponding to each image in the real normal 18F-FDG image dataset as a simulated metabolic abnormality 18F-FDG image dataset.

[0043] In one embodiment, the cross-modal image conversion model includes a CFT image generator and an FDG image generator.

[0044] Specifically, the core concept of abnormal metabolism inhibition is to increase the difficulty of cyclic mode transitions by simulating metabolic abnormalities in the input image, enabling the model to learn and memorize normal metabolic patterns, thereby gaining a sensitive perception of brain region metabolism. Based on prior clinical knowledge, Poisson noise is the main noise source in PET imaging. The method in this embodiment increases the unpredictability of Poisson noise, thus preventing the model from easily identifying abnormal patterns.

[0045] For each image in the real normal 18F-FDG image dataset (all images in the real normal 18F-FDG image dataset are real and normal) 18 First, a volume patch is selected on the F-FDG PET image. A volume patch refers to a local region or cube in the 3D image data. Then, since lower-resolution noise is more helpful for anomaly detection than pixel-level noise, the selected volume patch is downsampled and filled with Poisson noise to generate a noise map. Finally, the noise map is upsampled to the size of the selected volume patch, and then the upsampled noise image is cyclically shifted and added to the selected volume patch to complete the anomaly simulation of the image, obtaining the corresponding simulated metabolic anomaly 18F-FDG image.

[0046] It should be understood that Poisson noise is a common type of noise in image processing, often occurring during image acquisition, especially under low-light conditions. A key characteristic of Poisson noise is that its intensity is proportional to the signal strength of the image. It is caused by the randomness of photons, thus exhibiting a granular visual effect. The generation of Poisson noise is related to the Poisson distribution. Cyclic shift is a simple image processing operation that moves image pixels in a certain direction (usually horizontal or vertical) with a set step size. The image will visually change after cyclic shift, but the content remains consistent.

[0047] It should be noted that in the abnormal metabolism suppression, simulated abnormal metabolism is added to real, normal 18F-FDG positron emission tomography (PET) images to form pseudo-abnormal 18F-FDG PET images. The CFT image generator learns to recover a de-abnormal 11C-CFT PET image containing full, normal dopamine levels from the pseudo-abnormal 18F-FDG PET image. The FDG image generator then converts the de-abnormal 11C-CFT PET image containing full, normal dopamine levels into a de-abnormal 18F-FDG PET image. The above training setup can also be applied to a reverse recurrent mode conversion pipeline.

[0048] In one embodiment, based on the obtained head 18F-FDG positron emission tomography (PET) image, a pseudo-normal 18F-FDG PET image of the subject is reconstructed using a trained dominant cross-modal image transformation model. This includes: generating a pseudo-normal 11C-CFT PET image based on the obtained real head 18F-FDG PET image using a CFT image generator; and using an FDG image generator to perform a reverse generation mapping on the generated pseudo-normal 11C-CFT PET image to reconstruct the pseudo-normal 18F-FDG PET image of the subject.

[0049] It should be noted that the 11C-CFT positron emission tomography (PET) image is... 11 C-CFT PET image.

[0050] Specifically, 18F-FDG positron emission tomography (PET) images of the subject's head were obtained and subjected to quality control and preprocessing. Then, a CFT image generator was used as an encoder to map the subject's actual 18F-FDG PET image onto a synthesized 11C-CFT PET image space, resulting in a synthesized pseudo-normal 11C-CFT PET image with normal dopamine level characteristics. This synthesized pseudo-normal 11C-CFT PET image represents normal dopamine levels, providing a baseline for subsequent metabolic abnormality detection. Next, an FDG image generator was used as a decoder to reconstruct, through reverse reconstruction, a pseudo-normal 18F-FDG PET image of the head that most closely resembled the subject's normal (non-disease) state from the synthesized pseudo-normal 11C-CFT PET image.

[0051] Both the CFT image generator and the FDG generator employ the same 3D U-Net architecture to ensure that the intermediately generated pseudo-normal 11C-CFT positron emission tomography (PET) images can capture normal metabolic features. A 3D fully convolutional PatchGAN discriminator is used to distinguish between real normal 11C-CFT PET images and synthesized pseudo-normal 11C-CFT PET images, aiming to match the metabolic features of the synthesized 11C-CFT PET images with the real data distribution in an adversarial environment. Standard adversarial loss is used as the objective function for optimizing both the generator and discriminator. In addition to the training of the aforementioned forward recurrent modality transition process, the same training settings are implemented in the reverse recurrent modality transition process, namely, the transformation from 11C-CFT PET images to 18F-FDG PET images and back to 11C-CFT PET images, thereby enhancing the generator's ability in cross-modal transitions and normal metabolic modeling.

[0052] It should be understood that 3D U-Net is a deep learning network for 3D image segmentation. It originates from the classic U-Net architecture but is specifically optimized for 3D data (such as medical images). The PatchGAN discriminator is a special type of discriminator that does not judge the entire image as real or fake. Instead, it divides the image into multiple patches and independently classifies each patch. The output of this discriminator is a probability matrix corresponding to each image patch, where each element represents the probability that the corresponding patch is a real image. In this way, PatchGAN can pay more attention to the local details and texture information of the image, thus producing higher quality results in image generation tasks.

[0053] Step S13: Based on the real 18F-FDG positron emission tomography (PET) image of the head and the pseudo-normal 18F-FDG PET image, a residual image is obtained by calculating the intensity difference between the two; based on the residual image, metabolic abnormalities are detected in the subject using the set abnormality assessment index.

[0054] In one embodiment, when the subject's head is normal (i.e., in a disease-free state), the subject's actual 18F-FDG positron emission tomography (PET) image is input into a trained cross-modal image conversion model to reconstruct a pseudo-normal 18F-FDG PET image, which has a small difference in pixel intensity compared to the actual 18F-FDG PET image. When the subject's head has abnormalities (i.e., in a disease-free state), the subject's 18F-FDG PET image is input into the trained cross-modal image conversion model. After the model conversion, pseudo-normal 18F-FDG positron emission tomography (PET) images are obtained. The pixel intensity difference between the pseudo-normal 18F-FDG PET images and the head 18F-FDG PET images is relatively large. Therefore, the residual images obtained from the pseudo-normal 18F-FDG PET images and the head 18F-FDG PET images can be used to intuitively show the abnormal metabolic areas in the subject's head and the degree of abnormality, providing doctors with clear visual indications about abnormal metabolic patterns.

[0055] In one embodiment, the defined anomaly assessment indicators include: highest metabolic anomaly, lowest metabolic anomaly, abnormal metabolic range, and average metabolic change; wherein, the maximum value of the measured residual image is output as the highest metabolic anomaly; the minimum value of the measured residual image is output as the lowest metabolic anomaly; the difference between the maximum and minimum values ​​of the measured residual image is output as the abnormal metabolic range; and the average value of the cycle consistency loss between the measured real 18F-FDG-PET images and pseudo-normal 18F-FDG-PET images is output as the average metabolic change.

[0056] Specifically, a residual image is obtained by comparing genuine 18F-FDG positron emission tomography (PET) images of the head with pseudo-normal 18F-FDG PET images and calculating the pixel intensity difference between the two. The residual image is defined as follows: Maximum Metabolic Anomaly (HMA): The maximum value in the residual image represents the peak of abnormal metabolic increase. Minimum Metabolic Anomaly (LMA): The minimum value in the residual image represents the trough of abnormal metabolic decrease. Abnormal Metabolic Range (AMR): The difference between the maximum and minimum values ​​in the residual image represents the range of abnormal metabolic changes. Average Metabolic Change (AMC): The average value of the cycle consistency loss between genuine and pseudo-normal 18F-FDG PET images of the head represents the average change in metabolic abnormalities.

[0057] It should be understood that Cycle Consistency Loss is a loss function used in Generative Adversarial Networks (GANs), particularly in tasks involving unpaired image-to-image translation. The purpose of this loss function is to ensure that the mapping between two domains is invertible and bijective.

[0058] It should also be understood that positive values ​​in the residual image (represented in red) indicate an increase in metabolic abnormalities, while negative values ​​(represented in blue) indicate a decrease in metabolic abnormalities. Although visual differences can visually represent abnormal metabolism, pre-defined abnormality assessment indicators can quantitatively evaluate the subjects.

[0059] In one embodiment, the visual detection capabilities of this invention and other unsupervised anomaly detection methods in identifying PD metabolic abnormalities in 18F-FDG positron emission tomography (PET) images were compared. It was found that while AE and U-Net could reconstruct 18F-FDG PET images of normal individuals well, they were not effective in detecting PD metabolic abnormalities. Although both CycleGAN and this invention could detect high-metabolic regions, this invention outperformed CycleGAN in accurately depicting low-metabolic regions. Furthermore, this invention demonstrated stronger capabilities in modeling normal metabolism in 18F-FDG PET and normal dopamine level patterns in 11C-CFT PET, particularly in identifying abnormal metabolism in 11C-CFT PET images of real PD patients. Quantitative performance was compared based on four pre-defined anomaly scoring metrics. It was observed that this invention exhibited the best performance across all anomaly scores. When using the residual images generated by this invention as additional input, the performance of three common classifiers (ResNet, DenseNet, and SEResNet) was significantly improved. This demonstrates that the present invention can effectively assist supervised CAD systems in performing more accurate PD diagnosis and provide a credible visual interpretation by directly highlighting abnormal metabolic regions in 18F-FDG positron emission tomography images.

[0060] Figure 3 This is a schematic block diagram of a metabolic abnormality detection device based on cross-modal generation provided in an embodiment of this application. Figure 3 As shown, the metabolic anomaly detection device 3 based on cross-modal generation includes:

[0061] Acquisition module 31 is used to acquire real 18F-FDG positron emission tomography images of the subject's head;

[0062] The reconstruction module 32, connected to the acquisition module 31, is used to reconstruct the pseudo-normal 18F-FDG positron emission tomography image of the subject based on the acquired real 18F-FDG positron emission tomography image of the head, using a trained dominant cross-modal image conversion model.

[0063] The detection module 33, connected to the reconstruction module 32, is used to obtain a residual image by calculating the intensity difference between the head 18F-FDG positron emission tomography image and the pseudo-normal 18F-FDG positron emission tomography image; and to detect metabolic abnormalities in the subject based on the residual image using a set abnormality assessment index.

[0064] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0065] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0066] In one embodiment, the method for training the dominant cross-modal image conversion model includes: training the CycleGAN network using the obtained real normal 18F-FDG image dataset and unpaired real normal 11C-CFT positron emission tomography image dataset to obtain an initial dominant cross-modal image conversion model; obtaining a simulated metabolically abnormal 18F-FDG image dataset by performing abnormal metabolism simulation on the real normal 18F-FDG images based on the real normal 18F-FDG image dataset; and jointly training the initial dominant cross-modal image conversion model using the real normal 18F-FDG image dataset and the simulated metabolically abnormal 18F-FDG image dataset to obtain the final dominant cross-modal image conversion model.

[0067] In one embodiment, the explicit cross-modal image conversion model includes a CFT image generator and an FDG image generator.

[0068] In one embodiment, based on the obtained real 18F-FDG positron emission tomography (PET) images of the head, a pseudo-normal 18F-FDG positron emission tomography image of the subject is reconstructed using a trained dominant cross-modal image transformation model. This includes: generating a pseudo-normal 11C-CFT positron emission tomography image based on the obtained real 18F-FDG positron emission tomography image of the head using a CFT image generator; and performing a reverse generation mapping on the generated pseudo-normal 11C-CFT positron emission tomography image using an FDG image generator to reconstruct the pseudo-normal 18F-FDG positron emission tomography image of the subject.

[0069] In one embodiment, the specific process of obtaining a simulated metabolic abnormality 18F-FDG image dataset includes: for each image in the real normal 18F-FDG image dataset, randomly selecting a volume block in the image; downsampling the volume block and filling it with Poisson noise to obtain a noise map; upsampling the noise map to the size of the volume block and performing a cyclic shift, then adding it to the volume block to obtain the corresponding simulated metabolic abnormality 18F-FDG image; and outputting the simulated metabolic abnormality 18F-FDG images corresponding to each image in the obtained real normal FDG image dataset as a simulated metabolic abnormality 18F-FDG image dataset.

[0070] In one embodiment, the anomaly assessment indicators include: highest metabolic anomaly, lowest metabolic anomaly, abnormal metabolic range, and average metabolic change; wherein, the maximum value of the measured residual image is output as the highest metabolic anomaly; the minimum value of the measured residual image is output as the lowest metabolic anomaly; the difference between the maximum and minimum values ​​of the measured residual image is output as the abnormal metabolic range; and the average value of the cycle consistency loss between the measured true 18F-FDG positron emission tomography (PET) images of the head and pseudo-normal 18F-FDG PET images is output as the average metabolic change.

[0071] Figure 4 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 4 As shown, the electronic terminal includes at least one processor 401, a memory 402, at least one network interface 403, and a user interface 405. The various components in the device are coupled together via a bus system 404. It is understood that the bus system 404 is used to implement communication between these components. In addition to a data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 4 The general will label all buses as bus systems.

[0072] The user interface 405 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0073] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0074] In this embodiment of the invention, the memory 402 is used to store various types of data to support the operation of the electronic terminal 400. Examples of this data include: any executable program for operation on the electronic terminal 400, such as the operating system 4021 and application program 4022; the operating system 4021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 4022 may contain various applications, such as media players, browsers, etc., for implementing various application services. The metabolic anomaly detection method based on cross-modal generation provided in this embodiment of the invention can be included in the application program 4022.

[0075] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 401 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0076] In an exemplary embodiment, the electronic terminal 400 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0077] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figure 1 The metabolic anomaly detection method based on cross-modal generation in the illustrated embodiment.

[0078] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when executed on a computer, causes the computer to perform... Figure 1 The metabolic anomaly detection method based on cross-modal generation in the illustrated embodiment.

[0079] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0080] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0081] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0085] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0086] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0087] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0088] In summary, this application provides a method, device, terminal, medium, and program product for detecting metabolic abnormalities based on cross-modal generation. The method includes: reconstructing a pseudo-normal 18F-FDG positron emission tomography (PET) image of the subject using a trained dominant cross-modal image conversion model based on a real 18F-FDG PET image of the subject's head; obtaining a residual image by calculating the intensity difference between the real 18F-FDG PET image and the reconstructed pseudo-normal 18F-FDG PET image; and detecting metabolic abnormalities in the subject based on the residual image using a set abnormality assessment index. This application utilizes a trained dominant cross-modal image conversion model to process 18F-FDG PET images of the head in three-dimensional space to obtain a residual image, which intuitively displays the metabolic abnormality patterns in the subject's brain, thereby assisting doctors in conducting efficient and accurate diagnostic work. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0089] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A metabolic abnormality detection method based on cross-modal generation, characterized in that, The method comprises the following steps: obtaining a real 18F-FDG head positron emission tomography image of a subject; based on the obtained real 18F-FDG head positron emission tomography image, using a trained explicit cross-modality image conversion model to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject; wherein the training of the explicit cross-modality image conversion model comprises the following steps: using the obtained real normal 18F-FDG image dataset and the unpaired real normal 11C-CFT positron emission tomography image dataset to train the CycleGAN network to obtain an initial explicit cross-modality image conversion model; based on the real normal 18F-FDG image dataset, obtaining a simulated metabolic abnormal 18F-FDG image dataset by simulating abnormal metabolism of real normal 18F-FDG images; using the real normal 18F-FDG image dataset and the simulated metabolic abnormal 18F-FDG image dataset, jointly training the initial explicit cross-modality image conversion model to obtain a final explicit cross-modality image conversion model; based on the real 18F-FDG head positron emission tomography image and the pseudo-normal 18F-FDG positron emission tomography image, obtaining a residual image by calculating the intensity difference between the two images; based on the residual image, using a set of abnormality evaluation indicators to detect metabolic abnormalities of the subject.

2. The metabolic abnormality detection method based on cross-modality generation according to claim 1, characterized in that, The explicit cross-modality image conversion model comprises a CFT image generator and an FDG image generator. 3.The metabolic abnormality detection method based on cross-modality generation according to claim 2, characterized in that, Based on the obtained real 18F-FDG head positron emission tomography image, using a trained explicit cross-modality image conversion model to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject, comprising: based on the obtained real 18F-FDG head positron emission tomography image, using the CFT image generator to generate a pseudo-normal 11C-CFT positron emission tomography image; using the FDG image generator to perform reverse generation mapping on the generated pseudo-normal 11C-CFT positron emission tomography image to reconstruct the pseudo-normal 18F-FDG positron emission tomography image of the subject. 4.The metabolic abnormality detection method based on cross-modality generation according to claim 1, wherein, The specific process of obtaining the simulated metabolic abnormal 18F-FDG image dataset comprises: for each image in the real normal 18F-FDG image dataset, randomly selecting a volume block in the image; down-sampling the volume block and filling Poisson noise to obtain a noise map; up-sampling the noise map to the size of the volume block and performing a circular shift, and then adding it to the volume block to obtain the corresponding simulated metabolic abnormal 18F-FDG image; outputting the simulated metabolic abnormal 18F-FDG images corresponding to each image in the real normal 18F-FDG image dataset as the simulated metabolic abnormal 18F-FDG image dataset. 5.The metabolic abnormality detection method based on cross-modality generation according to claim 1, wherein, The abnormality evaluation indicators include the highest metabolic abnormality, the lowest metabolic abnormality, the abnormal metabolism range, and the average metabolic change. The maximum value of the measured residual image is output as the highest metabolic abnormality; the minimum value of the measured residual image is output as the lowest metabolic abnormality; the difference between the maximum value and the minimum value of the measured residual image is output as the abnormal metabolic range; and the average value of the cycle consistency loss between the measured head real 18F-FDG positron emission tomography image and the pseudo-normal 18F-FDG positron emission tomography image is output as the average metabolic change. 6.A metabolic abnormality detection device based on cross-modal generation, characterized by, Comprise: An acquisition module is configured to obtain a head real 18F-FDG positron emission tomography image of a subject; A reconstruction module is connected with the acquisition module and is configured to reconstruct a pseudo-normal 18F-FDG positron emission tomography image of the subject based on the obtained head real 18F-FDG positron emission tomography image and using a trained explicit cross-modality image conversion model; wherein the training of the explicit cross-modality image conversion model comprises: Using the obtained real normal 18F-FDG image dataset and the non-paired real normal 11C-CFT positron emission tomography image dataset, the CycleGAN network is trained to obtain an initial explicit cross-modality image conversion model; Based on the real normal 18F-FDG image dataset, a simulated metabolic abnormality 18F-FDG image dataset is obtained by simulating abnormal metabolism of real normal 18F-FDG images; Using the real normal 18F-FDG image dataset and the simulated metabolic abnormality 18F-FDG image dataset, the initial explicit cross-modality image conversion model is jointly trained to obtain a final explicit cross-modality image conversion model; A detection module is connected with the reconstruction module and is configured to obtain a residual image by calculating the intensity difference between the head real 18F-FDG positron emission tomography image and the pseudo-normal 18F-FDG positron emission tomography image; and based on the residual image, metabolic abnormality detection of the subject is performed using a set of abnormality evaluation indicators.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 5.

8. A computer program product, characterised in that, The computer program product includes computer program code, which, when executed on a computer, causes the computer to implement the method of any one of claims 1 to 5.

9. An electronic terminal comprising a memory, a processor and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 5. The processor executes the computer program to implement the method of any one of claims 1 to 5.

Images