A functional medical image progressive fusion method and system

Abstract

The application discloses a functional medical image progressive fusion method and system, and relates to the technical field of image processing. The method comprises the following steps: acquiring PET images and CT images to be processed, and taking an improved 3D U-Net structure as a generator of a generative adversarial network (GAN); acquiring PET feature combination images and CT feature combination images, training the generator, and obtaining an image fusion model; inputting the PET images and the CT images to be processed into the image fusion model, acquiring the PET feature combination images and the CT feature combination images, performing down sampling again, and obtaining PET feature maps and CT feature maps; performing a splicing operation on the added images and the up-sampled images, and obtaining splicing images; and performing up sampling on the splicing images again, outputting fusion images through a last up-sampling module. The application can observe the gradual details of lesion metabolism information and anatomical structure information, and is suitable for the multi-modal fusion requirements of various functional medical images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a progressive fusion method and system for functional medical images. Background Technology

[0002] Medical imaging technology plays a crucial role in modern clinical diagnosis and treatment, providing tissue and functional information through various imaging modalities. In recent years, fused PET and CT imaging has become a powerful tool. PET images focus on reflecting metabolic activity and the functional state of lesion areas, while CT images provide precise anatomical details. Therefore, fusion of PET / CT images combines the advantages of both modalities, providing a more comprehensive view of the functional and structural characteristics of lesions.

[0003] In the current field of medical image fusion, traditional methods are mainly based on fixed-ratio image fusion schemes. A common approach is to directly superimpose different imaging modalities (such as PET / CT) according to set weights. This scheme can combine the metabolic information of PET and the anatomical details of CT, providing doctors with a more comprehensive view of lesions. In recent years, GANs have been widely used in medical image fusion, but most schemes still use a 50 / 50 fusion method for PET / CT, failing to effectively solve the problem of dynamically adjusting the ratio. These GAN-based schemes are mainly trained using fixed-ratio PET / CT samples and generate images of the ratio used in training during the generation process.

[0004] The shortcomings of existing technologies are as follows: while these GANs schemes can generate high-quality fused images at a single scale, they lack the ability to achieve a continuous transition between multiple scale values, and cannot generate gradually changing image effects between PET / CT. Furthermore, they cannot flexibly adjust the fusion ratio between PET / CT features according to specific clinical needs, limiting the diversity and flexibility of image analysis. For physicians who wish to observe the dynamic changes between lesion metabolic and anatomical structural features, fused images at a single scale cannot intuitively demonstrate the gradual fusion effect from PET to CT, affecting their in-depth analysis and interpretation of lesions. Summary of the Invention

[0005] Therefore, it is necessary to provide a progressive fusion method and system for functional medical images to address the aforementioned technical problems.

[0006] This invention provides a progressive fusion method for functional medical images, comprising:

[0007] Acquire PET and CT images to be processed;

[0008] A low-frequency feature extraction module for PET images and a high-frequency feature extraction module for CT images are added to the input of the 3D U-Net convolutional neural network. Each processing layer of the 3D U-Net includes two downsampling modules and a corresponding upsampling module. A concatenation operation Cat is added between adjacent upsampling modules. A controllable adjustment module is added between the concatenation operation Cat at the output of the two downsampling modules and the input of the upsampling module in each processing layer. Finally, the convolution operation after each upsampling module is removed to form an improved 3D U-Net structure. The improved 3D U-Net structure is used as a generator for generative adversarial networks (GANs).

[0009] The generator's low-frequency feature extraction module extracts features from the PET image to be processed, and then stitches the extracted low-frequency features with the original PET image to obtain a PET feature composite image; the generator's high-frequency feature extraction module extracts features from the CT image to be processed, and then stitches the extracted high-frequency features with the original CT image to obtain a CT feature composite image.

[0010] The generator is trained by combining PET and CT feature images, enabling it to learn and adjust the information weights of PET / CT images, thus obtaining an image fusion model for fusing medical images.

[0011] The PET and CT images to be processed are input into the image fusion model. A low-frequency feature extraction module obtains a combined PET feature image, and a high-frequency feature extraction module obtains a combined CT feature image. The combined PET and CT feature images are downsampled using a downsampling module in each processing layer to obtain PET and CT feature maps. A controllable adjustment module adjusts the fusion ratio between the PET and CT feature maps. MFR According to the fusion ratio MFR The PET and CT feature maps are added together. This added image is then stitched together with the upsampled image obtained from the previous upsampling module to produce a stitched image containing PET and CT features at different levels. This stitched image is then upsampled again by the current upsampling module. Finally, the output from the last upsampling module represents the PET and CT images to be processed at different fusion ratios. MFR Fusion images from the fusion process.

[0012] Optionally, the generator is connected to a discriminator of a generative adversarial network (GAN) that includes a splicing operation (Cat) and multiple 4×4 convolutional layers;

[0013] The fused image generated by the generator, the original PET image, and the original CT image are stitched together by the stitching operation Cat to obtain a multimodal fused image; the authenticity of the fused image is obtained by jointly judging the multimodal fused image through multiple convolutional layers.

[0014] Optionally, both the low-frequency feature extraction module and the high-frequency feature extraction module include a Fourier transform module and a splicing operation Cat;

[0015] The PET image is subjected to Fourier transform by the Fourier transform module in the low-frequency feature extraction module to obtain the low-frequency features of the PET image. The low-frequency features and the original PET image are then stitched together by the stitching operation Cat in the low-frequency feature extraction module to obtain the PET feature combined image.

[0016] The CT images are subjected to Fourier transform by the Fourier transform module in the high-frequency feature extraction module to obtain the high-frequency features of the CT images. The high-frequency features and the original CT images are then stitched together by the stitching operation Cat in the high-frequency feature extraction module to obtain a combined CT feature image.

[0017] Optionally, size standardization and intensity normalization preprocessing operations are performed before inputting the PET and CT images to be processed into the image fusion model;

[0018] Size normalization means resampling and normalizing the PET and CT images to be processed to a uniform resolution and size, while intensity normalization means scaling the pixel values ​​of the PET and CT images to be processed to [0, 1].

[0019] Optionally, the last layer upsampling module outputs a representation of the PET image and CT image to be processed at different fusion ratios. MFR The fused images in the next fusion process specifically include:

[0020] According to the fusion ratio MFR The PET feature map and the CT feature map are added together to obtain a stitched image of the PET feature map and the CT feature map; the formula for the addition process is: [PET feature map * MFR + CT feature map * (1 - MFR)];

[0021] The summed image and the upsampled image obtained from the previous upsampling module are stitched together to obtain a stitched image containing PET and CT features at different levels.

[0022] The stitched image is then upsampled by the upsampling module in this layer to achieve convolution operation, resulting in a fused image of the PET image and the CT image to be processed.

[0023] Optionally, it also includes a loss function, specifically including:

[0024] Generator overall loss L G The formula is:

[0025] ;

[0026] Progressive fusion loss L PF The formula is:

[0027] ;

[0028] Discriminator loss L D The formula is:

[0029] ;

[0030] in, L adv This represents the discriminator's loss in discriminating between the fused images. x Represents PET images, y Represents CT images, f This represents the expected progressively fused image controlled by a controllable adjustment module, where λ represents the weights of the loss function. MFR Indicates the fusion ratio. G Represents a generator model. D This represents the discriminator model.

[0031] This invention also provides a progressive fusion system for functional medical images, comprising:

[0032] The image acquisition module is used to acquire PET and CT images to be processed.

[0033] A generator building module is used to add low-frequency feature extraction modules for processing PET images and high-frequency feature extraction modules for processing CT images to the input of the 3D convolutional neural network 3D U-Net. Each processing layer of the 3D convolutional neural network 3D U-Net includes two downsampling modules and a corresponding upsampling module. A concatenation operation Cat is added between adjacent upsampling modules. A controllable adjustment module is added between the concatenation operation Cat at the output of the two downsampling modules and the input of the upsampling module in each processing layer. Finally, the convolution operation after each upsampling module is removed to form an improved 3D U-Net structure, which is used as a generator for generative adversarial networks (GANs).

[0034] The feature combination module is used to extract features from the low-frequency feature extraction module of the generator to process the PET image, and then stitch the extracted low-frequency features with the original PET image to obtain a PET feature combination image; the high-frequency feature extraction module of the generator is used to extract features from the CT image to process, and then stitch the extracted high-frequency features with the original CT image to obtain a CT feature combination image.

[0035] The model training module is used to train the generator using PET feature-combined images and CT feature-combined images, enabling the generator to learn and adjust the information weights of PET / CT images to obtain an image fusion model for fusing medical images.

[0036] The image fusion module inputs PET and CT images into the image fusion model. It obtains combined PET feature images through a low-frequency feature extraction module and combined CT feature images through a high-frequency feature extraction module. The downsampling modules in each processing layer downsample the combined PET and CT feature images to obtain PET and CT feature maps. A controllable adjustment module adjusts the fusion ratio between the PET and CT feature maps. MFR According to the fusion ratio MFR The PET and CT feature maps are added together. This added image is then stitched together with the upsampled image obtained from the previous upsampling module to produce a stitched image containing PET and CT features at different levels. This stitched image is then upsampled again by the current upsampling module. Finally, the output from the last upsampling module represents the PET and CT images to be processed at different fusion ratios. MFR Fusion images from the fusion process.

[0037] The progressive fusion method and system for functional medical images provided in this invention have the following advantages compared with the prior art:

[0038] This invention improves the generator of Generative Adversarial Networks (GANs) by introducing a controllable adjustment module into the 3D convolutional neural network 3D U-Net. The controllable adjustment module can adjust the fusion ratio between PET feature maps and CT feature maps. MFR This allows the generated fused image to represent the PET and CT images to be processed at different fusion ratios. MFR The fusion process allows for observation of the gradual details of metabolic and anatomical information in lesions, making it suitable for the multimodal fusion needs of various functional medical images. Attached Figure Description

[0039] Figure 1 This is an overall flowchart of a progressive fusion method for functional medical images in PET / CT, provided in one embodiment.

[0040] Figure 2 A flowchart of PET / CT progressive fusion training for a progressive fusion method of functional medical images provided in one embodiment;

[0041] Figure 3 This is a generator structure diagram of a progressive fusion method for functional medical images provided in one embodiment;

[0042] Figure 4 This is a diagram of the discriminator structure of a progressive fusion method for functional medical images provided in one embodiment;

[0043] Figure 5 This is a flowchart illustrating the PET / CT progressive fusion generation process of a progressive fusion method for functional medical images provided in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0045] In one embodiment, a progressive fusion method for functional medical images is provided, the method comprising:

[0046] Acquire PET and CT images to be processed.

[0047] A low-frequency feature extraction module for processing PET images and a high-frequency feature extraction module for processing CT images are added to the input of the 3D convolutional neural network 3D U-Net. Each processing layer of the 3D convolutional neural network 3D U-Net includes two downsampling modules and a corresponding upsampling module. A concatenation operation Cat is added between adjacent upsampling modules. A controllable adjustment module is added between the concatenation operation Cat at the output of the two downsampling modules and the input of the upsampling module in each processing layer. Finally, the convolution operation after each upsampling module is removed to form an improved 3D U-Net structure. The improved 3D U-Net structure is used as a generator for generative adversarial networks (GANs).

[0048] The generator's low-frequency feature extraction module extracts features from the PET image to be processed, and then stitches the extracted low-frequency features with the original PET image to obtain a PET feature composite image. Similarly, the generator's high-frequency feature extraction module extracts features from the CT image to be processed, and then stitches the extracted high-frequency features with the original CT image to obtain a CT feature composite image.

[0049] The generator is trained by combining PET and CT feature images, enabling it to learn and adjust the information weights of PET / CT images, resulting in an image fusion model for fusing medical images.

[0050] The PET and CT images to be processed are input into the image fusion model. A low-frequency feature extraction module obtains a combined PET feature image, and a high-frequency feature extraction module obtains a combined CT feature image. The combined PET and CT feature images are downsampled using a downsampling module in each processing layer to obtain PET and CT feature maps. A controllable adjustment module adjusts the fusion ratio between the PET and CT feature maps. MFR According to the fusion ratio MFR The PET and CT feature maps are added together. This added image is then stitched together with the upsampled image obtained from the previous upsampling module to produce a stitched image containing PET and CT features at different levels. This stitched image is then upsampled again by the current upsampling module. Finally, the output from the last upsampling module represents the PET and CT images to be processed at different fusion ratios. MFR Fusion images from the fusion process.

[0051] This invention mainly consists of four key steps: data preprocessing, frequency feature extraction, dual-branch fusion training, and progressive fusion generation. The overall flowchart is shown below. Figure 1 As shown.

[0052] 1. Data Preprocessing

[0053] Before model training, PET / CT image data is preprocessed to ensure the model can learn effectively on consistent, standardized image data, thereby generating high-quality fused images. The data preprocessing steps specifically include the following aspects:

[0054] (1) Extraction of the thoracic cavity region: In order to focus on key anatomical regions and improve the fusion effect, this invention extracts the thoracic cavity region from PET / CT image data. Extraction of the thoracic cavity region helps to reduce the interference of irrelevant background information, making the model more focused on the lesions, tumors and other areas of interest.

[0055] (2) Size standardization: PET / CT image data generated by different imaging devices have different resolutions and sizes. In order to ensure the consistency of the model input data, this invention resamples and standardizes all image data to a uniform resolution and size (128×128×128 pixels) to ensure that the model is not disturbed by size differences during training.

[0056] (3) Intensity normalization: Since the pixel value range of PET / CT images varies greatly, in order to further improve the model training effect, this invention performs intensity normalization on each image type and scales the pixel value to the range of [0, 1].

[0057] 2. Frequency Domain Feature Extraction

[0058] To extract different features from PET / CT images, this invention introduces a frequency separation method. This method uses 3D Fourier transform to decompose the image into high-frequency and low-frequency components, and then combines these components with the original image to preserve the key features of PET / CT.

[0059] Low-frequency feature extraction: Low-frequency components represent the overall structural information of an image. This invention stitches the low-frequency components of a PET image with its original image to preserve the metabolic activity characteristics of the PET image. The PET image is subjected to a Fourier transform by the Fourier transform module in the low-frequency feature extraction module to obtain its low-frequency features. The low-frequency features and the original PET image are then stitched together using the stitching operation (Cat) in the low-frequency feature extraction module to obtain a combined PET feature image.

[0060] High-frequency feature extraction: High-frequency components mainly contain edge and detail information of the image. This invention stitches the high-frequency components of the CT image with the original image to enhance the anatomical structure information of the CT image. The CT image is subjected to Fourier transform by the Fourier transform module in the high-frequency feature extraction module to obtain the high-frequency features of the CT image. The high-frequency features and the original CT image are then stitched together by the stitching operation Cat in the high-frequency feature extraction module to obtain a combined CT feature image.

[0061] Combined input: After frequency separation processing, the combined PET / CT feature images are used as dual-branch inputs to the model, which enhances the feature representation capability of each modality.

[0062] 3. Dual-branch fusion training

[0063] To learn the gradient characteristics of PET / CT images at different fusion ratios, this invention designs a dual-branch structure in the GAN generator. By introducing different MFRs during training, it achieves multi-ratio learning and fusion. The training flowchart is as follows. Figure 2 As shown.

[0064] Generator Design: The generator employs an improved 3D U-Net architecture. In the encoding stage, two independent input branches are designed to receive stitched PET / CT images respectively. In the decoding stage, MFR adjustment is implemented to control the fusion ratio of the PET / CT images. MFR .

[0065] MFR Design: During decoding at each layer of the generator, the MFR is adjusted (e.g., 0.2, 0.4, 0.6, etc.), and the PET and CT features from the corresponding encoding stage are fused using a weighted sum of feature maps, i.e., [PET feature map * MFR + CT feature map * (1-MFR)]. This is then followed by a skip connection with the upsampled feature map for 3D convolution to preserve the characteristics of each modality and achieve controllable feature ratio adjustment. The generator structure diagram is shown below. Figure 3 As shown.

[0066] Discriminator Design: The discriminator includes a stitching operation (Cat) and multiple 4×4 convolutional layers. The stitching operation (Cat) stitches together the fused image generated by the generator, the original PET image, and the original CT image to obtain a multimodal fused image. Multiple convolutional layers then jointly discriminate the multimodal fused image to determine its authenticity. The discriminator structure diagram is shown below. Figure 4 As shown.

[0067] 4. Gradual fusion generation

[0068] During the testing phase, by adjusting the MFR (Mean Fractional Frame), PET / CT fused images of different scales can be generated, thus creating a progressive PET / CT fusion process. The generation flowchart is as follows: Figure 5 As shown.

[0069] Progressive fusion process: During the testing phase, by adjusting the MFR (such as 0.1, 0.3, 0.5, 0.7, etc.), the generator can generate continuous progressively fused images between PET / CT images, resulting in an image sequence that gradually transitions from CT features to PET features.

[0070] Dynamic adjustability: Doctors can select the required fusion ratio according to different diagnostic needs. MFR Adjust the fusion ratio MFR To view image details at different feature scales in PET / CT.

[0071] 5. Loss Function

[0072] Unlike traditional generative models that perform single image fusion using pixel-level correlation loss, this invention proposes an innovative progressive fusion loss (…). L PF By adjusting the MFR, progressive fusion generation of PET / CT was effectively achieved.

[0073] Commonly used losses in generative models include, but are not limited to, L1 similarity loss. L L1 L2 similarity loss L L2 The loss is constructed using L1 similarity loss as an example. L1 similarity loss is shown in formula (1).

[0074] ; (1)

[0075] in, x Represents PET images, y Represents a CT image.

[0076] Progressive fusion loss L PF As shown in formula (2):

[0077] ; (2)

[0078] in, f Indicates the fusion ratio MFR The expected progressive fusion of images under control, MFR Indicates the fusion ratio.

[0079] Adding a discriminator to the fused image discrimination loss L adv The overall loss of the complete generator is shown in Equation (3):

[0080] ; (3)

[0081] Where λ represents the weight of the loss function, G Represents a generator model. D This represents the discriminator model.

[0082] The discriminator aims to distinguish whether the input fused image is real or generated by the generator. Through adversarial learning, the discriminator loss guides the system to continuously optimize, outputting a high confidence score for real fused images and a low confidence score for generated fused images. The discriminator loss is shown in formula (4):

[0083] ; (4)

[0084] The effects of this invention are analyzed as follows:

[0085] (1) Traditional image fusion methods use fixed ratio fusion, which limits the doctor's ability to flexibly adjust between different proportions of PET / CT information, making it difficult to generate gradual images with arbitrary proportions between PET / CT.

[0086] This invention introduces MFR (Mean Fractional Feature Rendering) into GANs, enabling the model to learn fused image features at different proportions during the training phase. This allows for the generation of progressively fused images with arbitrary proportions between PET / CT images during the testing phase, achieving flexible control over PET / CT features. This technology allows physicians to customize the proportion of PET / CT features during image analysis, gaining more diagnostically valuable perspectives and providing more accurate image support for clinical diagnosis.

[0087] (2) Traditional methods are difficult to maintain the integrity and physiological accuracy of anatomical structures during the fusion process, which affects doctors’ judgment of the tissues and structures around the lesion.

[0088] This invention utilizes frequency separation and feature extraction techniques to extract and fuse low-frequency and high-frequency features from PET / CT images, effectively improving the structural consistency and detail representation of the generated images. Furthermore, feature analysis enhances the identification of tissue information surrounding lesions, ensuring that the generated fused images are not only visually clear but also conform to real physiological structures. This high-precision structure guarantees the reliability and consistency of functional medical image fusion.

[0089] Based on the same inventive concept, the present invention also provides a progressive fusion system for functional medical images, the system comprising:

[0090] The image acquisition module is used to acquire PET and CT images to be processed.

[0091] The generator building module adds low-frequency feature extraction modules for PET images and high-frequency feature extraction modules for CT images to the input of the 3D convolutional neural network 3D U-Net. Each processing layer of the 3D U-Net includes two downsampling modules and a corresponding upsampling module. A concatenation operation Cat is added between adjacent upsampling modules. A controllable adjustment module is added between the concatenation operation Cat at the output of the two downsampling modules and the input of the upsampling module in each processing layer. Finally, the convolution operation after each upsampling module is removed to form an improved 3D U-Net structure, which is then used as a generator for generative adversarial networks (GANs).

[0092] The feature combination module extracts features from the low-frequency feature extraction module of the generator into the PET image to be processed, and then stitches the extracted low-frequency features with the original PET image to obtain a PET feature combination image. Similarly, the high-frequency feature extraction module of the generator extracts features from the CT image to be processed, and then stitches the extracted high-frequency features with the original CT image to obtain a CT feature combination image.

[0093] The model training module is used to train the generator using PET feature-combined images and CT feature-combined images, enabling the generator to learn and adjust the information weights of PET / CT images to obtain an image fusion model for fusing medical images.

[0094] The image fusion module inputs PET and CT images into the image fusion model. It obtains a combined PET feature image through a low-frequency feature extraction module and a combined CT feature image through a high-frequency feature extraction module. Downsampling modules in each processing layer downsample the combined PET and CT feature images to obtain PET and CT feature maps. A controllable adjustment module adjusts the fusion ratio between the PET and CT feature maps. MFR According to the fusion ratio MFR The PET and CT feature maps are added together. This added image is then stitched together with the upsampled image obtained from the previous upsampling module to produce a stitched image containing PET and CT features at different levels. This stitched image is then further upsampled by the current upsampling module. Finally, the output of the last upsampling module represents the PET and CT images to be processed at different fusion ratios. MFR Fusion images from the fusion process.

[0095] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A functional medical image progressive fusion method, characterized in that, The application relates to a method for fusing PET / CT images. The method comprises the following steps: acquiring PET images and CT images to be processed; adding a low-frequency feature extraction module for processing the PET images and a high-frequency feature extraction module for processing the CT images at the input end of a three-dimensional convolutional neural network 3D U-Net; each processing layer of the three-dimensional convolutional neural network 3D U-Net comprises two down-sampling modules and one corresponding up-sampling module, a splicing operation Cat is added between adjacent up-sampling modules, and a controllable adjustment module is added between the splicing operation Cat of the output end of the two down-sampling modules and the input end of the up-sampling module in each processing layer; finally, the convolution operation after each up-sampling module is removed to form an improved 3D U-Net structure, and the improved 3D U-Net structure is used as a generator of a generative adversarial network GANs; the low-frequency feature extraction module of the generator is used for extracting features of the PET images to be processed, and the extracted low-frequency features are spliced with the original PET images to obtain PET feature combination images; the high-frequency feature extraction module of the generator is used for extracting features of the CT images to be processed, and the extracted high-frequency features are spliced with the original CT images to obtain CT feature combination images; The PET image to be processed and the CT image are input into the image fusion model, the PET feature combined image is obtained through the low-frequency feature extraction module, and the CT feature combined image is obtained through the high-frequency feature extraction module; the PET feature combined image and the CT feature combined image are down-sampled through the down-sampling module in each processing layer to obtain the PET feature map and the CT feature map; the fusion ratio between the PET feature map and the CT feature map is adjusted through the controllable adjustment module the PET feature combination images and the CT feature combination images are used for training the generator, so that the generator learns to adjust the information weight of the PET / CT images, and an image fusion model for fusing medical images is obtained; , according to the fusion ratio MFR , the PET feature map and the CT feature map are added, the added image is spliced with the up-sampling image obtained by the up-sampling module of the previous layer to obtain a spliced image containing PET features and CT features at different levels; the spliced image is up-sampled through the up-sampling module of the current layer, and the fusion image representing the fusion process of the PET image to be processed and the CT image to be processed at different fusion ratios MFR is output through the up-sampling module of the last layer.

2. A functional medical image progressive fusion method according to claim 1, characterized in that, MFR the generator is connected with a discriminator of the generative adversarial network GANs which comprises a splicing operation Cat and a plurality of 4*4 convolutional layers; 3. A functional medical image progressive fusion method according to claim 1, wherein, the fusion image generated by the generator, the original PET image and the original CT image are spliced through the splicing operation Cat to obtain a multi-modal fusion image; the multi-modal fusion image is jointly discriminated through the plurality of convolutional layers to obtain the authenticity of the fusion image. The low-frequency feature extraction module and the high-frequency feature extraction module both comprise a Fourier transform module and a splicing operation Cat; the Fourier transform module in the low-frequency feature extraction module is used for performing Fourier transform on the PET images to obtain low-frequency features of the PET images, and the splicing operation Cat in the low-frequency feature extraction module is used for splicing the low-frequency features with the original PET images to obtain the PET feature combination images; 4. The functional medical image progressive fusion method of claim 1, wherein, the Fourier transform module in the high-frequency feature extraction module is used for performing Fourier transform on the CT images to obtain high-frequency features of the CT images, and the splicing operation Cat in the high-frequency feature extraction module is used for splicing the high-frequency features with the original CT images to obtain the CT feature combination images. Before the PET images and the CT images to be processed are input into the image fusion model, a preprocessing operation of size standardization and intensity normalization is performed; 5. The functional medical image progressive fusion method of claim 1, wherein, The output of the last layer up-sampling module represents the to-be-processed PET image and the to-be-processed CT image at different fusion ratios the size standardization means that the PET images and the CT images to be processed are resampled and standardized to a unified resolution and size, and the intensity normalization means that the pixel values of the PET images and the CT images to be processed are scaled to [0, 1]. The fusion image of the down fusion process specifically comprises: According to the fusion ratio MFR The PET feature map and the CT feature map are added to obtain a spliced image of the PET feature map and the CT feature map; a formula of the adding process is: [PET feature map*MFR+CT feature map*(1-MFR)]. MFR the added image and the up-sampling image obtained through the up-sampling module of the last layer are subjected to a splicing operation to obtain a splicing image containing PET features and CT features of different levels; The spliced image is up-sampled by a layer up-sampling module to realize convolution operation, so as to obtain a fusion image of the to-be-processed PET image and the to-be-processed CT image.

6. A functional medical image progressive fusion method as claimed in claim 1, characterized in that, Further comprising a loss function, specifically comprising: Generator overall loss L G The formula is: ； wherein the progressive fusion loss L PF The formula is: ； discriminator loss L D The formula is: ； wherein, L adv represents the discriminator loss of the fusion image, x represents the PET image, y represents the CT image, f represents the expected progressive fusion image controlled by the controllable adjustment module, λ represents the loss function weight, MFR represents the fusion ratio, G represents the generator model, D represents the discriminator model.

7. A functional medical image progressive fusion system, characterized by, Comprising: An image acquisition module is configured to acquire a to-be-processed PET image and a CT image; A generator construction module is configured to add a low-frequency feature extraction module for processing the PET image and a high-frequency feature extraction module for processing the CT image at an input end of a three-dimensional convolutional neural network (3D U-Net); each processing layer of the three-dimensional convolutional neural network (3D U-Net) comprises two down-sampling modules and one corresponding up-sampling module, a concatenation operation (Cat) is added between adjacent up-sampling modules, a controllable adjustment module is added between the concatenation operation (Cat) at the output end of the two down-sampling modules and the input end of the up-sampling module in each processing layer; finally, the convolution operation after each up-sampling module is removed to form an improved 3D U-Net structure, and the improved 3D U-Net structure is used as a generator of a generative adversarial network (GANs); A feature combination module is configured to extract features from the to-be-processed PET image by the low-frequency feature extraction module of the generator, concatenate the extracted low-frequency features with the original PET image to obtain a PET feature combination image, extract features from the to-be-processed CT image by the high-frequency feature extraction module of the generator, and concatenate the extracted high-frequency features with the original CT image to obtain a CT feature combination image; A model training module is configured to train the generator by using the PET feature combination image and the CT feature combination image, so that the generator learns to adjust the information weight of the PET / CT image, and obtains an image fusion model for fusing medical images; The image fusion module is used for inputting the PET image and the CT image to be processed into an image fusion model, acquiring a PET feature combined image through a low-frequency feature extraction module, and acquiring a CT feature combined image through a high-frequency feature extraction module; the PET feature combined image and the CT feature combined image are down-sampled through a down-sampling module in each processing layer to obtain a PET feature map and a CT feature map; the fusion ratio between the PET feature map and the CT feature map is adjusted through a controllable adjustment module MFR , the PET feature map and the CT feature map are added according to the fusion ratio MFR , the added image is spliced with a up-sampling image obtained by a up-sampling module of the last layer to obtain a spliced image containing PET features and CT features at different levels; and the spliced image is up-sampled through a up-sampling module of the current layer, and a fusion image representing a fusion process of the PET image to be processed and the CT image to be processed at different fusion ratios MFR is output through a up-sampling module of the last layer.

Images