Deep recursive medical image fusion method based on self-supervised generative distillation
By combining teacher and student networks and employing dense multi-resolution preservation and deep recursive attention techniques, the problem of poor performance of self-supervised learning on small models is solved, achieving efficient multi-resolution feature extraction and deep fusion, thus improving the quality and efficiency of medical image fusion.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- YUNNAN UNIV
- Filing Date
- 2025-05-18
- Publication Date
- 2026-06-23
AI Technical Summary
Existing self-supervised learning performs poorly on small models and is difficult to run on mobile devices or embedded systems. Furthermore, most medical image fusion models have single feature resolutions and cannot fully integrate multi-resolution semantic information. Cross-attention mechanisms also have limitations in the deep fusion of shallow and deep features at different scales.
By combining teacher and student networks, and through dense multi-resolution maintenance networks and unimodal deep recurrent attention, multi-resolution feature extraction and deep fusion are achieved. Self-supervised training and generative distillation techniques are used to improve knowledge transfer efficiency. Feature integration is performed by combining multimodal deep recurrent fusion networks, thus constructing a deep recurrent medical image fusion method based on self-supervised generative distillation.
It improves the quality of medical image fusion, enhances knowledge transfer efficiency, achieves high-quality multimodal image fusion, improves the ability to preserve image details and edge information, and significantly improves visual effects.
Smart Images
Figure CN120543393B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image fusion technology, specifically a deep recursive medical image fusion method based on self-supervised generative distillation. Background Technology
[0002] With the continuous development of medical imaging technology, various imaging modalities such as X-rays, computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and single-photon emission computed tomography (SPECT) have become important tools for clinical diagnosis, playing a crucial role in medical diagnosis, treatment planning, and surgical navigation. Different imaging modalities, due to their different imaging principles, display different angles of human tissue and pathological information. However, the information provided by a single modality of medical images is limited and often insufficient to provide sufficient pathological evidence for accurate diagnosis.
[0003] Medical images are generally divided into two categories: anatomical images and functional images. CT images have high resolution and can easily distinguish the internal structure of organs, but their function is limited in soft tissue imaging. MRI images provide excellent soft tissue contrast and can better show tumors or tissue abnormalities in different parts of the body, but MRI images lack signals from cortical bone. PET images show color-accumulated shadows in the lesion area and can show color information changes, but cannot distinguish the location of individual organs and tissues. Therefore, multimodal medical image fusion technology has emerged, aiming to effectively fuse information provided by different imaging modalities to overcome the limitations of single-modal images and provide healthcare providers with more complete and accurate information in clinical decision-making.
[0004] Currently, deep learning methods, with their powerful feature representation capabilities, have been widely applied in medical image fusion within computer vision tasks. Deep learning-based methods can be categorized into unsupervised and supervised medical image fusion based on model generation. Supervised medical image fusion methods rely on expensive and scarce labeled ground truth data, significantly limiting their widespread adoption in practice. To address this issue, researchers have gradually shifted towards utilizing the inherent information within images to generate high-quality fused images through unsupervised image fusion techniques. In this approach, self-supervised learning (SSL) automatically extracts supervisory signals from massive amounts of unlabeled data by designing a series of auxiliary tasks. This allows the model to effectively learn and extract the feature representations required for downstream tasks without the need for expensive labeled data. Therefore, self-supervised learning has become a typical example in the field of unsupervised learning and has shown great application potential in medical image fusion. However, while SSL-based methods perform well on large models, they perform poorly on small models, making them difficult to run on mobile devices or embedded systems. Model compression is necessary for compatibility.
[0005] To address these issues, some scholars have proposed a technique combining self-supervised learning (SSL) and knowledge distillation (KD). This involves pre-training a large model using self-supervision and then transferring its trained proprietary features to a smaller model, thus mitigating the inherent limitations of self-supervision. However, most self-supervised knowledge distillation relies on students imitating teachers, and this inefficient cross-modal knowledge transfer negatively impacts the fusion task. Furthermore, feature extraction and fusion are crucial components determining the quality of the fused image. However, most current networks use a single resolution, failing to capture multi-resolution semantic information and achieve feature reuse, resulting in suboptimal image fusion performance. Additionally, while the widely used cross-attention mechanism can integrate cross-modal features, it has limitations in the deep fusion of shallow and deep features at different scales. Summary of the Invention
[0006] This invention aims to address the problems of low knowledge transfer efficiency in traditional self-supervised distillation and insufficient fusion due to the limited feature resolution of most fusion models. To this end, this invention provides a deep recursive medical image fusion method based on self-supervised generative distillation.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] The data model used in this invention includes three modules: a teacher network, a student network, and a converged network.
[0009] The teacher network module includes a feature extraction stage and a feature fusion stage; it performs self-supervised training to generate reconstructed images to guide student networks.
[0010] The student network includes a feature extraction stage and a feature fusion stage, which utilizes implicit neural networks to obtain prior knowledge from source images and combines it with generative distillation to improve the efficiency of knowledge transfer.
[0011] Fusion networks integrate proprietary features learned by student networks in a multimodal manner, ultimately outputting a fused image.
[0012] The operational steps of the teacher network include:
[0013] S1-1. Perform grayscale processing on multimodal images;
[0014] S1-2. A Dense Multi-Resolution Preservation Network (DMPB) is used to extract features from the processed image. The DMPB consists of four similar preservation blocks (MPBs). The horizontal preservation blocks (MPBs) vary in the number of channels but maintain the same resolution. The MPBs form a vertical pyramid structure with the number of channels remaining constant. High, medium, and low resolution features are output. The high, medium, and low resolutions are the original size, half the original size, and one-quarter of the original size, respectively. The high-resolution features output by the four preservation blocks (MPBs) are densely connected to aggregate multi-resolution features and achieve feature reuse.
[0015] S1-3. In the feature fusion stage, a single-modal deep recursive attention (DRCA2) is adopted. The single-modal deep recursive attention (DRCA2) is based on cross attention and recursively connects the beginning and end of high, medium and low resolution features, and iteratively refines them to achieve deep fusion of shallow and deep features.
[0016] S1-4. The iteratively generated features are fed into the cascaded decoder, and the output feature loss of the decoder is subjected to self-supervised training. The cascaded decoder consists of the high-resolution features output by 4 layers of hold-up blocks (MPB) and the outputs of 4 core reconstructive (CR) modules. Each CR module consists of ReLU and a 3×3 convolution. To generate a reconstructed image that approximates the original image, a teacher loss function is constructed by applying pixel loss, structural similarity index, gradient loss, and perceptual loss to the decoder output. Conduct self-monitoring training.
[0017] The student network includes the following steps:
[0018] S2-1. Perform grayscale processing on the multimodal image;
[0019] S2-2, Feature Extraction Stage: First, the implicit multi-resolution module (IMB) is used for coordinate mapping to obtain prior knowledge features. At the same time, three filters with a kernel size of 3 and strides of 1, 2 and 4 are used to obtain multi-resolution features. The generative distillation module (MGD) first performs a random masking operation on the multi-resolution features, and then uses the generator block to push the student network to reconstruct the features of the teacher network.
[0020] S2-3, Feature Fusion Stage: First, the features output by the generative distillation module are fused using a single-modal deep recursive fusion (DRCA2) to obtain fused features. Then, a non-cascaded decoder consisting of four CRs is used to process the fused features to obtain reconstructed features. To constrain the teacher-student network, the reconstructed features are also subjected to distillation constraints by the generative distillation module, which are defined as pixel-level distillation loss. During distillation training, pixel-level and semantic distillation losses are combined to obtain the reconstructed image. Finally, the multi-resolution features recovered by the student network are output for use in subsequent tasks. The fusion network includes the following steps:
[0021] S3. The multi-resolution features learned through the student network are processed by the multimodal deep recursive fusion network (DRCA2 and DRCA1) to achieve deep fusion of deep and shallow features. The fused image is then obtained through a decoder composed of four cross-attention (CA) layers. To obtain high-quality fusion results, gradient loss is used. and fidelity loss Total fusion loss We perform fusion training to obtain edge detail information from the source image.
[0022] Preferably, step S1-1 performs grayscale processing on two multimodal images a and b. and The steps in S1-2 are as follows:
[0023] Feature extraction is performed only on image a, using four preserver blocks: MPB0, MPB1, MPB2, and MPB3. First, high-resolution (L), mid-resolution (M), and low-resolution (S) features of image a's modality are obtained from MPB0. Then through the remaining three MPB i (i = 1, 2, 3) respectively obtain the high, medium, and low resolution features of mode a corresponding to the MPB output. Finally, the high-resolution features output from each layer of the Dense Multi-Resolution Preservation Network (DMPB) are reused through fully connected layers. The overall process of the Dense Multi-Resolution Preservation Network (DMPB) can be represented as follows:
[0024]
[0025] The MPB (Preservation Block) consists of channel attention and a U-shaped convolutional neural network. First, a 1×1 filter and channel attention are used to enhance the channel features. Then, two sets of 4×4 filters are combined with channel attention to perform downsampling twice. Finally, two sets of 1×1 filters are combined with ReLU pooling and concatenated with the downsampled features. Concat(·) represents the concatenation function along the channel dimension.
[0026] Preferably, the specific steps for self-supervised training of the decoder's output feature loss in steps S1-4 are as follows:
[0027] S1-4-1. The teacher network provides multi-resolution feature soft labels for the student network; therefore, it is crucial to adopt an appropriate loss function to constrain the reconstructed image and the input image.
[0028] S1-4-2 Calculate the total network loss for teachers This invention employs pixel loss (MSE), structural similarity index (SSIM), gradient loss, and perceptual loss to construct a teacher loss function. MSE measures the difference between the output image and the input image, SSIM evaluates their structural similarity, gradient loss ensures that the output image is closer to the corresponding input image, and perceptual loss enhances the visual effect of the reconstructed image. These losses can reconstruct the structure, details, and perceptual features of the original image.
[0029]
[0030] in These represent pixel loss, structural similarity index, gradient loss, and perceptual loss, respectively. The balance factor representing the teacher network, and the various losses expressed by the formulas are:
[0031]
[0032] Where ||·||2 refers to the l2 norm, and O and I represent the output image and input image, respectively. It is the Euclidean distance between the features obtained by inputting the input and output images into the Vgg16 network; Θ m,n This represents the nth convolutional feature map before the mth max-pooling layer, where the preferred feature map is...
[0033] Preferably, in step S2-2, the feature extraction stage uses an implicit multi-resolution module (IMB) for coordinate mapping. The IMB mainly consists of an implicit neural network (INR) and three filters. The specific steps for obtaining prior knowledge features are as follows:
[0034] S2-2-1, Input image I aThe coordinates of the original image and the shallow features INB are extracted by feeding them into the implicit neural network INR. After being calculated by the concatenation function Concat(·) along the channel dimension, the mixed coordinates (x, y) are obtained.
[0035] S2-2-2. The mixed coordinates (x, y) are mapped to the RGB space using a Multilayer Perceptron (MLP) to extract prior information such as color and contour. S2-2-3. The features processed by the Implicit Neural Network (INR) are decomposed into high-, medium-, and low-resolution features of modality a in the student network using three filters with 3 convolutional kernels and strides of 1, 2, and 4. Similarly, the high, medium, and low resolution features of the modality in Figure b were obtained. The structures of INR and INB can be represented as follows:
[0036] INR = MLP(Concat(INB(I a ),corrdinates(I a )))
[0037] INB = Conv 3×3 (Sigmoid(GN(Conv 3×3 (Sigmoid(GN(Conv 3×3 (I a )))))+I a
[0038] Where coordinates(·) represents a coordinate function, Concat(·) represents a function that concatenates along the channel dimensions, and Conv 3×3 It is a convolution with a kernel of 3, GN is group normalization, and Sigmoid is the activation function. MLP extracts prior knowledge during the mapping process, and has some information about the source image without retraining during distillation training, which is beneficial for knowledge transfer in generative distillation.
[0039] Preferably, since pixels in a neural network contain information about neighboring pixels, even if a portion is occluded, the pixel's semantics can still be inferred from the context. Based on this idea, the specific steps of distillation training in S2-3 of this invention are as follows:
[0040] S2-3-1. Set an arbitrary mask M for the multi-resolution characteristics of the student network. h,w The mask features are obtained after multiplying with the multi-resolution features of the student network.
[0041] S2-3-2. Use the generating function G(·) to enable the student network to generate complete features of the teacher, where the generating function consists of a ReLU and two convolutional layers W with 3×3 kernels. n1 and W n2composition;
[0042] The mask and the generator block can be represented as:
[0043]
[0044] I' S =I S ⊙M h,w
[0045] G(I′ S = Relu(W) n2 (Relu(W n1 (I′ s ))))
[0046] Where h and w are the horizontal and vertical coordinates of the feature map, and R h,w Let η be a random number in the range (0,1), and let η be a hyperparameter of the masking ratio. Experimental results show that the higher the masking ratio, the better the student feature recovery effect. However, when the masking ratio exceeds a certain threshold, the teacher features cannot be completely recovered. Tests showed that the recovery effect was best when η = 0.5.
[0047] S2-3-3, The generated complete teacher features are processed by semantic distillation loss. and pixel-level distillation loss Composition After distillation training, high, medium, and low resolution features were recovered. and output recovery features The specific formula for distillation loss is expressed as follows:
[0048]
[0049] Where j represents the logarithm of the distillation characteristics, I T and I′ S represents the soft label resolution features generated by the teacher network and the mask features of the student, respectively. MSE is the L2-loss, and C, H, and W are the number of feature channels, height, and width.
[0050] Preferably, the fusion process of the multimodal deep recursive fusion networks DRCA2 and DRCA1 in S3 is as follows:
[0051] S3-1. Multimodal deep recursive attention (DRCA2) is used to iteratively fuse features of different resolutions in the student network through cross-attention, resulting in a high-low resolution fused feature I. F1 ,I F2 ,I F3 ;
[0052] S3-2, In order to unify the resolution of features so that subsequent processing can be performed on I F2I was obtained by performing a 2x upsampling. F2↑ ; to I F3 I was obtained by upsampling by 4 times F3↑ ;
[0053] S3-3. Using Single-Modal Deep Recursive Attention (DRCA), the features processed as described above are fused at the modal level to achieve deep and shallow fusion; thereby further mining and integrating information from different modal features and improving the overall model performance.
[0054] The calculation formulas for DRCA2 and DRCA1 are as follows:
[0055]
[0056]
[0057] K represents the number of iterations, with K=4 being the preferred value; These represent the high-resolution features of the graph a modality and graph b modality input into the cross-attention module at the (K-1)th iteration, respectively. This represents the high-resolution fused features input into the cross-attention module during the (K-1)th iteration.
[0058] S3-4. The fused features after passing through the multi-modal deep recurrent fusion network are input into the non-cascaded decoder to obtain the fused image, which is crucial for obtaining a high-quality fused image I. f Using gradient loss and fidelity loss Total fusion loss The specific formula for fusion training is as follows:
[0059]
[0060] λ is the fidelity loss. and gradient loss The optimal balance factor is λ = 6; It can be represented as:
[0061]
[0062] Where I a I b For the source images of mode a and mode b in Figure 1, I f For image fusion, H and W are the length and width of the source images, and ||·||1 is the l1 norm. Representative source image I a I b The relevant weight values, represents the gradient operator, and |·| represents the absolute value operator.
[0063] The beneficial effects of this invention are:
[0064] (1) The teacher network adopts a dense multi-resolution preservation strategy, which makes the semantic information transmitted to the fused image richer;
[0065] (2) The teacher network, student network and fusion network all use deep recursive cross-attention fusion network to improve the deep fusion of features of different modalities and scales, thereby achieving high-quality fused images;
[0066] (3) The local interactions of cross-spatial feature blocks were refined, and the more global interactions of cross-modal channel groups were captured based on spatial similarity and channel similarity, respectively. Attached image description:
[0067] Figure 1 This is a network structure diagram of the RSGD model in an embodiment of the present invention;
[0068] Figure 2 This is a detailed structural diagram of the teacher network and the student network;
[0069] Figure 3 This is an iterative graph of the single-modal deep recursive attention (HRCA2) and multi-modal deep recursive attention (HRCA2) of this invention;
[0070] Figure 4 This is a qualitative analysis diagram of grayscale dataset fusion in an embodiment of the present invention;
[0071] Figure 5 This is a qualitative analysis diagram of grayscale dataset fusion in an embodiment of the present invention;
[0072] Figure 6 This is a qualitative analysis diagram of the color dataset fusion in an embodiment of the present invention; Detailed Implementation
[0073] The present invention will now be described in further detail with reference to specific embodiments.
[0074] Experimental Dataset: This invention uses the Harvard University publicly available brain dataset to train RSGD. This dataset contains seven grayscale image sets (CT / MR-T1, CT / MR-T2, MR-T1 / MR-T2, MR-T1 / MR-GAD, MR-T2 / MR-PD, MR-T1 / MR-PD, MR-T2 / MR-GAD) and three color image sets (MR-T2 / PET, MR-T2 / SPECT-TC, MR-T2 / SPECT-Ti). Both training and test images are 256×256 pixels in size, with 256 grayscale levels.
[0075] Evaluation metrics: To further visualize the objective performance of different methods, five commonly used objective evaluation metrics were used, including NMI, MI, QY, Qabf, and VIFP.
[0076] Experimental environment: The model was trained on the PyTorch platform using a single Ge-Force RTX 3090 GPU with 600 epochs and a batch size of 2. The Adam optimizer was used to optimize the framework parameters with a learning rate of 0.0001.
[0077] Comparison Method: To comprehensively demonstrate the superiority of our proposed method in image fusion tasks, this section will compare it with several state-of-the-art (SOTA) methods, including traditional sharpening methods (ConvCFL) and deep learning-based methods (DDcGAN, PPTFusion, MDRANT, SEAFusion, MRSCFusion, SSN-CAE+IE, GeSeNet, ALMFnet, EMMA, and MsgFusion). This comparison will highlight the advantages of our proposed method in handling different data types, further validating its feasibility and potential in practical applications.
[0078] Comparison and analysis of fusion results:
[0079] exist Figure 4 , 5 In Figure 6, we can see that the areas pointed to by the arrows in the CT / MR-T2 images are mainly the frontal sinus and corpus callosum. DDcGAN images exhibit severe artifacts. Traditional methods ConvCFL and ALMFnet show some improvement, but still have shortcomings. MDRANet and EMMA retain more bone density CT information, but the fused images are too bright and lack MRI structural information. PPTFusion and MsgFusion retain MRI structural information, but lose important CT density details. SSN-CAE+IE and MRSCFusion solve the brightness problem, but the texture quality is poor. SeAFusion and GeSeNet perform well, but their fine-grained and global feature extraction is insufficient. Compared with these methods, our method not only retains the features of the source image but also maintains fine-grained features, fusing rich MRI texture and PET / SPECT functional information, significantly improving the visual effect, especially in the preservation of edge information.
[0080]
[0081]
[0082]
[0083] Table 1. Quantitative results of different methods on different datasets.
[0084] Table 1 shows the performance of different methods when fusing 10 different datasets. For CT / MR-T1, our method performs better on NMI, MI, VIFP, and Q. Y The methods outperformed the best model by 8.45%, 7.27%, 24.03%, and 0.74% in terms of metrics, respectively, indicating that the proposed method effectively preserves source image information and performs excellently in terms of texture detail. In color image MR-T2 / PET, our method outperformed the suboptimal model on all five metrics, with improvements of 5.03% in NMI, 6.64% in MI, and Q... Y The improvement was 3.83%, Qabf was improved by 6.42%, and VIFP by 11.2%. This shows that our model effectively fuses color images with minimal loss of color information and clearly highlights disease locations. Overall, our method excels in detail and edge extraction capabilities, and the improved fusion results demonstrate that RSGD significantly enhances knowledge transfer.
[0085] In summary, the deep recurrent medical image fusion model RSGD based on self-supervised generative distillation in this application outperforms 11 other fusion methods used for comparison. Applying it to multimodal medical image fusion can assist doctors in making accurate diagnoses and treatments of lesions.
[0086] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.
Claims
1. A deep recursive medical image fusion method based on self-supervised generative distillation, characterized in that... The data model used includes three modules: teacher network, student network, and converged network. The teacher network module includes a feature extraction stage and a feature fusion stage; it performs self-supervised training to generate reconstructed images to guide student networks. The student network includes a feature extraction stage and a feature fusion stage, which utilizes implicit neural networks to obtain prior knowledge from source images and combines it with generative distillation to improve the efficiency of knowledge transfer. Fusion networks integrate proprietary features learned by student networks in a multimodal manner, ultimately outputting a fused image. The operational steps of the teacher network include: S1-1. Perform grayscale processing on multimodal images; S1-2. A Dense Multi-Resolution Preservation Network (DMPB) is used to extract features from the processed image. The DMPB consists of four similar preservation blocks (MPBs). The horizontal preservation blocks (MPBs) have different channels but the resolution remains constant. The MPBs form a vertical pyramid structure with the number of channels remaining constant. They output high, medium, and low resolution features. The high-resolution features output by the four MPBs are densely connected to aggregate multi-resolution features and achieve feature reuse. S1-3. In the feature fusion stage, a single-modal deep recursive attention (DRCA2) is adopted. The single-modal deep recursive attention (DRCA2) is based on cross attention and recursively connects the beginning and end of high, medium and low resolution features, and iteratively refines them to achieve deep fusion of shallow and deep features. S1-4. The iterative features are fed into the cascaded decoder, and the output feature loss of the decoder is subjected to self-supervised training. The cascaded decoder consists of the high-resolution features output by 4 layers of hold-up blocks (MPB) and the outputs of 4 CR modules. Each CR module consists of ReLU and 3×3 convolution. The feature loss includes: pixel loss (MSE), structural similarity index (SSIM), gradient loss, and perceptual loss. The student network includes the following steps: S2-1. Perform grayscale processing on the multimodal image; S2-2, Feature Extraction Stage: First, the implicit multi-resolution module (IMB) is used for coordinate mapping to obtain prior knowledge features. At the same time, three filters with a kernel size of 3 and strides of 1, 2 and 4 are used to obtain multi-resolution features. The generative distillation module (MGD) first performs a random masking operation on the multi-resolution features, and then uses the generator block to push the student network to reconstruct the features of the teacher network. S2-3, Feature Fusion Stage: First, the features output by the generative distillation module are fused using a single-modal deep recursive fusion (DRCA2) to obtain fused features. Then, a non-cascaded decoder consisting of four CRs is used to process the fused features to obtain reconstructed features. To constrain the teacher-student network, the reconstructed features are also subjected to distillation constraints by the generative distillation module, which are defined as pixel-level distillation loss. During distillation training, pixel-level and semantic distillation losses are combined to obtain the reconstructed image. Finally, the multi-resolution features recovered by the student network are output for use in subsequent tasks. Converged networks include the following steps: S3. The multi-resolution features learned through the student network are processed by the multimodal deep recursive fusion network (DRCA2 and DRCA1) to achieve deep fusion of deep and shallow features. The fused image is then obtained through a decoder composed of four cross-attention (CA) layers. To obtain high-quality fusion results, gradient loss is used. and fidelity loss Total fusion loss We perform fusion training to obtain edge detail information from the source image.
2. The deep recursive medical image fusion method based on self-supervised generative distillation according to claim 1, characterized in that: S1-1 refers to two multimodal images. and Perform grayscale processing; the steps in S1-2 are as follows: Images only Feature extraction is performed using four preserver blocks: MPB0, MPB1, MPB2, and MPB3. First, high-resolution (L), medium-resolution (M), and low-resolution (S) features of modality a in Figure are obtained from MPB0. , , Then through the remaining three The high, medium, and low resolution features of the corresponding MPB output mode a were obtained respectively. , , Finally, the high-resolution features output from each layer of the MPB are reused through the Dense fully connected layer. The overall process of the Dense Multiresolution Preservation Network (DMPB) can be represented as follows: ； ； ； ； The MPB (Multi-Level Block) consists of channel attention and a U-shaped convolutional neural network. First, a 1×1 filter and channel attention are used to enhance the channel features. Then, two sets of 4×4 filters are combined with channel attention to perform downsampling twice. Finally, two sets of 1×1 filters are combined with ReLU pooling and concatenated with the downsampled features. This represents a function that is concatenated along the channel dimension.
3. The deep recursive medical image fusion method based on self-supervised generative distillation according to claim 1, characterized in that: The specific steps for self-supervised training of the decoder's output feature loss in S1-4 are as follows: S1-4-1. The teacher network provides multi-resolution feature soft labels for the student network; S1-4-2 Calculate the total network loss for teachers : ； in , , , These represent pixel loss, structural similarity index, gradient loss, and perceptual loss, respectively. The balance factor representing the teacher network, and the various losses expressed by the formulas are: ； ； ； in It refers to Norms, O and I represent the output and input images, It is the Euclidean distance between features obtained by inputting the input and output images into the Vgg16 network; This represents the nth convolutional feature map before the mth max-pooling layer, where .
4. The deep recursive medical image fusion method based on self-supervised generative distillation according to claim 1, characterized in that: S2-1 refers to two multimodal images. and In the grayscale processing, the implicit multi-resolution module (IMB) is used for coordinate mapping in the feature extraction stage of S2-2 to obtain prior knowledge features. The specific steps are as follows: S2-2-1, Input image The data is fed into the implicit neural network INR to extract the coordinates of the original image and the shallow feature INB, which is then processed by a concatenation function along the channel dimension. The calculated mixed coordinates (x, y) are obtained. S2-2-2: The mixed coordinates (x, y) are mapped to the RGB space through a multilayer perceptron (MLP) to extract prior information such as color and contour. S2-2-3. The features processed by the implicit neural network INR are decomposed into high-, medium-, and low-resolution features of mode a in the student network using three filters with three convolutional kernels and strides of 1, 2, and 4, respectively. , , Similarly, the high, medium, and low resolution features of the mode in Figure b were obtained. , , .
5. The deep recursive medical image fusion method based on self-supervised generative distillation according to claim 1, characterized in that: The specific steps of the distillation training in S2-3 are as follows: S2-3-1. Set an arbitrary mask for the multi-resolution characteristics of the student network. The mask features are obtained after multiplying with the multi-resolution features of the student network. ; S2-3-2, Using Generating Functions The student network generates complete features of the teacher, where the generating function consists of a ReLU and two convolutional layers with 3×3 kernels. and composition; S2-3-3, The generated complete teacher features are processed by semantic distillation loss. and pixel-level distillation loss Composition After distillation training, high, medium, and low resolution features were recovered. , , and output recovery features ; The specific formula for distillation loss is expressed as follows: ； in The logarithm representing the characteristics of distillation. and These represent the resolution features of the soft labels generated by the teacher network and the mask features of the students, respectively. for C, H, and W are the number of channels, height, and width of the feature.
6. The deep recursive medical image fusion method based on self-supervised generative distillation according to claim 1, characterized in that: The fusion process of the multimodal deep recursive fusion networks DRCA2 and DRCA1 in S3 is as follows: S3-1. Multimodal deep recursive attention (DRCA2) is used to perform cross-attention iterative fusion of features at different resolutions in the student network to obtain high, medium, and low resolution fused features. , , ; S3-2, To 2x upsampling ;right 4x upsampling was performed to obtain ; S3-3. Use the features processed by the above-mentioned Single Modal Deep Recursive Attention (DRCA) to perform deep and shallow fusion at the modal level. The calculation formulas for DRCA2 and DRCA1 are as follows: ； ； K represents the number of iterations, K=4; , These represent the high-resolution features of the graph a modality and graph b modality input into the cross-attention module at the (K-1)th iteration, respectively. This represents the high-resolution fused features input into the cross-attention module during the (K-1)th iteration. S3-4, Utilizing gradient loss and fidelity loss Total fusion loss Perform fusion training and finally output the fused image; The specific formula is as follows: ； It is a loss of authenticity. and gradient loss The balance factor ; , It can be represented as: ； ； in , The source images for modes a and b are shown in Figure 1. To merge images, W represents the length and width of the source image. yes Norm, , Representative source image , The relevant weight values, Represents the gradient operator. Represents the absolute value operator.