Unified fusion method for multiple degraded medical images based on degraded prototype learning

Abstract

The present application relates to a multi-degradation medical image unified fusion method based on degradation prototype learning, belonging to the field of medical image fusion. The present application comprises: generating low-dose PET data, CT metal artifact data, and MRI data with motion artifacts through imaging principles; learning degradation prototypes using a feature selection mechanism; decomposing the basic fusion model into multiple branches through a low-rank decomposition strategy, which can process different branches when fusing different degraded data; designing a prompt module based on learnable feature prototypes, which promotes fusion by injecting degradation-related invariant features in different LoRa branches; and constructing a fused image through an output layer by integrating degradation-eliminating fusion features in different scales. The present application can effectively fuse medical images containing degradation, improving robustness and practicality in reality.

Description

TECHNICAL FIELD

[0001] The present application relates to a multi-degradation medical image unified fusion method based on degradation prototype learning, belonging to the technical field of medical image fusion. BACKGROUND

[0002] The existing multi-modal medical image fusion technology mainly generates images containing more comprehensive lesion information by integrating data of different imaging modalities. Most of these methods require high-quality source images without degradation as input to ensure that the final fusion can obtain good results. However, this requirement for the quality of the source image often cannot be met in real-world applications, because in reality, the acquisition of medical images will be disturbed by objective factors, resulting in various degradation information in the image content. The existence of these degradation information makes it necessary to first restore the degraded source image data before performing medical image fusion, and then send it into the fusion network for feature fusion. However, this two-stage processing mode has many limitations in clinical applications, especially in dealing with degradation. A single restoration network is often designed and trained only for a specific degradation. This results in a single network that cannot well restore and process multiple degradations, or even unknown degradations, when faced with diversified actual data input. In addition, the two-stage feature processing mode also makes the network structure redundant and complex, making it difficult to widely deploy in application scenarios. In view of this, the present application proposes a multi-degradation medical image unified fusion method based on degradation prototype learning. SUMMARY

[0003] In order to solve the shortcomings of the existing method, the present application provides a multi-degradation medical image unified fusion method based on degradation prototype learning, which can effectively fuse medical images containing degradation and improve the robustness and practicality in reality, in view of the problem that the quality of the fused image is poor due to the multi-degradation and unknown degradation of the source image in the existing multi-modal medical image fusion.

[0004] The technical scheme of the present application is: a multi-degradation medical image unified fusion method based on degradation prototype learning, the method comprising:

[0005] Step 1, obtaining an aligned multi-modal medical image dataset: obtaining an aligned multi-modal medical image dataset from a public dataset, for generating image degradation to provide training samples;

[0006] Step 2, constructing degradation data according to the real imaging principle: generating low-dose PET data, CT metal artifact data, and MRI data with motion artifacts through the imaging principle;

[0007] Step 3, learning degradation prototypes using a feature selection mechanism;

[0008] Step4、Utilize LoRa branch to process multiple degradation: through low-rank decomposition strategy, decompose the basic fusion model into multiple branches, and when fusing different degraded data, different branches can be used for processing;

[0009] Step5、Through the degradation prototype prompt to promote degradation elimination and feature fusion: design a prompt module based on learnable feature prototypes, inject degradation-related invariant features in different LoRa branches to help LoRa branches eliminate degradation and promote fusion;

[0010] Step6、Integrate branch features to construct a fused image: by integrating the degradation-eliminated and fused features in different scales, a fused image is constructed through the output layer.

[0011] Further, the Step1 includes:

[0012] Each pair of aligned multi-modal medical image data in the multi-modal medical image data includes a set of complementary medical images, including: MRI-T1, MRI-T2; MRI, CT; PET, CT;

[0013] First, the multi-modal medical image dataset is preprocessed. The specific method of preprocessing is: the resolution of the three-dimensional image is uniformly resampled to 256x256x256; data augmentation is performed by random inversion, and the pixel value of the image is normalized to [0,1].

[0014] Further, the Step2 includes:

[0015] (1) For the MRI-T1, MRI-T2 fusion scenario, the imaging characteristics of patient movement in K-space during T2 imaging are simulated to construct MRI data with motion artifacts;

[0016] (2) For the MRI, CT fusion scenario, metal artifact masks from clinical acquisition are collected, and noise and beam hardening are added to simulate metal artifact imaging to construct CT metal artifact data;

[0017] (3) For the PET, CT fusion scenario, standard dose PET data is downsampled and Poisson noise is added to simulate low-dose PET imaging, and low-dose PET data is constructed.

[0018] Further, the Step3 includes:

[0019] Step3.1, input the degraded image and the reference image to the convolution-based feature encoder to extract shallow features , and deep features , ;

[0020] Step 3.2, through deep features , The image is divided into blocks and encoded to obtain sequence information, which is then input into the Transformer for further feature extraction. The extracted features are recorded as , ; In order to aggregate information related to image degradation categories, a learnable degradation category token is also spliced , ;

[0021] Step 3.3, degenerate category tokens containing global information , As the degenerate representation of each set of features to be fused:

[0022] , Classification prediction results obtained through the fully connected layer Degenerate category label with the current feature consistent, making , Guide the learning of degenerate prototypes;

[0023] Step 3.5: Introducing a degenerate prototype consisting of learnable parameters ; When the network performs feature fusion, the prototype is degraded Through the feature selection mechanism, the prompt information related to the current degradation is screened out from the degradation prototype; among them, the information screening depends on the degradation-related feature degradation category token 、 The degradation information contained in

[0024] Step 3.6, degenerate the category tokens by degenerating the relevant features 、 Send it into the linear layer and Softmax function to get the same as the degraded prototype Information filtering heads of the same shape ; At this time, the information filtering header Degenerate prototype Dot product, from the degenerate prototype Select the degradation-related feature representation, that is, the prompt information , This will serve as a hint to help the subsequent network eliminate degradation.

[0025] Furthermore, the Step 4 includes:

[0026] Step 4.1. Using low-rank decomposition technology, replace the multi-channel convolution in the basic fusion model with low-rank convolution to construct three different LoRa branches;

[0027] The shallow features obtained in Step 4.2 and Step 3 are cascaded and used as features to be fused. The degraded fused features are sent to the basic fusion model and the constructed LoRa branch for de-degradation and fusion respectively. The respective results are integrated and sent to the output layer to obtain the final fusion result.

[0028] At the beginning of training, the parameters of the basic fusion model and the LoRa branch are updated together; in the second half of training, the parameters of the basic fusion model will be frozen, and the entire network framework will be fine-tuned by updating only the LoRa branch.

[0029] Furthermore, the Step 5 includes:

[0030] In the basic fusion model And the LoRa branch has added a prompt module that can learn feature prototypes;

[0031] Shallow features , After cascading, it is encoded into multi-scale features , (i=1, 2, 3, 4), are input into the basic network of corresponding scales and LoRa network 、 、 The features are restored and integrated in , where i represents the scale of the network;

[0032] Basic Network and LoRA network 、 、 In processing features When you need the prompt information you got earlier Assistance is required to obtain fusion features ; Fusion features The calculation formula is:

[0033] ;

[0034] in 、 、 The classification score representing the degradation type of the source image obtained by the classifier.

[0035] Furthermore, the Step 6 includes:

[0036] Step6.1、In the multi-scale feature integration architecture, a fusion branch of 4 scales is designed; shallow features , are cascaded and respectively obtained through convolution down-sampling operation to obtain multi-scale features .

[0037] Step6.2、The multi-scale features are respectively sent into a jump link embedding module to obtain the fused features after eliminating degradation .

[0038] Step6.3、The fused features are integrated through up-sampling operation and finally sent into an output layer output() composed of convolution and ReLU activation function to obtain the fused image after eliminating degradation .

[0039] Further, the Step6 further includes:

[0040] pixel intensity loss is introduced to highlight the high-contrast information in the source image in visual effect; the pixel intensity loss is expressed as:

[0041]

[0042] gradient loss is introduced to retain the edge contour and detail texture in the source image:

[0043]

[0044] wherein, denotes the to-be-fused image without degradation, denotes the clear image label corresponding to the degraded image, denotes a maximum value operation, denotes a gradient operator.

[0045] The application further provides a multi-degradation medical image unified fusion system based on degradation prototype learning, which comprises a module for executing the multi-degradation medical image unified fusion method based on degradation prototype learning.

[0046] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to realize the multi-degradation medical image unified fusion method based on degradation prototype learning.

[0047] The application has the following beneficial effects:

[0048] 1. The application meets the dual needs of degradation elimination and feature fusion by using a single image feature encoding module; in the fusion process, the complementary information between modalities is fully exploited, the feature fusion is carried out while the degradation information is eliminated, and the problem of degradation information of the source image leading to degradation of the fusion result is solved in the same framework;

[0049] 2. The application proposes a degradation prototype learning mechanism, which can flexibly model and represent the degradation information when facing various degradations or unknown degradations; through the learnable degradation prototype prompting module, the degradation information is input into the fusion network as a prompt, so that the fusion network can robustly eliminate the degradation information and complete feature fusion;

[0050] 3. The application introduces multiple LoRa branches on the basis of the basic fusion model, each branch network has a greatly reduced parameter amount, and the training cost is smaller and easier to deploy; through the LoRa branch, the features containing different degradations are processed by the corresponding branch network, and through feature integration, more robust degradation elimination fusion features are obtained; in the latter half of the training, by fixing the network parameters of the basic model, the training is further accelerated, so that the LoRa branch can play a more full role while saving the training cost;

[0051] 4. The application constructs a unified fusion framework capable of processing various degraded data for the problem of multi-modal medical image fusion, and flexibly processes various known data and even unknown data through degradation prototype learning and branch architecture; the drawbacks of the existing fusion methods in processing medical images with degradation, that is, the need to call multiple degradation elimination networks, are solved; the architecture of the method is simple and easy to deploy, so that the application has a significant advantage in practicality compared with the existing methods;

[0052] 5. Through the test on the public data set and the collected sample index, the method proposed by the application can effectively fuse medical images containing degradation, and the effect is better than that of the existing methods. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 It is a flow structure schematic diagram of the application;

[0054] Figure 2 It is a degradation medical image construction schematic diagram of the application;

[0055] Figure 3 It is a degradation prototype learning module schematic diagram of the application;

[0056] Figure 4 It is a basic fusion architecture LoRa branch architecture schematic diagram of the application;

[0057] Figure 5A schematic diagram of the multi-scale feature integration architecture of the present application;

[0058] Figure 6 A comparison chart of the test results of the method of the present application and the existing method. DETAILED DESCRIPTION

[0059] Example 1: As shown, the multi-degradation medical image unified fusion method based on degenerative prototype learning includes: Figures 1-6

[0060] Step 1, obtaining an aligned multi-modal medical image dataset: obtaining an aligned multi-modal medical image dataset from a public dataset for generating image degradation to provide training samples; each pair of multi-modal medical images in the aligned multi-modal medical image dataset has a set of complementary medical images, such as MRI-T1, MRI-T2; MRI, CT; PET, CT;

[0061] First, the dataset of multi-modal medical images is preprocessed, and the specific method of preprocessing is: the resolution of three-dimensional images is uniformly resampled to 256x256x256; the data is randomly reversed for data augmentation, and the pixel values of the images are normalized to [0,1].

[0062] Step 2, constructing degradation data according to the real imaging principle: generating low-dose PET data, CT metal artifact data, and MRI data with motion artifacts through imaging principles;

[0063] Further, the Step 2 includes:

[0064] (1) For the MRI-T1, MRI-T2 fusion scene, since the T2 modality has a longer imaging time during imaging, the movement of the patient during imaging can cause the appearance of motion artifacts. According to this feature, the imaging characteristics of the patient movement in the K space during the T2 imaging process are simulated, as shown in Figure 2 to construct MRI data with motion artifacts; specifically represented as:

[0065]

[0066] wherein represents a random affine transformation operation, represents a Fourier transform operation, represents an inverse Fourier transform operation, represents the generated degraded image, represents the original T2 modality image; this operation randomly deflects the original image into three angles, then combines the deflected images in the k space, and finally obtains the degraded image through inverse Fourier transform; ​

[0067] (2) For the MRI-CT fusion scenario, due to the imaging characteristics of CT images, when the patient has metal instruments in the body, it will cause reflection to the imaging rays of CT. Reflected in the imaging results, the metal instruments in the patient's body will cause radial metal artifacts. Based on this feature, as shown in Figure 2 , the metal artifact masks from clinical acquisition are collected, and noise and beam hardening are added to simulate metal artifact imaging to construct CT metal artifact data; specifically represented as:

[0068]

[0069] wherein represents the beam hardening function, represents the generated CT degradation data, represents the original CT data, represents the clinically acquired metal instrument mask, represents the added Poisson noise;

[0070] (3) For the PET-CT fusion scenario, PET image imaging often has noise when considering the effect of radioactive contrast agents on the patient's body at low dose radio imaging. By downsampling standard dose PET data and adding Poisson noise, the imaging effect of low dose PET is simulated, and low dose PET data is constructed. Specifically represented as:

[0071]

[0072] wherein represents the downsampling operation, represents the upsampling operation, represents the generated degradation data, represents the original PET data, represents the added Poisson noise. By downsampling and adding noise data, the imaging effect of PET under low dose contrast agent is simulated;

[0073] Step3, learn the degradation prototype using a feature selection mechanism;

[0074] Further, the Step3 includes:

[0075] Step3.1, as shown in Figure 3 , the degradation image and the reference image are input into the convolution-based feature encoder to extract shallow features , and deep features , ; where the shallow features contain rich image detail information, which will be sent to the subsequent network as detail information for degradation removal and feature fusion; the deep features are further obtained from the shallow features through several convolutional encoding layers, which contain higher-dimensional image information, and we will use the high-dimensional information of the deep features to promote the learning of the degradation prototype;

[0076] Step3.2, the image block encoding is performed on the deep features , to obtain sequence information, which is input into the Transformer for further feature extraction, and the extracted features are denoted as , ; in order to aggregate the image degradation category related information, a learnable degradation category token , is also spliced;

[0077] Step3.3, the degradation category token , containing global information is taken as the degradation representation carried by each group of to-be-fused features, in order to achieve this point:

[0078] , the classification prediction result obtained through the fully connected layer should be consistent with the degradation category label of the current feature, that is, the prediction result of the classification should minimize the following loss:

[0079] ;

[0080] where CE represents the cross-entropy loss. Thus, , can guide the learning of the degradation prototype;

[0081] Step3.5, as shown in Figure 3 , the degradation prototype composed of learnable parameters is introduced; when the network performs feature fusion, the degradation prototype selects the degradation-related prompt information from the degradation prototype through a feature selection mechanism; among them, the information screening is performed depending on the degradation information contained in the degradation-related feature degradation category token , ;

[0082] Step3.6, by sending the degradation-related feature degradation category token , into the linear layer and the Softmax function, the degradation prototype Information filtering head of same shape ; at this time, the information filtering head Will be degraded from the prototype Dot product, from the degraded prototype Select the degraded related feature representation, that is, the prompt information , Will help the subsequent network eliminate degradation as prompt information; prompt feature selection representation is:

[0083] ,

[0084] ;

[0085] Through the feature selection mechanism, the corresponding prompt information can be dynamically extracted from the degraded prototype according to the degradation existing in the current feature. At the same time, the degraded prototype will also complete gradient transmission through the extracted prompt feature and carry out differentiated update. When encountering unknown degradation, the information filtering head Can extract information from known degradation prompts in combination with input data for use in characterizing unknown degradation, thereby increasing the robustness of the network when processing unknown data.

[0086] Step4, use LoRa branch to process multiple degradation: through the low-rank decomposition strategy, the basic fusion model Is decomposed into multiple branches, which can be processed through different branches when fusing different degraded data;

[0087] The basic fusion model In the application adopts a multi-scale convolution architecture in the form of U-net, and the skip link part of the architecture is designed as a Transformer-like module based on convolution as shown in Figure 3 ; it is composed of a multi-head attention module and a feedforward module; the multi-head attention network expands the feature channel by multiple times, and each group of channel features calculates self-attention, thereby flexibly processing and strengthening the features; the feedforward module sends the features into a convolution with a kernel size of 1*1*1, and then obtains the processed features through an activation function; it can effectively integrate the output results of the multi-head attention, and help gradient transmission;

[0088] Further, the Step4 includes:

[0089] Step4.1, use low-rank decomposition technology, such as Figure 3 , replace the multi-channel convolution In the basic fusion model With low-rank convolution , wherein, Indicates the number of input channels, Indicates the number of output channels, denotes the rank of low-rank convolution, thereby constructing 3 groups of different LoRa branches 、 、 ;

[0090] Step4.2, shallow features obtained in Step3 are concatenated as features to be fused, and the degenerated fused features are sent to the base fusion model , and the constructed LoRa branches 、 、 、 for de-degeneration and fusion, respectively, and the results are integrated into the output layer to obtain the final fusion result;

[0091] In the LoRa branch, we set r to 4, which will greatly reduce the number of LoRa parameters, making it easier to train than the base network;

[0092] In the early stage of training, the base fusion model and the LoRa branch are updated together; in the later stage of training, the parameters of the base fusion model will be frozen, and the entire network framework will be fine-tuned by updating only the LoRa branch, further reducing the training overhead and improving the practicality of the network.

[0093] Step5, promote degeneration elimination and feature fusion through degeneration prototype prompts: design a prompt module based on learnable feature prototypes, inject invariant features related to degeneration in different LoRa branches to help LoRa branches eliminate degeneration and promote fusion;

[0094] Further, the Step5 includes:

[0095] A prompt module based on learnable feature prototypes is added to the base fusion model and the LoRa branch;

[0096] After concatenation, the shallow features , are encoded into multi-scale features , (i=1, 2, 3, 4), are respectively input into the base network and the LoRa network 、 、 for feature restoration and fusion, where i represents the scale of the network;

[0097] The base network and the LoRA network 、 、 In processing features , the previously obtained hint information is needed to assist to obtain fused features , as shown in Figure 4 ; the calculation formula of the fused features is:

[0098] ;

[0099] Among them 、 、 represent the classification score of the source image degradation type, obtained by the classifier.

[0100] The hint information is mapped to the spatial dimension of the feature inside the network, and the information is injected into the fused feature in the form of spatial attention, to help the network better eliminate degradation and perform feature fusion.

[0101] Among them corresponds to the specific category probability of the degradation contained in the current feature, and when it is multiplied to the output feature of each network branch in the form of weight, the network output feature will be represented as a linear combination of each network branch. This will enable the network to call the corresponding branch to process the feature when dealing with known degradation, and when encountering unknown degradation, it can regard the unknown degradation information as a linear combination of known degradation, and complete the repair of the feature containing unknown degradation information by modulating the feature output of each network branch.

[0102] Step6、Integrate each branch feature to construct a fused image: by integrating the degradation elimination fused features in different scales, a fused image is constructed through the output layer.

[0103] Further, the Step6 includes:

[0104] Step6.1、Design 4-scale fusion branches in a multi-scale feature integration architecture; cascade the shallow features , and respectively obtain multi-scale features through convolution downsampling operation; as shown in Figure 4 ;

[0105] Step6.2、Multi-scale features are respectively sent into the skip link embedding module to obtain the fused features eliminating degradation;

[0106] Step6.3、Fused features The images integrated together via the up-sampling operation are finally sent to the output layer output() composed of convolution and ReLU activation function to obtain the fusion image eliminating degradation: .

[0107] Further, the Step 6 further comprises:

[0108] The pixel intensity loss is introduced For ensuring that the fusion result highlights the high-contrast information in the source images in visual effect; the pixel intensity loss is expressed as:

[0109]

[0110] The gradient loss is introduced For making the edge contour and detail texture in the source images be preserved:

[0111]

[0112] wherein, represents the to-be-fused image without degradation, represents the clear image label corresponding to the degraded image, represents the maximum value operation, represents the gradient operator.

[0113] The application further provides a multi-degradation medical image unified fusion system based on degradation prototype learning, which comprises a module for executing the multi-degradation medical image unified fusion method based on degradation prototype learning.

[0114] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-degradation medical image unified fusion method based on degradation prototype learning.

[0115] In order to verify the effectiveness of the method of the application, the application performs training and testing on the BraTs2020, TCIA_FDG-PET-CT-Lesions, and SynthRAD2023 public data sets. The three data sets respectively contain 2500 sets of strictly aligned image data. The application selects 100 samples for testing from each of the three data sets, and the rest are used for model training. The model proposed by the application is trained on a server platform equipped with an RTX 4090 and a Pytorch environment. In the training process, the application sets the number of epochs to 200, the learning rate to 2e-5, the Batch size to 3, and uses the AdamW optimizer to update the network parameters.

[0116] Further, the present application arranges and displays the fusion effect diagram after testing. The fusion result of the method of the present application is shown in Figure 6 It can be seen that the method of the present application can well solve the negative influence of the degradation information contained in the source image on the final fusion result when medical image fusion is performed, and obtain a high-fidelity fusion image without degradation noise.

[0117] The method of the present application represents various degradation information through learning of the degradation prototype, and can even represent unknown degradation. Through the prompt learning mode, the degradation information is injected into the LoRa branch of the fusion network, so that the network can mine the complementary information between modalities according to the actual situation, eliminate the degradation, and perform feature fusion at the same time, so as to obtain a high-quality fusion result not affected by the degradation. The robustness and practicability of the fusion network in real application are improved.

[0118] The specific embodiments of the present application are described in detail above in combination with the drawings, but the present application is not limited to the above-mentioned embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the present application.

Claims

1. A unified fusion method for multiple degraded medical images based on degraded prototype learning, characterized by: The method comprises: Step 1. Obtain aligned multimodal medical image datasets: Obtain aligned multimodal medical image data from public datasets to generate image degradation to provide training samples; Step 2: Construct degraded data based on real imaging principles: Generate low-dose PET data, CT metal artifact data, and MRI data with motion artifacts through imaging principles; Step 3: Use feature selection mechanism to learn degenerate prototypes; Step 4. Use LoRa branches to handle multiple degradations: Through the low-rank decomposition strategy, the basic fusion model is decomposed into multiple branches. When fusing different degradation data, different branches can be used for processing. Step 5. Promote degradation elimination and feature fusion through degradation prototype prompts: Design a prompt module based on learnable feature prototypes to help LoRa branches eliminate degradation and promote fusion by injecting degradation-related invariant features into different LoRa branches; Step 6: Integrate the features of each branch and construct a fused image: By integrating the degraded fusion features at different scales, a fused image is constructed through the output layer; Step 3 includes: Step 3.1, degrade the image and reference images Input to the convolution-based feature encoder to extract shallow features 、 and deep features 、 ; Step 3.2, through deep features 、 The image is divided into blocks and encoded to obtain sequence information, which is then input into the Transformer for feature extraction. The extracted features are recorded as 、 ; In order to aggregate information related to image degradation categories, a learnable degradation category token is also spliced 、 ; Step 3.3, degenerate category tokens containing global information 、 As the degenerate representation of each set of features to be fused: 、 Classification prediction results obtained through the fully connected layer Degenerate category label with the current feature consistent, making 、 Guide the learning of degenerate prototypes; Step 3.5: Introducing a degenerate prototype consisting of learnable parameters ; When the network performs feature fusion, the prototype is degraded Through the feature selection mechanism, the prompt information related to the current degradation is screened out from the degradation prototype; among them, the information screening depends on the degradation-related feature degradation category token 、 The degradation information contained in Step 3.6, degenerate the category tokens by degenerating the relevant features 、 Send it into the linear layer and Softmax function to get the same as the degraded prototype Information filtering heads of the same shape ; At this time, the information filtering header Degenerate prototype Dot product, from the degenerate prototype Select the degradation-related feature representation, that is, the prompt information , This will serve as a hint to help the subsequent network eliminate degradation.

2. The unified fusion method for multiple degraded medical images based on degraded prototype learning according to claim 1 is characterized by: Step 1 includes: Each multimodal medical image pair in the aligned multimodal medical image data is composed of a group of medical images with complementary features, including: MRI-T1, MRI-T2; MRI, CT; PET, CT; First, the multimodal medical image dataset is preprocessed. The specific preprocessing method is as follows: the resolution of the three-dimensional image is uniformly resampled to 256×256×256; the data is randomly inverted for data augmentation, and the pixel values ​​of the image are normalized to between [0,1].

3. The unified fusion method of multiple degraded medical images based on degradation prototype learning according to claim 1 is characterized by: Step 2 includes: (1) For MRI-T1 and MRI-T2 fusion scenarios, MRI data with motion artifacts were constructed by simulating the imaging characteristics of the patient's movement in K space during T2 impression imaging; (2) For the MRI and CT fusion scenario, metal artifact masks from clinical acquisition were collected, and metal artifact imaging was simulated by adding noise and beam hardening to construct CT metal artifact data; (3) For the PET and CT fusion scenario, the imaging effect of low-dose PET is simulated by downsampling the standard-dose PET data and adding Poisson noise, thus constructing low-dose PET data.

4. The unified fusion method for multiple degraded medical images based on degradation prototype learning according to claim 1 is characterized by: Step 4 includes: Step 4.

1. Using low-rank decomposition technology, replace the multi-channel convolution in the basic fusion model with low-rank convolution to construct three different LoRa branches; The shallow features obtained in Step 4.2 and Step 3 are cascaded and used as features to be fused. The degraded fused features are sent to the basic fusion model and the constructed LoRa branch for de-degradation and fusion respectively. The respective results are integrated and sent to the output layer to obtain the final fusion result. At the beginning of training, the parameters of the basic fusion model and the LoRa branch are updated together; in the second half of training, the parameters of the basic fusion model will be frozen, and the entire network framework will be fine-tuned by updating only the LoRa branch.

5. The unified fusion method of multiple degraded medical images based on degradation prototype learning according to claim 1 is characterized by: Step 5 includes: In the basic fusion model And the LoRa branch has added a prompt module that can learn feature prototypes; Shallow features 、 After cascading, it is encoded into multi-scale features , i=1, 2, 3, 4, are input into the basic network of corresponding scales and LoRa network 、 、 The features are restored and integrated in , where i represents the scale of the network; Basic Network and LoRA network 、 、 In processing features When you need the prompt information you got earlier Assistance is required to obtain fusion features ; Fusion features The calculation formula is: ; in 、 、 is the classification prediction result obtained through the fully connected layer, which represents the classification score of the source image degradation type.

6. The unified fusion method of multiple degraded medical images based on degraded prototype learning according to claim 1, characterized in that: Step 6 includes: Step 6.1, design 4-scale fusion branches in the multi-scale feature integration architecture; 、 Cascade and convolution downsampling operations to obtain multi-scale features ; Step 6.2: Multi-scale features are fed into the skip link embedding module to obtain the fused features after eliminating degradation. ; Step 6.3: Fusion Features The upsampling operation is integrated together and finally sent to the output layer output() composed of convolution and ReLU activation function to obtain the fused image with degradation eliminated: .

7. The unified fusion method of multiple degraded medical images based on degraded prototype learning according to claim 1, characterized in that: The Step 6 also includes: Introducing pixel intensity loss Used to ensure that the fusion result highlights the high contrast information in the source image in terms of visual effect; pixel intensity loss Expressed as: ； Introducing gradient loss Used to preserve edge contours and detail textures in the source image: ； in, Indicates that there is no degradation in the image to be fused. represents the clear image label corresponding to the degraded image, Indicates the maximum value operation. represents the gradient operator.

8. A unified fusion system for multiple degraded medical images based on degraded prototype learning, characterized by: The system includes: a module for executing the unified fusion method of multiple degraded medical images based on degradation prototype learning as described in any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the program, the unified fusion method of multiple degraded medical images based on degraded prototype learning according to any one of claims 1 to 7 is implemented.

Images