Multi-granularity prompt interaction integrated medical image restoration method and system
Through the optimization of multi-grained prompt interaction network and loss function, the problems of high complexity and insufficient prompt interaction in the prior art are solved, and efficient integrated medical image recovery is achieved, which can handle image recovery of multiple degradation types and levels.
Patent Information
- Application Number
- CN202510441343.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-08-05
AI Technical Summary
The existing integrated medical image recovery method based on prompt learning is complex, the recovery effect and performance are difficult to balance, and the interaction between multi-grained prompts is lacking, and multiple prompt information cannot be fully utilized.
A multi-grained cues interactive network is adopted. By dividing the input images into fine-grained, medium-grained and coarse-grained sub-blocks, they enter the corresponding network for feature extraction and fusion. The multi-grained cues are fused between multiple particle sizes, and image recovery is achieved through the Charbonnier loss and edge loss optimization model.
It reduces the complexity of the model, improves the balance between recovery effect and performance, and can better restore medical images of multiple degradation types and levels, providing an integrated medical image recovery system.
Smart Images
Figure CN120430969A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image restoration, and combines a prompt learning method with multi-granularity prompt interaction to realize an integrated medical image restoration method and system. Background Art
[0002] During medical imaging, images are susceptible to noise, blur, and motion artifacts due to interference from the acquisition equipment or the patient's breathing and movement, which can affect patient treatment. Medical image restoration not only improves image quality and diagnostic accuracy, but also helps reduce radiation risks during imaging, enhances patient cooperation, and promotes the development of personalized and precision medicine. In the future, as medical image restoration technology continues to advance, it will continue to provide more reliable support for the field of medical image analysis, helping doctors and patients jointly address increasingly complex disease challenges.
[0003] Traditional image restoration methods solve this problem through mathematical and probabilistic models, and adopt different restoration methods according to the size of the degraded area and the type of degradation. Deep learning methods provide higher restoration accuracy and stronger adaptability. Appropriate models can be trained for different degradation types. In order to further improve the restoration efficiency, integrated methods have emerged, which can handle multiple degradations through a single model without the need to train each degradation separately. Current medical image restoration methods, especially integrated methods based on prompt learning: 1) The integrated medical image restoration model is highly complex, and it is difficult to balance the restoration effect and performance; 2) There is a lack of interaction between prompts, and multiple prompt information cannot be fully utilized; 3) There is no integrated medical image restoration method based on prompt learning. In order to improve the efficiency of medical diagnosis, reduce the cost of primary healthcare and make full use of available medical images, integrated medical image restoration is very important. Summary of the Invention
[0004] To address these challenges, this paper proposes a multi-granularity prompt interaction network for medical image restoration. The proposed multi-granularity prompt interaction module integrates prompts across multiple granularities, ensuring their full utilization. This network significantly reduces the number of parameters by leveraging multiple granularity prompts, achieving a balance between model performance and effectiveness. This provides guidance for integrated medical image restoration models and leads to the design of an integrated medical image restoration system based on prompt learning.
[0005] In view of this, the technical solution adopted by the present invention is: an integrated medical image restoration method with multi-granularity prompt interaction, comprising the following steps:
[0006] Through multi-granularity scanning, the input image is divided into fine-grained sub-blocks, medium-grained sub-blocks and coarse-grained sub-blocks, which enter the fine-grained, medium-grained and coarse-grained networks respectively;
[0007] The fine-grained sub-block and the medium-grained sub-block respectively calculate shallow features through the multi-granularity general module to obtain fine-grained preliminary shallow features and medium-grained preliminary shallow features, and then enter the fine-grained Prompt network and the medium-grained Prompt network respectively. The coarse-grained sub-block performs shallow feature extraction to obtain coarse-grained preliminary shallow features and medium-grained output features, and then enters the coarse-grained Prompt network after splicing;
[0008] After being processed by the fine-grained Prompt network and the medium-grained Prompt network, the fused features are obtained through CSFF respectively, and the fine-grained and medium-grained restored images and output features are obtained through SAM; the output features need to be spliced with the preliminary shallow features of the next granularity to form new shallow features, that is, the fine-grained output features and the medium-grained output features are respectively combined with the medium-grained preliminary shallow features and the coarse-grained preliminary shallow features to obtain the medium-grained shallow features and the coarse-grained shallow features; the fused features need to be combined with the shallow features of the next granularity, that is, the fine-grained fused features and the medium-grained fused features are respectively combined with the medium-grained shallow features and the coarse-grained shallow features to input the corresponding Prompt network;
[0009] The multi-granularity prompt interaction module (FMDTA+GDFN) is used to fuse multi-granularity prompts, that is, the previous granularity prompt needs to interact with the next granularity prompt;
[0010] The final output image of the coarse-grained network is used as the restored image.
[0011] Furthermore, the multi-granularity scanning formula Represented as follows, where M is the M-dimensional input feature sample, K is the sampling window of length K, S is the span of the sampling window, and n is the number of K-dimensional feature subsamples. For multi-granularity samples with an input image size of [H, W], the sampling window size is [K W ,K H ], the corresponding sampling step size is S W and S H , the formula for calculating the subsample is where n ln is the number of multi-granularity subsamples. H, W are the width and height of the input image, K W ,K H are the width and height of the sampling window.
[0012] Furthermore, the fine-grained sub-block and the medium-grained sub-block respectively calculate shallow features through the multi-grained general module, which are respectively expressed as:
[0013] Each fine-grained sub-block I ij , i = 1, 2, j = 1, 2, respectively, through a 3 × 3 convolution and a channel attention block, to obtain the fine-grained shallow feature Fij :
[0014] F ij =CAB(C 3×3 (I ij ))
[0015] The medium-grained sub-blocks I1 and I2 are respectively subjected to a 3×3 convolution and a channel attention block to extract the medium-grained shallow features of each sub-block.
[0016] The coarse-grained input I is processed to extract the coarse-grained shallow features F CCF :
[0017] F CCF =CC(CDC(CC(CC(CDC(CC(CC(CC(CC(CDC(CC(CC(CDC(I))))))))))))
[0018] F=Concat(F CCF ,F in )
[0019] CC represents the combination of complex convolution (ComplexConv), complex batch normalization (ComplexBN), and complex activation function (ComplexRelu). CDC represents the combination of dilated complex convolution (Dilated ComplexConv), complex batch normalization (ComplexBN), and complex activation function (ComplexRelu). F represents coarse-grained shallow features. is the fine-grained output feature. in is the medium-grained output feature.
[0020] Furthermore, the fusion features and output features in the fine-grained Prompt network can be expressed as:
[0021] Prompt network codec structure to get codec output:
[0022]
[0023] The outputs of the encoder and decoder of the four sub-blocks are respectively fine-grained, where enc m represents the i-th layer code, dec n represents the j-th layer decoding, They are the output concatenation features of the fine-grained codec, and Concat represents concatenation;
[0024] The fusion features are obtained by CSFF:
[0025]
[0026] The output features are obtained by the supervised attention mechanism and transferred to the medium granularity;
[0027]
[0028] represents fine-grained fusion features, represents the fine-grained recovery feature, I i is a medium-granularity input, It is a fine-grained output feature, sigmoid activation function.
[0029] Furthermore, the multi-granularity prompt interaction module fuses the multi-granularity prompts including fine-granularity and medium-granularity fusion, and medium-granularity and coarse-granularity fusion;
[0030] The fusion of fine-grained and medium-grained granularity specifically includes:
[0031] For fine-grained hints P fine Since the fine-grained input is sampled, the prompt after fine-grained learning is first Splice to make it conform to the medium-grained tip P medi Scale;
[0032]
[0033] The four fine-grained sub-blocks, the upper two of which correspond to the learned prompts, are concatenated to become the same size as the medium-grained prompts.
[0034] Operate on fine-grained spliced cues to obtain fused fine-grained cues
[0035]
[0036] DC 3×3 represents a 3×3 depthwise separable convolution, C 1×1 represents 1×1 convolution, LN represents layer normalization, and Re represents Reshape.
[0037] Centering particle size prompt P medi The operation was performed to obtain the separation medium particle size prompt F1P medi , and F2P medi ;
[0038] F1P medi =Re(DC 3×3 (C 1×1 (LN(P medi ))))
[0039] F2P medi =Re(DC3×3 (C 1×1 (LN(P medi ))))
[0040] Will fuse fine-grained hints F1P with isolated medium particle size tips medi The operation obtains the fine-grained and medium-grained attention distribution F fm ;
[0041]
[0042] Fine-grained and medium-grained attention distribution F fm F2P with isolated medium particle size tips medi After fusion selection
[0043]
[0044] GDFN stands for Gated Depthwise Separable Convolutional Feedforward Network.
[0045] The fusion of medium-sized and coarse-sized particles can be obtained in the same way.
[0046] Furthermore, the image finally output by the coarse-grained network is used as the restored image, which specifically includes the coarse-grained decoder output feature Q dec After convolution with a convolution kernel of 3×3, it is added to the coarse-grained feature sub-block I to obtain the coarse-grained restored image I coarse :I coarse =C 3×3 (Q dec )+I.
[0047] Furthermore, the method designed in the present invention also includes calculating multi-granularity loss between the restored image and the target image to ensure the restoration effect; mainly adopting Charbonnier loss and edge loss to optimize, a variant of Charbonnier loss L1 loss. For fine-grained output I fine Brightness preservation is achieved by performing pixel-level reconstruction error with the real image I;
[0048]
[0049] where ε is set to 10 -3 , It is a fine-grained Charbonnier loss. In addition, the edge loss is used to constrain the high-frequency components between the real image and the restored image;
[0050]
[0051] Where Δ is the Laplace operator, It is a fine-grained edge loss that combines Charbonnier and edge loss Can Get the total loss of the fine-grained stage;
[0052]
[0053] The weight coefficient λ is set to 0.05 to balance the above two losses, L fine is the total loss at fine-grained level.
[0054] Similar to the above fine-grained loss, the final loss function of the network is:
[0055] L=L fine +L medi +L coarse
[0056] L medi represents the total loss of medium particle size, L coarse represents the total loss of coarse grain.
[0057] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned integrated medical image restoration method with multi-granularity prompt interaction are implemented.
[0058] Finally, the present invention provides an integrated medical image restoration system with multi-granularity prompt interaction, including
[0059] The multi-granularity scanning module is used to divide the input image into fine-grained sub-blocks, medium-grained sub-blocks and coarse-grained sub-blocks through multi-granularity scanning, and then enter the three-part network of fine-grained, medium-grained and coarse-grained respectively;
[0060] A shallow feature calculation module is used to calculate shallow features of the fine-grained sub-block and the medium-grained sub-block through a multi-granularity general module to obtain fine-grained shallow features and medium-grained preliminary shallow features, which are then fed into the fine-grained Prompt network and the medium-grained Prompt network respectively. The coarse-grained sub-block performs shallow feature extraction to obtain coarse-grained preliminary shallow features, which are then spliced with the medium-grained output features and fed into the coarse-grained Prompt network.
[0061] The Prompt network module includes a fine-grained Prompt network, a medium-grained Prompt network, and a coarse-grained Prompt network. After being processed by the fine-grained Prompt network and the medium-grained Prompt network, fusion features are obtained through CSFF, and fine-grained and medium-grained restored images and output features are obtained through SAM. The fine-grained output features and medium-grained output features are respectively combined with the medium-grained preliminary shallow features and the coarse-grained preliminary shallow features to obtain medium-grained shallow features and coarse-grained shallow features. The fine-grained fusion features and medium-grained fusion features are respectively combined with the medium-grained shallow features and the coarse-grained shallow features to input the corresponding Prompt network.
[0062] A multi-granularity prompt interaction module is used to fuse multi-granularity prompts;
[0063] Output module, the final output image of the coarse-grained network is used as the restored image.
[0064] This system is designed to comprehensively implement multiple restoration tasks such as medical image deblurring, denoising, and motion artifact removal, providing researchers and developers with an efficient and flexible testing and research platform. The system supports both single-task and integrated restoration based on different degradation types and levels. This system is designed to address the challenges of medical image restoration, including deblurring, denoising, and motion artifact removal. This system transcends the limitations of traditional restoration tools by integrating not only standard image restoration capabilities but also degraded image generation and integrated image restoration capabilities, creating a more diverse platform for medical image restoration.
[0065] In summary, the advantages and beneficial effects of the present invention are as follows:
[0066] 1. A novel medical image restoration method based on multi-granularity interaction of prompts is proposed, which can simultaneously solve single-task restoration problems and integrated restoration problems.
[0067] 2. Introducing prompt learning, which reduces the complexity of prompt-based learning methods through multi-granularity prompts and makes the model more interpretable.
[0068] 3. To achieve interaction between multi-granularity cues and fully utilize their fusion to achieve a balance between restoration effect and performance, a multi-granularity cue interaction module is introduced to fuse learnable cues from multiple granularities, fusing medium-granularity cues with fine-granularity cues, and coarse-granularity cues with medium-granularity cues, thereby restoring a damaged image that is closer to the target image.
[0069] 4. Designed an integrated medical image restoration system based on multi-granularity prompt interaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0070] Figure 1This is the overall model diagram of the present invention;
[0071] Figure 2 This is a diagram of the multi-granularity prompt interaction module of the present invention;
[0072] Figure 3 An integrated medical image restoration system interface based on prompt learning;
[0073] Figure 4 Integrated restoration interface for an integrated medical image restoration system based on prompt learning. DETAILED DESCRIPTION
[0074] The present invention will be described in detail below with reference to the following examples, which are intended to illustrate the present invention and are not intended to limit the scope of application of the present invention. The present invention is not limited to the following embodiments or examples, and any modifications and variations that do not violate the spirit of the present invention are intended to be included within the scope of the present invention.
[0075] The present invention solves the above technical problems by inputting the degraded image into the model, dividing the input image into blocks corresponding to three granularities, inputting each block into the network corresponding to the three granularities, and then comprehensively calculating the loss and optimizing it to finally obtain the optimal result.
[0076] The following will be combined Figure 1 and Figure 2 Describe the technical solution of the present invention in detail:
[0077] A medical image restoration method based on multi-granularity prompt interaction includes the following steps:
[0078] Step 1: In the fine-grained stage, four sub-blocks (I 11 , I 12 , I 21 , I 22 ) to perform shallow feature extraction.
[0079]
[0080] I ij (i=1,2,j=1,2) represents the fine-grained sub-block. I represents the coarse-grained input image. The shallow feature F of the fine-grained sub-block ij (i=1,2j=1,2), channel attention block CAB, C 3×3 The convolution kernel size is 3×3.
[0081] Step 2: Then input the shallow features into the Prompt network.
[0082]
[0083] The outputs of the encoder and decoder of the four sub-blocks are respectively fine-grained, where enc m represents the i-th layer code, dec n represents the j-th layer decoding, They are the output concatenation features of the fine-grained codec, and Concat represents concatenation;
[0084] PromptSRNet_Encoder, PromptSRNet_Decoder_P fine AB is a fine-grained Prompt network codec.
[0085] Step 3: Calculate the fine-grained codec output splicing features, calculate the fine-grained restoration features and fine-grained output features.
[0086]
[0087] represents fine-grained fusion features, represents the fine-grained recovery feature, I i is a medium-granularity input, It is a fine-grained output feature, sigmoid activation function.
[0088] Step 4: Calculate the fine-grained restored image.
[0089] Step 5: In the medium-granularity stage, two sub-blocks (I1, I2) are scanned at multiple granularities for shallow feature extraction.
[0090]
[0091] I i (i=1,2) represents the medium granularity sub-block. Same as above, F i It is characterized by medium-grained shallow layers.
[0092] Step 6: Input shallow features into the Prompt network.
[0093]
[0094] are the output features of the medium-granularity codec. enc , F dec are the output concatenation features of the medium-granularity encoder and decoder. medi AB is the medium-grained and coarse-grained Prompt network codec.
[0095] Among them, P medi The process is as follows: For fine-grained hints P fineSince the fine-grained input is sampled, the fine-grained learning prompt is first Splice to make it conform to the medium-grained tip P medi scale.
[0096]
[0097] The four fine-grained sub-blocks, the upper two of which correspond to the learned prompts, are concatenated to become the same size as the medium-grained prompts.
[0098] The fine-grained spliced prompts are subjected to layer normalization LN, 1×1 convolution and 3×3 depth-separable convolution operations and the size is changed to obtain fused fine-grained prompts.
[0099]
[0100] DC 3×3 represents a 3×3 depthwise separable convolution, C 1×1 represents 1×1 convolution, LN represents layer normalization, and Re represents Reshape.
[0101] P medi After performing layer normalization LN, 1×1 convolution and 3×3 depth-separable convolution operations and changing the size, the separation medium granularity hint F1P is obtained medi and F2P medi .
[0102] F1P medi =Re(DC 3×3 (C 1×1 (LN(P medi ))))
[0103] F2P medi =Re(DC 3×3 (C 1×1 (LN(P medi ))))
[0104] Will fuse fine-grained hints F1P with isolated medium particle size tips medi Perform matrix multiplication and activation function softmax to obtain fine-grained and medium-grained attention distribution F fm .
[0105]
[0106] Distribute the fine-grained and medium-grained attention F fm F2P with isolated medium particle size tips mediDo matrix multiplication and then 1×1 convolution to get the fusion selected
[0107]
[0108] GDFN stands for Gated Depthwise Separable Convolutional Feedforward Network, as Figure 2 shown.
[0109] Step 7: Calculate the medium-granularity codec output splicing features, calculate the medium-granularity recovery features and the medium-granularity output features.
[0110] F csff =C 1×1 (F enc )+C 1×1 (F dec )
[0111] F in =sigmoid(C 1×1 (I out ))×C 1×1 (F dec )+F dec
[0112] sigmoid represents the sigmoid function. csff , F in , I out They are medium-granularity fusion features, medium-granularity output features and medium-granularity recovery features respectively.
[0113] Step 8: Calculate the medium-grained restored image.
[0114] Step 9: In the coarse-grained stage, a sub-block is scanned at multiple granularities for shallow feature extraction.
[0115] F CCF =CC(CDC(CC(CC(CDC(CC(CC(CC(CC(CDC(CC(CC(CDC(I))))))))))))
[0116] F=Concat(F CCF ,F in )
[0117] CC represents the combination of complex convolution (ComplexConv), complex batch normalization (ComplexBN), and complex activation function (ComplexRelu). CDC represents the combination of dilated complex convolution (Dilated ComplexConv), complex batch normalization (ComplexBN), and complex activation function (ComplexRelu).
[0118] Coarse-grained preliminary shallow features F CCF With medium-granularity output feature F in The coarse-grained shallow features F are obtained by splicing along the number of channels C.
[0119] Step 10: Calculate the output and prompt P of the coarse-grained Prompt network coarse , calculate the restored image. The Prompt network codec structure is consistent with the medium granularity. The codec output is obtained according to the Prompt network codec structure formula:
[0120]
[0121] The interaction between medium-grained and coarse-grained prompts is consistent with step six to obtain P coarse .Q enc , Q dec are the output features of the medium-granularity codec, respectively.
[0122] Step 11: Get the restored image:
[0123] I coarse =C 3×3 (Q dec )+I
[0124] An integrated medical image restoration system with multi-granularity prompt interaction: Figure 3 , Figure 4As shown, upon launching the system, the user first selects either the Degraded Image Restoration or Degraded Image Synthesis page, which leads to a specific operation process. If the user selects the Degraded Image Restoration function, the system will direct the user to the corresponding operation page. On this page, the user must sequentially configure several key settings. First, the user must select the restoration type. Two modes are available: single-task image restoration and integrated image restoration. Next, the user must specify the type of medical image data to be restored and the type and degree of degradation. This helps the system more accurately select an appropriate restoration strategy. Furthermore, the user must select a restoration model, which can be flexibly selected based on the specific situation. After completing all of the above selections, the user can select the medical image to be restored and click the "Restore" button. The system will then restore the image using the selected model. If the restored image is satisfactory, click the "Save" button to save the restored image. If the user selects the Degraded Image Synthesis function, the system will direct the user to the Degraded Image Synthesis page. On this page, users first select the type of degradation to be added to the medical image, such as blur, noise, or motion artifacts, and determine the corresponding degree of degradation to precisely control the degradation effect of the synthesized image. Subsequently, users select the medical image to be degraded and click the "Convert" button to synthesize the degraded image. The system then degrades the selected medical image according to the user-defined parameters, generating an image with specific degradation characteristics. Similarly, if the user wishes to save the synthesized degraded image, they can click the "Save" button to complete the operation. Through this operational process, users can conveniently use this prototype system to perform degradation synthesis and restoration processing on various medical images, providing support for research and practice related to medical image processing.
[0125] The conventional techniques and schemes not described in detail in the above embodiments are well known in the art and will not be described in detail here. The above embodiments and / or experimental examples describe the preferred embodiments of the present invention in detail, but the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various simple variations of the technical scheme of the present invention can be made, and these simple variations all fall within the scope of protection of the present invention.
Claims
1. An integrated medical image restoration method with multi-granularity prompt interaction, characterized in that: The following steps are involved: Through multi-granularity scanning, the input image is divided into fine-grained sub-blocks, medium-grained sub-blocks and coarse-grained sub-blocks, which enter the fine-grained, medium-grained and coarse-grained networks respectively; The fine-grained sub-block and the medium-grained sub-block respectively calculate shallow features through the multi-grained general module to obtain fine-grained shallow features and medium-grained preliminary shallow features, and then enter the fine-grained Prompt network and the medium-grained Prompt network respectively. The coarse-grained sub-block performs shallow feature extraction to obtain coarse-grained preliminary shallow features and medium-grained output features, and then enters the coarse-grained Prompt network after splicing; After being processed by the fine-grained Prompt network and the medium-grained Prompt network, fusion features are obtained through CSFF, and fine-grained and medium-grained restored images and output features are obtained through SAM. The fine-grained output features and medium-grained output features are respectively combined with the medium-grained preliminary shallow features and coarse-grained preliminary shallow features to obtain medium-grained shallow features and coarse-grained shallow features. The fine-grained fusion features and medium-grained fusion features are respectively combined with the medium-grained shallow features and coarse-grained shallow features to input the corresponding Prompt network. The multi-granularity prompt interaction module (FMDTA+GDFN) is used to fuse multi-granularity prompts, that is, the previous granularity prompt needs to interact with the next granularity prompt; The final output image of the coarse-grained network is used as the restored image.
2. The integrated medical image restoration method with multi-granularity prompt interaction according to claim 1, characterized in that: The multi-granularity scanning formula Represented as follows, where M is the M-dimensional input feature sample, K is the sampling window of length K, S is the span of the sampling window, and n is the number of K-dimensional feature subsamples. For multi-granularity samples with an input image size of [H, W], the sampling window size is [K W ,K H ], the corresponding sampling step size is S W and S H , the formula for calculating the subsample is where n ln is the number of multi-granularity subsamples, H, W are the width and height of the input image, K W ,K H are the width and height of the sampling window.
3. The integrated medical image restoration method with multi-granularity prompt interaction according to claim 1, characterized in that: The fine-grained sub-block and the medium-grained sub-block respectively calculate shallow features through the multi-granularity general module, which are respectively expressed as: Each fine-grained sub-block I ij , i = 1, 2, j = 1, 2, respectively, through a 3 × 3 convolution and a channel attention block, to obtain the fine-grained shallow feature F ij : F ij =CAB(C 3×3 (I ij )) The medium-grained sub-blocks I1 and I2 are respectively subjected to a 3×3 convolution and a channel attention block to extract the medium-grained shallow features F of each sub-block. i : The coarse-grained input I is processed to extract the coarse-grained shallow features F CCF : F CCF =CC(CDC(CC(CC(CDC(CC(CC(CC(CDC(CC(CC(CDC(I)))))))))))) F=Concat(F CCF ,F in ) Where CC represents the combination of complex convolution, complex batch normalization, and complex activation function; CDC represents the combination of void complex convolution, complex batch normalization, and complex activation function; F represents the coarse-grained shallow feature. represents the fine-grained output features, F in Represents medium-grained output features.
4. The integrated medical image restoration method with multi-granularity prompt interaction according to claim 1, characterized in that: The fusion features and output features in the fine-grained Prompt network can be expressed as: Prompt network codec structure to get codec output: They are the outputs of the encoder and decoder of the four fine-grained sub-blocks, where enc m represents the i-th layer code, dec n represents the j-th layer decoding, They are the output concatenation features of the fine-grained codec, and Concat represents concatenation; The fusion features are obtained by CSFF: The output features are obtained by the supervised attention mechanism and transferred to the medium granularity; represents fine-grained fusion features, represents the fine-grained recovery feature, I i is a medium-granularity input, It is a fine-grained output feature, sigmoid activation function.
5. The integrated medical image restoration method with multi-granularity prompt interaction according to claim 1, characterized in that: The multi-granularity prompt interaction module fuses multi-granularity prompts including fine-granularity and medium-granularity fusion, and medium-granularity and coarse-granularity fusion; The fusion of fine-grained and medium-grained granularity specifically includes: For fine-grained hints P fine Since the fine-grained input is sampled, the prompt after fine-grained learning is first Splice to make it conform to the medium-grained tip P medi Scale; The four fine-grained sub-blocks, the upper two of which correspond to the learned prompts, are concatenated to become the same size as the medium-grained prompts. Operate on fine-grained spliced cues to obtain fused fine-grained cues DC 3×3 represents a 3×3 depthwise separable convolution, C 1×1 represents 1×1 convolution, LN represents layer normalization, and Re represents Reshape; Centering particle size prompt P medi The operation was performed to obtain the separation medium particle size prompt F1P medi and F2P medi ; F1P medi =Re(DC 3×3 (C 1×1 (LN(P medi )))) F2P medi =Re(DC 3×3 (C 1×1 (LN(P medi )))) Will fuse fine-grained hints F1P with isolated medium particle size tips medi The operation obtains the fine-grained and medium-grained attention distribution F fm ; Fine-grained and medium-grained attention distribution F fm F2P with isolated medium particle size tips medi After fusion selection GDFN stands for Gated Depthwise Separable Convolutional Feedforward Network.
6. The integrated medical image restoration method with multi-granularity prompt interaction according to claim 1, characterized in that: The final output image of the coarse-grained network is used as the restored image, specifically including the coarse-grained decoder output feature Q dec After convolution with a convolution kernel of 3×3, it is added to the coarse-grained feature sub-block I to obtain the coarse-grained restored image I coarse :I coarse =C 3×3 (Q dec )+I.
7. The integrated medical image restoration method with multi-granularity prompt interaction according to claim 1, characterized in that: It also includes calculating multi-granularity losses between the restored image and the target image; it mainly uses Charbonnier loss and edge loss to optimize, for fine-grained output I fine Brightness preservation is achieved by performing pixel-level reconstruction error with the real image I; where ε is set to 10 -3 , It is a fine-grained Charbonnier loss; in addition, the edge loss is used to constrain the high-frequency components between the real image and the restored image; Where Δ is the Laplace operator, It is the fine-grained edge loss. Combining Charbonnier and edge loss can get the total loss of the fine-grained stage. L fine is the total loss of fine-grained, λ represents the weight coefficient; Similar to the above fine-grained loss, the final loss function of the network is: L=L fine +L medi +L coarse L medi represents the total loss of medium particle size, L coarse represents the total loss of coarse grain.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the integrated medical image restoration method with multi-granularity prompt interaction as described in any one of claims 1 to 7 are implemented.
9. An integrated medical image restoration system with multi-granularity prompt interaction, characterized by: include The multi-granularity scanning module is used to divide the input image into fine-grained sub-blocks, medium-grained sub-blocks and coarse-grained sub-blocks through multi-granularity scanning, and then enter the three-part network of fine-grained, medium-grained and coarse-grained respectively; A shallow feature calculation module is used to calculate shallow features of the fine-grained sub-block and the medium-grained sub-block through a multi-granularity general module to obtain fine-grained shallow features and medium-grained preliminary shallow features, which are then fed into the fine-grained Prompt network and the medium-grained Prompt network respectively. The coarse-grained sub-block performs shallow feature extraction to obtain coarse-grained preliminary shallow features, which are then spliced with the medium-grained output features and fed into the coarse-grained Prompt network. The Prompt network module includes a fine-grained Prompt network, a medium-grained Prompt network, and a coarse-grained Prompt network. After being processed by the fine-grained Prompt network and the medium-grained Prompt network, CSFF is used to obtain fusion features, and SAM is used to obtain fine-grained and medium-grained restored images and output features. The fine-grained output features and the medium-grained output features are respectively combined with the medium-grained preliminary shallow features and the coarse-grained preliminary shallow features to obtain the medium-grained shallow features and the coarse-grained shallow features; the fine-grained fusion features and the medium-grained fusion features are respectively combined with the medium-grained shallow features and the coarse-grained shallow features to input the corresponding Prompt network; A multi-granularity prompt interaction module is used to fuse multi-granularity prompts; Output module, the final output image of the coarse-grained network is used as the restored image.