Medical image restoration method and device, computer device and storage medium
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2025-06-04
- Publication Date
- 2026-07-03
Smart Images
Figure CN120707437B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for medical image restoration. Background Technology
[0002] Medical image restoration technology aims to reconstruct low-quality images into high-quality images, thereby reducing patient scan time and radiation dose, increasing the throughput efficiency of large hospital equipment, reducing radiation damage to patients, and increasing image quality.
[0003] Traditional integrated medical image restoration methods often use models with fixed parameters to address medical image restoration tasks of different modalities, but their practical application is limited in terms of generalization, efficiency, and data availability. Summary of the Invention
[0004] Therefore, it is necessary to provide a medical image recovery method, apparatus, computer equipment, and storage medium to address the aforementioned technical problems.
[0005] In a first aspect, this application provides a method for medical image restoration, the method comprising:
[0006] Acquire initial medical images;
[0007] The initial medical image is input into the image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptive module, an image transformation network, and a computation network; wherein, the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features;
[0008] High-quality medical images are generated using the image restoration model.
[0009] In one embodiment, generating high-quality medical images through the image restoration model includes:
[0010] The initial features are obtained by extracting features from the initial medical image using the initial feature extraction network.
[0011] The initial features are encoded in multiple stages by the multi-stage encoder to obtain the latent representation;
[0012] The task feature extraction network extracts gradient-decoupled copies of the latent representation to generate task-specific features; these task-specific features are used to guide the multi-stage decoder in weight allocation.
[0013] The multi-stage decoder generates task-adaptive weights based on the task-specific features, and the latent representation is decoded based on the task-adaptive weights to obtain deep features.
[0014] The image conversion network is used to convert the depth features into an image, resulting in a residual image.
[0015] The computational network performs summation operations on the initial medical image and the residual image to obtain a high-quality medical image.
[0016] In one embodiment, the multi-stage encoder includes a first encoding module, a second encoding module, and a third encoding module. The initial features are encoded sequentially through the first encoding module, the second encoding module, and the third encoding module. In each encoding, the spatial height and width are halved, and the channel dimension is doubled to obtain the potential representation.
[0017] In one embodiment, the task feature extraction network includes multiple sequentially connected convolutional blocks, capable of directly extracting features from latent features I. LF Extract task-specific features Z∈R d Its expression is:
[0018] Z = TREN(SG(I) LF ))
[0019] Where TREN(·) represents the task feature extraction network, and SG(·) represents the stopping gradient operator, which will extract the latent features I LF The extraction of features is decoupled from the extraction of task-specific features Z to avoid interference between processes with different objectives.
[0020] In one embodiment, the step of generating task-adaptive weights based on the task-specific features using the multi-stage decoder, and decoding the latent representation based on the task-adaptive weights to obtain deep features includes:
[0021] The task-specific features are subjected to a first transformation process to obtain a first transformation result;
[0022] The potential representation is subjected to a first normalization process to obtain a first intermediate parameter;
[0023] The transformation result is subjected to a first reshaping process to obtain a first separable convolution weight. The first separable convolution weight is then summed with the first shared weight of the first intermediate parameter to obtain a second intermediate parameter.
[0024] The second intermediate parameter is weighted using a channel attention mechanism to obtain the weighting result.
[0025] Based on the weight allocation result, the latent features are subjected to a second normalization process to obtain a third intermediate parameter;
[0026] The transformation result is subjected to a second reshaping process to obtain a second separable convolution weight. The second separable convolution weight is then summed with the second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter.
[0027] The fifth intermediate parameter is subjected to a second transformation process to obtain the second transformation result;
[0028] The third intermediate parameter and the second transformation result are combined to obtain the depth feature.
[0029] In one embodiment, the loss weights during the training of the image restoration model are represented as follows:
[0030]
[0031] Where L1(·) represents the L1 distance, and SG(·) represents the stopping gradient operator, used to decouple loss balance and model optimization. LQ ,I HQ ), and The three terms encode the training dynamics associated with the samples, and MLP(·) represents the transformation process.
[0032] Secondly, this application also provides a medical image restoration device, the device comprising:
[0033] The acquisition module is used to acquire initial medical images;
[0034] An input module is used to input the initial medical image into an image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptive module, an image conversion network, and a computation network; wherein, the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features;
[0035] A generation module is used to generate high-quality medical images using the image restoration model.
[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above embodiments.
[0037] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above embodiments.
[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in any of the above embodiments.
[0039] In the aforementioned medical image restoration method, a low-quality initial medical image is first acquired and input into an image restoration model to generate a high-quality medical image. The image restoration model extracts initial features from the initial medical image through an initial feature extraction network. Subsequently, a multi-stage encoder encodes the initial features into latent representations, which are then processed by a multi-stage decoder to generate deep features. A decoupled copy of the gradient is input into a task-specific feature extraction network to extract task-specific features. The decoder uses these task-specific features to generate task-adaptive weights, thereby achieving customized feature optimization during the decoding process. An image conversion network converts the deep features into a residual image, which is then element-wise added to the low-quality initial medical image to obtain the reconstructed high-quality medical image. This medical image restoration method can improve the quality of initial image models of different formats, resulting in high-quality medical images with improvements in generalization, efficiency, and data availability. Attached Figure Description
[0040] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0041] Figure 1 This is a diagram illustrating the application environment of a medical image restoration method in one embodiment;
[0042] Figure 2 This is a flowchart illustrating a medical image restoration method in one embodiment;
[0043] Figure 3 Here is a structural block diagram of an image restoration model in one embodiment;
[0044] Figure 4 This is a comparison diagram of experiments with AMIR in another embodiment;
[0045] Figure 5 This is a structural block diagram of a medical image restoration device in one embodiment;
[0046] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0048] Traditional dedicated models trained for a single task in the field of medical imaging suffer from the following problems: (1) Limited generalization. In complex multimodal image restoration tasks (such as PET / CT or PET / MRI), multiple medical image restoration tasks often need to be processed simultaneously. However, due to the inherent differences between imaging modalities and degradation types, dedicated models trained for a single task are difficult to adapt to other tasks, resulting in a significant performance drop. (2) Inefficiency and bloated models. Dedicated models trained for a single task require repetitive development work and resource investment. Each task requires an independent network architecture, training process, storage scheme, and computing resources. This discrete development significantly increases costs and makes clinical deployment more difficult. (3) Scarce data and low overall utilization. Dedicated models trained for a single task rely on narrow dedicated datasets (the data scale in the field of medical imaging is limited), and are therefore highly susceptible to insufficient data. This not only increases the risk of model overfitting, but also loses the potential for cross-task and cross-modal collaborative training due to its training method for a single task.
[0049] In summary, the limitations of dedicated models in terms of generalization, efficiency, and data availability collectively restrict their scalability and practical application value. Traditional integrated medical image restoration methods mostly use models with fixed parameters to address medical image restoration tasks of different modalities, and only input different modal information as conditional information into the model, lacking consideration of the following two aspects: (1) Task interference, where gradient update directions conflict for different tasks on the same parameter. (2) Task imbalance, where imbalanced optimization problems arise due to the inherent differences in learning difficulty among tasks.
[0050] The medical image restoration method provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0051] In one exemplary embodiment, such as Figure 2 As shown, a medical image restoration method is provided, which can be applied to... Figure 1 Taking server 102 as an example, the explanation includes the following steps 202 to 206. Wherein:
[0052] Step 202: Obtain initial medical images.
[0053] For example, the initial medical image may be a low-resolution image.
[0054] Step 204: Input the initial medical image into the image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptive module, an image transformation network, and a computation network; wherein, the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features.
[0055] For example, the image restoration model is a task-adaptive Transformer network trained on a specific task, such as... Figure 3 As shown, Figure 3 A block diagram of the image restoration model is shown.
[0056] Most traditional all-in-one models share a common limitation: they rely on a single model with fixed parameters to handle multiple tasks. This "one-size-fits-all" approach often fails due to "task interference," where gradient updates for different tasks conflict, hindering effective parameter optimization. Ultimately, the weight parameters cannot be specialized for any specific task, leading to overall performance degradation. To address this issue, this application proposes a novel task-adaptive weight generation strategy that dynamically generates task-specific parameters for specialized processing, thereby eliminating potential interference.
[0057] Previous general models for natural images typically employ contrastive learning or auxiliary classification tasks to learn task-specific features to guide image restoration. However, these methods are not necessary for medical image restoration tasks. Due to significant semantic differences among various medical imaging modalities, the latent features encoding semantic information inherently exhibit task-specific differences. Therefore, as... Figure 3 As shown in the t-sne diagram, even simple features extracted directly from latent features can achieve task differentiation, thus eliminating the need for complex representation learning. Based on this, this application proposes a task feature extraction network to generate task-specific features Z.
[0058] Task interference occurs when different tasks conflict in the same direction of weight parameter updates. To address this issue, we propose generating task-specific parameters for each task. Utilizing task-specific features Z, this application employs a multi-stage decoder to estimate the weight parameters for each decoding Transformer module.
[0059] Step 206: Generate high-quality medical images using the image restoration model.
[0060] In the aforementioned medical image restoration method, a low-quality initial medical image is first acquired and input into an image restoration model to generate a high-quality medical image. The image restoration model extracts initial features from the initial medical image through an initial feature extraction network. Subsequently, a multi-stage encoder encodes the initial features into latent representations, which are then processed by a multi-stage decoder to generate deep features. A decoupled copy of the gradient is input into a task-specific feature extraction network to extract task-specific features. The decoder uses these task-specific features to generate task-adaptive weights, thereby achieving customized feature optimization during the decoding process. An image conversion network converts the deep features into a residual image, which is then element-wise added to the low-quality initial medical image to obtain the reconstructed high-quality medical image. This medical image restoration method can improve the quality of initial image models of different formats, resulting in high-quality medical images with improvements in generalization, efficiency, and data availability.
[0061] In some embodiments, generating high-quality medical images using an image restoration model includes: extracting features from an initial medical image using an initial feature extraction network to obtain initial features; performing multi-stage encoding processing on the initial features using a multi-stage encoder to obtain a latent representation; extracting gradient-decoupled copies of the latent representation using a task feature extraction network to generate task-specific features; using the task-specific features to guide the multi-stage decoder in weight allocation; generating task-adaptive weights based on the task-specific features using the multi-stage decoder; decoding the latent representation based on the task-adaptive weights to obtain deep features; performing image transformation on the deep features using an image transformation network to obtain a residual image; and performing summation operations on the initial medical image and the residual image using a computation network to obtain a high-quality medical image.
[0062] Optionally, the initial feature extraction network TAT (Task Adaptive Transformer) first extracts features from a low-quality initial medical image I through 3×3 convolutional layers. LQ ∈R H×W×1 Extracting initial feature I IF ∈R H×W×C Here, H, W, and C represent the spatial height, width, and channel dimension, respectively. TAT introduces an object-aware mechanism to identify and focus on the most task-critical parts of the input data. Traditional Transformer models treat all input elements (regardless of their task importance) equally when processing sequence data. TAT, however, guides the model to focus on task-relevant features, enabling the network to learn more effectively during training.
[0063] Subsequently, the Transformer-based multi-stage encoder encodes the initial features into latent representation I. LF The processing flow then splits into two branches: the first branch processes I through a multi-stage decoder. LF Processing is performed to generate deep features I DF The second branch will then take I LF Gradient-decoupled replica input task feature extraction network to extract task-specific features Z∈R d The decoder utilizes Z-generate task-adaptive weights to achieve customized feature optimization during the decoding process. Finally, the image transformation network converts the I... DF Convert to residual image I R ∈R H×W×1 Then, an element-wise addition operation is performed with the low-quality initial medical image to obtain a reconstructed high-quality output image: This means obtaining high-quality medical images.
[0064] In some embodiments, such as Figure 3As shown, the multi-stage encoder includes a first encoding module, a second encoding module, and a third encoding module. The initial features are encoded sequentially through the first encoding module, the second encoding module, and the third encoding module. In each encoding, the spatial height and width are halved, and the channel dimension is doubled to obtain the latent representation.
[0065] The first, second, and third encoding modules are all Transformer modules, corresponding to L1, L2, and L3 respectively. The first encoding module outputs H×W×C, the second encoding module outputs H / 2×W / 2×2C, and the third encoding module outputs H / 4×W / 4×4C and H / 8×W / 8×8C, representing the latent representation I. LF The input is fed into the TREN network for task feature extraction.
[0066] In some embodiments, the task feature extraction network includes multiple sequentially connected convolutional blocks, capable of directly extracting features from latent features. LF Extract task-specific features Z∈R d Its expression is:
[0067] Z = TREN(SG(I) LF ))
[0068] Where TREN(·) represents the task feature extraction network, and SG(·) represents the stopping gradient operator, which stops the latent features I LF The extraction of features is decoupled from the extraction of task-specific features Z to avoid interference between processes with different objectives.
[0069] In some embodiments, such as Figure 3 As shown, a multi-stage decoder generates task-adaptive weights based on task-specific features. These task-adaptive weights are then used to decode the latent representation to obtain deep features. The process includes: performing a first transformation on the task-specific features to obtain a first transformation result; performing a first normalization on the latent representation to obtain a first intermediate parameter; performing a first reshaping on the transformation result to obtain a first separable convolutional weight; summing the first separable convolutional weight with the first shared weight of the first intermediate parameter to obtain a second intermediate parameter; using a channel attention mechanism to assign weights to the second intermediate parameter to obtain a weight assignment result; performing a second normalization on the latent features based on the weight assignment result to obtain a third intermediate parameter; performing a second reshaping on the transformation result to obtain a second separable convolutional weight; summing the second separable convolutional weight with the second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter; performing a second transformation on the fifth intermediate parameter to obtain a second transformation result; and combining the third intermediate parameter and the second transformation result to obtain the deep features.
[0070] Task interference occurs when different tasks conflict in the same weight parameter update direction. To address this issue, this application proposes generating task-specific parameters for each task. Utilizing task-specific features Z, this application employs a multi-layer perceptron (MLP) to estimate the weight parameters for each decoding module.
[0071] Traditional methods for generating linear layers or standard convolutional weights face scalability issues: their parameter count increases quadratically with the channel dimension C (i.e., O(C)). 2 This leads to low computational efficiency and unreliable parameter estimation. To alleviate this problem, this application uses depthwise separable convolution 3x3Dconv, a lightweight alternative with only k×k×C parameters (where k is the kernel size), and the parameter count grows linearly (i.e., O(C)) because k << C.
[0072] Thus, depthwise separable convolutions, while preserving local spatial information, complement global attention mechanisms—a synergistic effect that has been demonstrated to improve performance in Vision Transformers. Furthermore, their parameter efficiency supports a precise and compact weight generation process.
[0073] Based on this, the weight generation process of this application can be formally represented as:
[0074] W G =Reshape(MLP(Z)),
[0075] Among them, W G This represents dynamically generated depthwise separable convolutional weights, obtained by first transforming Z through an MLP and then reshaping it into the shape of the target convolutional kernel. Finally, the generated task-specific weights W... G With the previously shared weight W S Sum:
[0076] W = W S +λW G ,
[0077] Where W represents the final weights of the depthwise separable convolution, and λ is a learnable parameter. By incorporating the generated task-specific weights into the Transformer module, the original Transformer module is transformed into a Weight Adaptive Transformer module (WATB), as shown below. Figure 3 As shown.
[0078] In one embodiment, the loss weights during the training of the image restoration model are represented as follows:
[0079]
[0080] Where L1(·) represents the L1 distance, and SG(·) represents the stopping gradient operator, used to decouple loss balance and model optimization. LQ ,I HQ ), and The three terms encode the training dynamics associated with the samples, and MLP(·) represents the transformation process.
[0081] Traditional integrated methods generally overlook the imbalance problem between tasks, where different tasks have varying learning difficulties, leading to some tasks dominating the optimization process while others are undertrained. In the field of multi-task learning, this problem has been initially addressed by employing loss balancing strategies, the core of which is dynamically allocating task-specific loss weights during training. This is typically expressed as:
[0082]
[0083] Where T represents the total number of tasks, L t Let σ represent the loss of the t-th task. t ∈R 1 Let be the learnable parameters. In the formula, The loss weights for each task are dynamically scaled, while logσ t Then regularize the scaling. When the loss L of a certain task t When σ is large and tends to dominate the overall loss function, t It will increase to suppress its weight, and vice versa. This mechanism can spontaneously balance the contribution of each task to the total loss, ensuring the balance of the training process without human intervention.
[0084] However, while this method is effective in task-level balancing, it lacks sample-level adaptability and faces difficulties in implementation within task-specific models. To overcome this limitation, this application proposes a novel task-adaptive balancing strategy by redefining σ∈R 1 The derivation method achieves balance at the sample level:
[0085]
[0086]
[0087] Where L1(·) represents the L1 distance, and SG(·) represents the stopping gradient operator, used to decouple loss balance from model optimization. LQ ,I HQ ), and The three terms encode the training dynamics related to the sample. By concatenating these three terms and inputting them into the MLP, this application achieves a balance at the fine-grained level by adaptively estimating σ for each sample. While this strategy derives σ from being based on the task index (σ in the above equation)... t It is transformed into a condition based on sample loss, but it is achieved through... The core mechanism for achieving dynamic weighting and regularization via logσ remains rooted in the original theory. This improvement ensures both the adaptive balance of the loss function and extends flexibility to the sample level.
[0088] Furthermore, regarding the model architecture, the number of feature extraction blocks in this method are L1=4, L2=L3=6, and L4=8, respectively. The number of residual blocks in TREN is L=2. The dimension of the task-specific representation Z is d=256. During model training, we use a total batch size of 12 (4 samples per dataset) and an image patch size of 128×128. The model is optimized using the AdamW optimizer with a learning rate of 1×10⁻⁶. -4 The number of training iterations is 4×10 5 In terms of model evaluation, PSNR, SSIM, and RMSE indicators were used to quantitatively evaluate recovery performance.
[0089] See Figure 4 and Table 1 below, Figure 4 The experimental results of the image restoration method provided in the above embodiments on three tasks—PET image synthesis, CT denoising, and MRI super-resolution—are shown. Compared with the AMIR (All-in-one Medical Image Restoration) algorithm, the results show that the present method outperforms the AMIR algorithm.
[0090] Table 1 Comparison results between this embodiment and the AMIR algorithm
[0091]
[0092] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0093] Based on the same inventive concept, this application also provides a medical image restoration apparatus for implementing the medical image restoration method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more medical image restoration apparatus embodiments provided below can be found in the limitations of the medical image restoration method described above, and will not be repeated here.
[0094] In one exemplary embodiment, such as Figure 5 As shown, a medical image restoration device 500 is provided, including: an acquisition module 502, an input module 504, and a generation module 506, wherein:
[0095] The acquisition module 502 is used to acquire initial medical images.
[0096] The input module 504 is used to input the initial medical image into the image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptive module, an image transformation network, and a computation network; wherein, the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features.
[0097] The generation module 506 is used to generate high-quality medical images through an image restoration model.
[0098] In some embodiments, the generation module 506 generates high-quality medical images using an image restoration model, including: extracting features from the initial medical image using an initial feature extraction network to obtain initial features; performing multi-stage encoding processing on the initial features using a multi-stage encoder to obtain latent representations; extracting gradient-decoupled copies of the latent representations using a task feature extraction network to generate task-specific features; using the task-specific features to guide the multi-stage decoder in weight allocation; generating task-adaptive weights based on the task-specific features using the multi-stage decoder; decoding the latent representations based on the task-adaptive weights to obtain depth features; performing image transformation on the depth features using an image transformation network to obtain a residual image; and performing summation operations on the initial medical image and the residual image using a computation network to obtain a high-quality medical image.
[0099] In some embodiments, the multi-stage encoder of the image restoration model includes a first encoding module, a second encoding module, and a third encoding module. The initial features are encoded sequentially through the first encoding module, the second encoding module, and the third encoding module. In each encoding, the spatial height and width are halved, and the channel dimension is doubled to obtain the latent representation.
[0100] In some embodiments, the task feature extraction network includes multiple sequentially connected convolutional blocks, capable of directly extracting features from latent features. LF Extract task-specific features Z∈R d Its expression is:
[0101] Z = TREN(SG(I) LF ))
[0102] Where TREN(·) represents the task feature extraction network, and SG(·) represents the stopping gradient operator, which stops the latent features I LF The extraction of features is decoupled from the extraction of task-specific features Z to avoid interference between processes with different objectives.
[0103] In some embodiments, the generation module 506 generates task-adaptive weights based on task-specific features using a multi-stage decoder, and decodes the latent representation based on the task-adaptive weights to obtain deep features, including: performing a first transformation on the task-specific features to obtain a first transformation result; performing a first normalization on the latent representation to obtain a first intermediate parameter; performing a first reshaping on the transformation result to obtain a first separable convolutional weight; summing the first separable convolutional weight with a first shared weight of the first intermediate parameter to obtain a second intermediate parameter; assigning weights to the second intermediate parameter using a channel attention mechanism to obtain a weight assignment result; performing a second normalization on the latent features based on the weight assignment result to obtain a third intermediate parameter; performing a second reshaping on the transformation result to obtain a second separable convolutional weight; summing the second separable convolutional weight with a second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter; performing a second transformation on the fifth intermediate parameter to obtain a second transformation result; and combining the third intermediate parameter and the second transformation result to obtain deep features.
[0104] In some embodiments, the loss weights during the training of the image restoration model are represented as follows:
[0105]
[0106] Where L1(·) represents the L1 distance, and SG(·) represents the stopping gradient operator, used to decouple loss balance and model optimization. LQ ,I HQ ), and The three terms encode the training dynamics associated with the samples, and MLP(·) represents the transformation process.
[0107] Each module in the aforementioned medical image restoration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0108] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores medical image data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a medical image recovery method.
[0109] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0110] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0111] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0112] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0115] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for medical image restoration, characterized in that, The method includes: Acquire initial medical images; The initial medical image is input into the image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptive module, an image transformation network, and a computation network; wherein, the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features; Generating high-quality medical images using the image restoration model includes: The initial features are obtained by extracting features from the initial medical image using the initial feature extraction network. The initial features are encoded in multiple stages by the multi-stage encoder to obtain the latent representation; The task feature extraction network extracts gradient-decoupled copies of the latent representation to generate task-specific features; these task-specific features are used to guide the multi-stage decoder in weight allocation. The multi-stage decoder generates task-adaptive weights based on the task-specific features, and the latent representation is decoded based on the task-adaptive weights to obtain deep features. The image conversion network is used to convert the depth features into an image, resulting in a residual image. The computational network performs summation operations on the initial medical image and the residual image to obtain a high-quality medical image.
2. The method according to claim 1, characterized in that, The multi-stage encoder includes a first encoding module, a second encoding module, and a third encoding module. The initial features are encoded sequentially through the first encoding module, the second encoding module, and the third encoding module. In each encoding, the spatial height and width are halved, and the channel dimension is doubled to obtain the latent representation.
3. The method according to claim 1, characterized in that, The task feature extraction network comprises multiple sequentially connected convolutional blocks, capable of directly extracting features from latent representations. Extract task-specific features Its expression is: in, This represents a task feature extraction network. This represents the stopping gradient operator, which will latently characterize... Extraction and task-specific features Extraction and decoupling are performed to avoid interference between processes with different objectives.
4. The method according to claim 1, characterized in that, The process involves generating task-adaptive weights based on the task-specific features using the multi-stage decoder, and then decoding the latent representation using these task-adaptive weights to obtain deep features, including: The task-specific features are subjected to a first transformation process to obtain a first transformation result; The potential representation is subjected to a first normalization process to obtain a first intermediate parameter; The transformation result is subjected to a first reshaping process to obtain a first separable convolution weight. The first separable convolution weight is summed with the first shared weight of the first intermediate parameter to obtain a second intermediate parameter. The second intermediate parameter is weighted using a channel attention mechanism to obtain the weighting result; Based on the weight allocation result, the potential representation is subjected to a second normalization process to obtain a third intermediate parameter; The transformation result is subjected to a second reshaping process to obtain a second separable convolution weight. The second separable convolution weight is then summed with the second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter. The fifth intermediate parameter is subjected to a second transformation process to obtain the second transformation result; The third intermediate parameter and the second transformation result are combined to obtain the depth feature.
5. A medical image restoration device, characterized in that, The device includes: The acquisition module is used to acquire initial medical images; An input module is used to input the initial medical image into an image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptive module, an image conversion network, and a computation network; wherein, the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features; A generation module is used to generate high-quality medical images through the image restoration model; it includes: extracting features from the initial medical image using the initial feature extraction network to obtain initial features; performing multi-stage encoding processing on the initial features using the multi-stage encoder to obtain latent representations; extracting gradient-decoupled copies of the latent representations using the task feature extraction network to generate task-specific features; the task-specific features are used to guide the multi-stage decoder in weight allocation; generating task-adaptive weights based on the task-specific features using the multi-stage decoder; decoding the latent representations based on the task-adaptive weights to obtain depth features; performing image transformation on the depth features using the image transformation network to obtain a residual image; and performing a summation operation on the initial medical image and the residual image using the computation network to obtain a high-quality medical image.
6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.
8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.