Medical image restoration method and device, computer equipment and storage medium

Abstract

The invention relates to a medical image restoration method and device, computer equipment and a storage medium. The medical image restoration method comprises the following steps: acquiring an initial medical image; inputting the initial medical image into an image restoration model; the image restoration model comprises an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder integrated with a weight adaptive module, an image conversion network and an operation network; wherein the task feature extraction network generates task exclusive features, and the multi-stage decoder generates task adaptive weights based on the task exclusive features; and generating a high-quality medical image through the image restoration model. By adopting the method, the quality of initial image models in different formats can be improved, a high-quality medical image is obtained, and generalization, efficiency and data availability are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a medical image restoration method, apparatus, computer equipment, and storage medium. Background Art

[0002] Medical image restoration technology aims to reconstruct low-quality images into high-quality images, thereby reducing patients' scanning time and radiation dose, increasing the throughput efficiency of large hospital equipment, reducing radiation damage to patients, and improving image quality.

[0003] Traditional integrated medical image restoration methods mostly use fixed-parameter models to solve medical image restoration tasks in different modalities, but their actual use is limited in terms of generalization, efficiency and data availability. Summary of the Invention

[0004] Based on this, it is necessary to provide a medical image restoration method, apparatus, computer equipment and storage medium to address the above technical problems.

[0005] In a first aspect, the present application provides a medical image restoration method, the method comprising:

[0006] Obtain initial medical images;

[0007] Inputting the initial medical image into an image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptation module, an image conversion network, and a computation network; wherein the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features;

[0008] High-quality medical images are generated through the image restoration model.

[0009] In one embodiment, generating a high-quality medical image using the image restoration model includes:

[0010] Extracting features from the initial medical image using the initial feature extraction network to obtain initial features;

[0011] Performing multi-stage encoding processing on the initial features by the multi-stage encoder to obtain a latent representation;

[0012] Extracting a gradient-decoupled copy of the latent representation through the task feature extraction network to generate task-specific features; the task-specific features are used to guide the multi-stage decoder to perform weight allocation;

[0013] generating task adaptive weights based on the task-specific features by the multi-stage decoder, and decoding the latent representation based on the task adaptive weights to obtain deep features;

[0014] Performing image conversion on the deep features through the image conversion network to obtain a residual image;

[0015] The initial medical image and the residual image are summed up by the computing network to obtain a high-quality medical image.

[0016] In one embodiment, the multi-stage encoder includes a first encoding module, a second encoding module and a third encoding module, and the initial features are encoded in sequence by the first encoding module, the second encoding module and the third encoding module, and the spatial height and width are halved and the channel dimension is doubled in each encoding to obtain a potential representation.

[0017] In one embodiment, the task feature extraction network includes a plurality of sequential convolution blocks connected in sequence, which can directly extract the latent features I LF Extract task-specific features Z∈R d , whose expression is:

[0018] Z=TREN(SG(I LF ))

[0019] Where TREN(·) represents the task feature extraction network, SG(·) represents the stop gradient operator, which converts the potential feature I LF The extraction of is decoupled from the extraction of task-specific features Z to avoid interference between the two processes with different objectives.

[0020] In one embodiment, generating task adaptive weights based on the task-specific features by the multi-stage decoder, and decoding the latent representation based on the task adaptive weights to obtain deep features includes:

[0021] Performing a first transformation on the task-specific feature to obtain a first transformation result;

[0022] Performing a first normalization process on the potential representation to obtain a first intermediate parameter;

[0023] Performing a first reshaping process on the transformation result to obtain a first separable convolution weight, and adding the first separable convolution weight to a first shared weight of the first intermediate parameter to obtain a second intermediate parameter;

[0024] The channel attention mechanism is used to weight the second intermediate parameter and obtain the weight distribution result;

[0025] Performing a second normalization process on the potential features based on the weight distribution result to obtain a third intermediate parameter;

[0026] performing a second reshaping process on the transformation result to obtain a second separable convolution weight, and adding the second separable convolution weight to the second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter;

[0027] performing a second transformation process on the fifth intermediate parameter to obtain a second transformation result;

[0028] The third intermediate parameter and the second transformation result are combined to obtain the depth feature.

[0029] In one embodiment, during the training of the image restoration model, the loss weight is expressed as:

[0030]

[0031] Among them, L1(·) represents the L1 distance, SG(·) represents the stop gradient operator, which is used to decouple loss balance and model optimization, and L1(I LQ ,I HQ ), and The three terms encode the training dynamics associated with the sample, and MLP(·) represents the transformation process.

[0032] In a second aspect, the present application further provides a medical image restoration device, comprising:

[0033] An acquisition module, used for acquiring initial medical images;

[0034] An input module, configured to input the initial medical image into an image restoration model; the image restoration model comprises an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptation module, an image conversion network, and a computation network; wherein the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features;

[0035] A generation module is used to generate high-quality medical images through the image restoration model.

[0036] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in any of the above embodiments when executing the computer program.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps in any of the above embodiments are implemented.

[0038] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps in any of the above embodiments when executed by a processor.

[0039] In the above-mentioned medical image restoration method, a low-quality initial medical image is first obtained, and the initial medical image is input into the image restoration model, and a high-quality medical image is generated by the image restoration model. The image restoration model extracts initial features from the initial medical image through an initial feature extraction network. Subsequently, the initial features are encoded into latent representations based on a multi-stage encoder, and the latent representations are processed by a multi-stage decoder to generate deep features. A gradient decoupled copy is input into a task feature extraction network to extract task-specific features. The decoder uses the task-specific features to generate task-adaptive weights, thereby achieving feature optimization and customization during the decoding process. The deep features are converted into residual images through an image conversion network, and an element-by-element addition operation is performed with the low-quality initial medical image to obtain a reconstructed high-quality medical image. Through the above-mentioned medical image restoration method, the quality of initial image models of different formats can be improved to obtain high-quality medical images, with improvements in generalization, efficiency, and data availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A diagram showing an application environment of a medical image restoration method in one embodiment;

[0042] Figure 2 1 is a flow chart of a medical image restoration method according to an embodiment;

[0043] Figure 3 is a structural block diagram of an image restoration model in one embodiment;

[0044] Figure 4 is an experimental comparison diagram with AMIR in another embodiment;

[0045] Figure 5 is a structural block diagram of a medical image restoration device in one embodiment;

[0046] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0048] Traditional dedicated models trained for a single task in the medical imaging field have the following problems: (1) Limited generalization. In complex multimodal image restoration tasks (such as PET / CT or PET / MRI), it is often necessary to process multiple medical image restoration tasks simultaneously. However, due to the inherent differences between imaging modalities and degradation types, dedicated models trained for a single task are difficult to adapt to other tasks, resulting in a significant decline in performance. (2) Inefficiency and model bloat. Dedicated models trained for a single task require repeated development work and resource investment. Each task requires an independent network architecture, training process, storage solution, and computing resources. This discrete development significantly increases costs and increases the difficulty of clinical deployment. (3) Data scarcity and low comprehensive utilization. Dedicated models trained for a single task rely on narrow dedicated datasets (the data scale in the medical imaging field is limited) and are therefore extremely susceptible to data shortages. This not only increases the risk of model overfitting, but also loses the potential for cross-task and cross-modality collaborative training due to its single-task training method.

[0049] In summary, the limitations of specialized models in terms of generalization, efficiency, and data availability have jointly restricted their scalability and practical application value. Traditional integrated medical image restoration methods often use fixed-parameter models to solve medical image restoration tasks for different modalities, and input different modal information into the model only as conditional information. This lacks consideration of the following two aspects: (1) Task interference, where different tasks have conflicting gradient update directions on the same parameter. (2) Task imbalance, where the inherent learning difficulty differences between tasks lead to unbalanced optimization problems.

[0050] The medical image restoration method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0051] In an exemplary embodiment, Figure 2 As shown, a medical image restoration method is provided, which is applied to Figure 1 Taking the server 102 in the example as an example, the following steps 202 to 206 are included. Among them:

[0052] Step 202: Acquire initial medical images.

[0053] For example, the initial medical image may be an image with a lower resolution.

[0054] Step 204: input the initial medical image into an image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptation module, an image conversion network, and an operation network; wherein the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features.

[0055] For example, the image restoration model is a trained task-adaptive Transformer network, such as Figure 3 As shown, Figure 3 Shows the structural block diagram of the image restoration model.

[0056] Most traditional all-in-one models share a common limitation: they rely on a single model with fixed parameters to handle multiple tasks. This "one-size-fits-all" approach often suffers from poor results due to "task interference," where gradient updates for different tasks conflict with each other, hindering effective parameter optimization. Ultimately, the weight parameters cannot be specialized for any particular task, leading to overall performance degradation. To address this issue, this application proposes a new task-adaptive weight generation strategy that dynamically generates task-specific parameters to achieve specialized processing, thereby eliminating potential interference.

[0057] In the past, general models for natural images usually used contrastive learning or auxiliary classification tasks to learn task-specific features to guide image restoration. However, these methods are not necessary for medical image restoration tasks. Due to the significant semantic differences between various medical imaging modalities, the latent features encoding semantic information essentially show differences in task-specific features. Therefore, Figure 3 As shown by the t-SNE, even if simple features are directly extracted from the latent features, it is possible to distinguish between tasks without resorting to complex representation learning. Based on this, this application proposes a task feature extraction network to generate task-specific features Z.

[0058] Task interference occurs when different tasks conflict in the update direction of the same weight parameters. To address this issue, we propose generating task-specific parameters for each task. Leveraging the task-specific features Z, we employ a multi-stage decoder to estimate the weight parameters for each decoding Transformer module.

[0059] Step 206: Generate a high-quality medical image using the image restoration model.

[0060] In the above-mentioned medical image restoration method, a low-quality initial medical image is first obtained, and the initial medical image is input into the image restoration model, and a high-quality medical image is generated by the image restoration model. The image restoration model extracts initial features from the initial medical image through an initial feature extraction network. Subsequently, the initial features are encoded into latent representations based on a multi-stage encoder, and the latent representations are processed by a multi-stage decoder to generate deep features. A gradient decoupled copy is input into a task feature extraction network to extract task-specific features. The decoder uses the task-specific features to generate task-adaptive weights, thereby achieving feature optimization and customization during the decoding process. The deep features are converted into residual images through an image conversion network, and an element-by-element addition operation is performed with the low-quality initial medical image to obtain a reconstructed high-quality medical image. Through the above-mentioned medical image restoration method, the quality of initial image models of different formats can be improved to obtain high-quality medical images, with improvements in generalization, efficiency, and data availability.

[0061] In some embodiments, high-quality medical images are generated through an image restoration model, including: extracting features from the initial medical image through an initial feature extraction network to obtain initial features; performing multi-stage encoding processing on the initial features through a multi-stage encoder to obtain a latent representation; extracting a gradient decoupled copy of the latent representation through a task feature extraction network to generate task-specific features; the task-specific features are used to guide the multi-stage decoder to perform weight allocation; generating task adaptive weights based on the task-specific features through the multi-stage decoder, decoding the latent representation based on the task adaptive weights to obtain deep features; performing image conversion on the deep features through an image conversion network to obtain a residual image; and performing a sum operation on the initial medical image and the residual image through a computational network to obtain a high-quality medical image.

[0062] Optionally, the initial feature extraction network TAT (Task Adaptive Transformer) first extracts the low-quality initial medical image I through a 3×3 convolutional layer. LQ ∈R H×W×1 Extract the initial features I IF ∈R H×W×C , where H, W, and C represent the spatial height, width, and channel dimensions, respectively. TAT introduces a target-aware mechanism to identify and focus on the most task-critical components of the input data. Traditional Transformer models treat all input elements equally when processing sequential data, regardless of their task importance. TAT, however, guides the model to focus on task-relevant features, enabling more efficient network learning during training.

[0063] Subsequently, the Transformer-based multi-stage encoder encodes the initial features into a latent representation I LF The processing flow is then divided into two branches: the first branch is a multi-stage decoder for I LF Processing to generate deep features I DF ; The second branch will I LF The gradient decoupled copy of the input task feature extraction network extracts the task-specific features Z∈R d The decoder uses Z to generate task-adaptive weights, thereby achieving feature optimization customization during the decoding process. Finally, I is converted to DF Convert to residual image I R ∈R H×W×1 , and perform element-wise addition operation with the low-quality initial medical image to obtain the reconstructed high-quality output image: That is to obtain high-quality medical images.

[0064] In some embodiments, as Figure 3As shown, the multi-stage encoder includes a first encoding module, a second encoding module and a third encoding module. The initial features are encoded by the first encoding module, the second encoding module and the third encoding module in sequence. In each encoding, the spatial height and width are halved and the channel dimension is doubled to obtain the potential representation.

[0065] The first encoding module, the second encoding module, and the third encoding module are all Transformer modules, corresponding to L1, L2, and L3 respectively. The first encoding module outputs H×W×C, the second encoding module outputs H / 2×W / 2×2C, and the third encoding module outputs H / 4×W / 4×4C and H / 8×W / 8×8C. LF Input into the task feature extraction network TREN.

[0066] In some embodiments, the task feature extraction network includes a plurality of sequential convolution blocks connected in sequence, which can directly extract the latent features I LF Extract task-specific features Z∈R d , whose expression is:

[0067] Z=TREN(SG(I LF ))

[0068] Among them, TREN(·) represents the task feature extraction network, SG(·) represents the stop gradient operator, and the stop gradient operator converts the potential feature I LF The extraction of is decoupled from the extraction of task-specific features Z to avoid interference between the two processes with different objectives.

[0069] In some embodiments, as Figure 3 As shown, a multi-stage decoder generates task adaptive weights based on task-specific features, and the latent representation is decoded based on the task adaptive weights to obtain deep features, including: performing a first transformation on the task-specific features to obtain a first transformation result; performing a first normalization on the latent representation to obtain a first intermediate parameter; performing a first reshaping on the transformation result to obtain a first separable convolution weight, and adding the first separable convolution weight and the first shared weight of the first intermediate parameter to obtain a second intermediate parameter; using a channel attention mechanism to weight-allocate the second intermediate parameter to obtain a weight allocation result; performing a second normalization on the latent features based on the weight allocation result to obtain a third intermediate parameter; performing a second reshaping on the transformation result to obtain a second separable convolution weight, and adding the second separable convolution weight and the second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter; performing a second transformation on the fifth intermediate parameter to obtain a second transformation result; combining the third intermediate parameter and the second transformation result to obtain deep features.

[0070] Task interference occurs when different tasks conflict in the update direction of the same weight parameter. To address this issue, this application proposes generating task-specific parameters for each task. Using the task-specific feature Z, this application uses a multi-layer perceptron (MLP) to estimate the weight parameters for each decoding module.

[0071] The traditional method of generating linear layers or standard convolution weights faces scalability problems: the number of parameters grows squared with the channel dimension C (i.e. O(C 2 ), resulting in inefficient computation and unreliable parameter estimation. To alleviate this problem, this application uses depthwise separable convolution 3x3Dconv, a lightweight alternative with only k×k×C parameters (where k is the convolution kernel size) and linear growth (i.e. O(C)) due to k<<C.

[0072] In this way, depthwise separable convolutions preserve local spatial information while complementing the global attention mechanism—a synergy that has been shown to improve performance in the Vision Transformer. Furthermore, its parameter efficiency supports a precise and compact weight generation process.

[0073] Based on this, the weight generation process of this application can be formally expressed as:

[0074] W G =Reshape(MLP(Z)),

[0075] Among them, W G Denotes the dynamically generated depth-wise separable convolution weights, which are obtained by first passing Z through the MLP transformation and then reshaping it into the target convolution kernel shape. Finally, the generated task-specific weights W G The weight W shared with the previous S Perform the summation:

[0076] W=W S +λW G ,

[0077] Where W is the final weight of the depthwise separable convolution and λ is a learnable parameter. By incorporating the generated task-specific weights into the Transformer module, the original Transformer module is transformed into a weight-adaptive Transformer module (WATB), as shown in Figure 3 shown.

[0078] In one embodiment, during the training of the image restoration model, the loss weight is expressed as:

[0079]

[0080] Among them, L1(·) represents the L1 distance, SG(·) represents the stop gradient operator, which is used to decouple loss balance and model optimization, and L1(I LQ ,I HQ ), and The three terms encode the training dynamics associated with the sample, and MLP(·) represents the transformation process.

[0081] Traditional integrated approaches generally ignore the imbalance between tasks, i.e., different tasks have different learning difficulties, resulting in some tasks dominating the optimization process while other tasks are undertrained. In the field of multi-task learning, this problem has been initially addressed by adopting loss balancing strategies, the core of which is to dynamically assign task-specific loss weights during training. It is usually expressed as:

[0082]

[0083] Among them, T represents the total number of tasks, L t represents the loss of the t-th task, σ t ∈R 1 is a learnable parameter. Dynamically scale the loss weights of each task, and logσ t Then the scaling is regularized. When the loss L of a task t When σ is large and tends to dominate the overall loss function, t The weight of each task will be increased to suppress it, and vice versa. This mechanism can spontaneously balance the contribution of each task to the total loss and ensure the balance of the training process without manual intervention.

[0084] However, although this method is effective in task-level balancing, it lacks sample-level adaptability and is difficult to implement in task-specific models. To eliminate this limitation, this application proposes a new task-adaptive balancing strategy by redefining σ∈R 1 The derivation method achieves balance at the sample level:

[0085]

[0086]

[0087] Where L1(·) represents the L1 distance and SG(·) represents the stop gradient operator, which is used to decouple loss balance and model optimization. LQ ,I HQ ), and The three terms encode the training dynamics related to the sample. By concatenating these three terms and feeding them into the MLP, this application is able to adaptively estimate σ for each sample, thereby achieving fine-grained balance. Although this strategy changes the derivation of σ from task index (σ in the above formula) t ) is transformed into a condition based on sample loss, but it is The core mechanism of implementing dynamic weighting and regularization via logσ remains rooted in the original theory. This improvement not only ensures the adaptive balance of the loss function but also extends flexibility to the sample level.

[0088] Furthermore, in terms of model architecture, the number of feature extraction blocks in this method is L1=4, L2=L3=6, and L4=8. The number of residual blocks in TREN is L=2. The dimension of the task-specific representation Z is d=256. During model training, we use a total batch size of 12 (4 samples per dataset) and an image patch size of 128×128. The model is optimized using the AdamW optimizer with a learning rate of 1×10 -4 , the number of training iterations is 4×10 5 In terms of model evaluation, PSNR, SSIM, and RMSE indicators are used to quantitatively evaluate the restoration performance.

[0089] See also Figure 4 and Table 1 below, Figure 4 The experimental results of the image restoration method provided by the above embodiment on three tasks: PET image synthesis, CT denoising and MRI super-resolution are shown. Compared with the AMIR (All-in-one Medical Image Restoration) algorithm, it is shown that the present method surpasses the AMIR algorithm.

[0090] Table 1 Comparison results between this embodiment and the AMIR algorithm

[0091]

[0092] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0093] Based on the same inventive concept, embodiments of the present application also provide a medical image restoration device for implementing the aforementioned medical image restoration method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more of the following embodiments of the medical image restoration device can be found in the aforementioned limitations on the medical image restoration method and will not be further elaborated here.

[0094] In an exemplary embodiment, Figure 5 As shown, a medical image restoration device 500 is provided, comprising: an acquisition module 502, an input module 504 and a generation module 506, wherein:

[0095] The acquisition module 502 is configured to acquire an initial medical image.

[0096] Input module 504 is configured to input the initial medical image into an image restoration model. The image restoration model comprises an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptation module, an image transformation network, and a computation network. The task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features.

[0097] The generating module 506 is configured to generate high-quality medical images using an image restoration model.

[0098] In some embodiments, in the generation module 506, a high-quality medical image is generated through an image restoration model, including: extracting features from the initial medical image through an initial feature extraction network to obtain initial features; performing multi-stage encoding processing on the initial features through a multi-stage encoder to obtain a latent representation; extracting a gradient decoupled copy of the latent representation through a task feature extraction network to generate task-specific features; the task-specific features are used to guide the multi-stage decoder to perform weight allocation; generating task adaptive weights based on the task-specific features through the multi-stage decoder, decoding the latent representation based on the task adaptive weights to obtain deep features; performing image conversion on the deep features through an image conversion network to obtain a residual image; and performing a sum operation on the initial medical image and the residual image through a computational network to obtain a high-quality medical image.

[0099] In some embodiments, the multi-stage encoder of the image restoration model includes a first encoding module, a second encoding module and a third encoding module, and the initial features are encoded in sequence by the first encoding module, the second encoding module and the third encoding module. In each encoding, the spatial height and width are halved and the channel dimension is doubled to obtain a potential representation.

[0100] In some embodiments, the task feature extraction network includes a plurality of sequential convolution blocks connected in sequence, which can directly extract the latent features I LF Extract task-specific features Z∈R d , whose expression is:

[0101] Z=TREN(SG(I LF ))

[0102] Among them, TREN(·) represents the task feature extraction network, SG(·) represents the stop gradient operator, and the stop gradient operator converts the potential feature I LF The extraction of is decoupled from the extraction of task-specific features Z to avoid interference between the two processes with different objectives.

[0103] In some embodiments, the generation module 506 generates task adaptive weights based on task-specific features through a multi-stage decoder, and decodes the potential representation based on the task adaptive weights to obtain deep features, including: performing a first transformation on the task-specific features to obtain a first transformation result; performing a first normalization on the potential representation to obtain a first intermediate parameter; performing a first reshaping on the transformation result to obtain a first separable convolution weight, adding the first separable convolution weight and the first shared weight of the first intermediate parameter to obtain a second intermediate parameter; using a channel attention mechanism to weight the second intermediate parameter to obtain a weight distribution result; performing a second normalization on the potential features based on the weight distribution result to obtain a third intermediate parameter; performing a second reshaping on the transformation result to obtain a second separable convolution weight, adding the second separable convolution weight and the second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter; performing a second transformation on the fifth intermediate parameter to obtain a second transformation result; combining the third intermediate parameter and the second transformation result to obtain deep features.

[0104] In some embodiments, during the training of the image restoration model, the loss weight is expressed as:

[0105]

[0106] Among them, L1(·) represents the L1 distance, SG(·) represents the stop gradient operator, which is used to decouple loss balance and model optimization, and L1(I LQ ,I HQ ), and The three terms encode the training dynamics associated with the sample, and MLP(·) represents the transformation process.

[0107] Each module in the medical image restoration device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0108] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store medical imaging data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a medical image restoration method is implemented.

[0109] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0110] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0111] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0112] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0113] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0114] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A medical image restoration method, characterized in that: The method comprises: Obtain initial medical images; Inputting the initial medical image into an image restoration model; the image restoration model includes an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptation module, an image conversion network, and a computation network; wherein the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features; High-quality medical images are generated through the image restoration model.

2. The method according to claim 1, characterized in that Generating high-quality medical images by using the image restoration model includes: Extracting features from the initial medical image using the initial feature extraction network to obtain initial features; Performing multi-stage encoding processing on the initial features by the multi-stage encoder to obtain a latent representation; Extracting a gradient-decoupled copy of the latent representation through the task feature extraction network to generate task-specific features; the task-specific features are used to guide the multi-stage decoder to perform weight allocation; generating task adaptive weights based on the task-specific features by the multi-stage decoder, and decoding the latent representation based on the task adaptive weights to obtain deep features; Performing image conversion on the depth features through the image conversion network to obtain a residual image; The initial medical image and the residual image are summed up by the computing network to obtain a high-quality medical image.

3. The method according to claim 2, characterized in that The multi-stage encoder includes a first encoding module, a second encoding module and a third encoding module. The initial features are encoded in sequence by the first encoding module, the second encoding module and the third encoding module. In each encoding, the spatial height and width are halved and the channel dimension is doubled to obtain a potential representation.

4. The method according to claim 2, characterized in that The task feature extraction network includes a plurality of sequential convolution blocks connected in sequence, which can directly extract the latent features I LF Extract task-specific features Z∈R d , whose expression is: Z=TREN(SG(I LF )) Among them, TREN(·) represents the task feature extraction network, SG(·) represents the stop gradient operator, which converts the potential feature I LF The extraction of is decoupled from the extraction of task-specific features Z to avoid interference between the two processes with different objectives.

5. The method according to claim 2, characterized in that Generating task adaptive weights based on the task-specific features by the multi-stage decoder, and decoding the potential representation based on the task adaptive weights to obtain deep features, including: Performing a first transformation on the task-specific feature to obtain a first transformation result; performing a first normalization process on the potential representation to obtain a first intermediate parameter; Performing a first reshaping process on the transformation result to obtain a first separable convolution weight, and adding the first separable convolution weight to a first shared weight of the first intermediate parameter to obtain a second intermediate parameter; The channel attention mechanism is used to weight the second intermediate parameter and obtain the weight distribution result; Performing a second normalization process on the potential features based on the weight distribution result to obtain a third intermediate parameter; performing a second reshaping process on the transformation result to obtain a second separable convolution weight, and adding the second separable convolution weight to the second shared weight of the third intermediate parameter to obtain a fifth intermediate parameter; performing a second transformation process on the fifth intermediate parameter to obtain a second transformation result; The third intermediate parameter and the second transformation result are combined to obtain the depth feature.

6. The method according to claim 5, characterized in that In the training of the image restoration model, the loss weight is expressed as: Among them, L1(·) represents the L1 distance, SG(·) represents the stop gradient operator, which is used to decouple loss balance and model optimization, and L1(I LQ ,I HQ ), and The three terms encode the training dynamics associated with the sample, and MLP(·) represents the transformation process.

7. A medical image restoration device, characterized in that: The device comprises: An acquisition module, used for acquiring initial medical images; An input module, configured to input the initial medical image into an image restoration model; the image restoration model comprises an initial feature extraction network, a multi-stage encoder, a task feature extraction network, a multi-stage decoder with an integrated weight adaptation module, an image conversion network, and a computation network; wherein the task feature extraction network generates task-specific features, and the multi-stage decoder generates task-adaptive weights based on the task-specific features; A generation module is used to generate high-quality medical images through the image restoration model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images