Medical image enhancement method and system, electronic equipment and storage medium

Abstract

The invention discloses a medical image enhancement method and system, electronic equipment and a storage medium. The method comprises the following steps: constructing an image enhancement model comprising a first self-encoder, a first self-decoder, a consistency control module and an implicit diffusion model; encoding the acquired plain-scan medical image into a pure noise feature map through a first auto-encoder; inputting the plain-scan medical image, the pure noise feature map and the time step into a consistency control module to obtain a first result; inputting the first result, the plurality of first coding results, the pure noise feature map and the time step into an implicit diffusion model to obtain a decoding feature map; and restoring the decoded feature map into a target medical image through the first self-decoder, wherein the target medical image is a medical image obtained after the plain-scan medical image is enhanced. According to the invention, the generated enhanced medical image and the plain-scan medical image can be ensured to be consistent in anatomical structure, and the enhancement effect of the medical image can be improved.

Description

Technical Field

[0001] The present application relates to the field of image enhancement technology, and in particular to a medical image enhancement method, system, electronic device and storage medium. Background Art

[0002] Computed tomography (CT) is an important imaging method for obtaining internal structural information of objects in a non-destructive way. It has many advantages such as high resolution, high sensitivity and multiple levels. It is one of the medical imaging diagnostic equipment with the largest installed capacity in my country and is widely used in various medical clinical examination fields. Enhanced CT is a method of highlighting lesions or other parts in CT images by injecting contrast agents into the patient's body. It has significant significance in the imaging diagnosis of liver cancer.

[0003] However, due to the presence of X-ray radiation during the CT scanning process, as people gradually understand the potential hazards of radiation, the issue of CT radiation dose has received more and more attention. At the same time, since CT contrast agents cannot be used for patients with poor basic conditions, renal failure, severe allergies, etc., enhanced CT has certain limitations in clinical application. However, when only plain scan CT is used for diagnosis, it is very easy to miss and misdiagnose liver lesions. Most current methods are difficult to accurately capture the image features in plain scan CT when generating enhanced CT images, and the image enhancement effect is relatively poor. Summary of the invention

[0004] The present application aims to propose a medical image enhancement method, system, electronic device and storage medium, which can ensure that the generated enhanced medical image is consistent with the plain scan medical image in anatomical structure, and can improve the enhancement effect of the medical image.

[0005] In a first aspect, an embodiment of the present application provides a medical image enhancement method, the method comprising:

[0006] Constructing an image enhancement model including a first autoencoder, a first autodecoder, a consistency control module, and an implicit diffusion model;

[0007] encoding the acquired plain scan medical image into a pure noise feature map by the first autoencoder;

[0008] Input the plain scan medical image, the pure noise feature map and the time step into the consistency control module to obtain a first result, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module; encode the time step by the first time encoder to obtain a first time encoding result; encode the plain scan medical image by the second autoencoder to obtain an output result of the second autoencoder; add the output result of the second autoencoder to the pure noise feature map to obtain a first addition result, input the first addition result and the first time encoding result into the first multi-scale encoder to obtain multiple first encoding results; input the last first encoding result of the multiple first encoding results and the first time encoding result into the first intermediate module to obtain a first result;

[0009] Input the first result, the multiple first encoding results, the pure noise feature map and the time step into the implicit diffusion model to obtain a decoding feature map, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder; encode the time step by the second time encoder to obtain a second time encoding result; input the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain multiple second encoding results; add the last second encoding result of the multiple second encoding results to the first result to obtain a second addition result; input the second addition result into the second intermediate module to obtain a second result; input the second result, the multiple second encoding results, the second time encoding result and the multiple first encoding results into the multi-scale decoder to obtain a decoding feature map;

[0010] The decoded feature map is restored to a target medical image through the first self-decoder, and the target medical image is a medical image enhanced from the plain scan medical image.

[0011] Compared with the prior art, the first aspect of the present application has the following beneficial effects:

[0012] The method obtains a first result by inputting a plain scan medical image, a pure noise feature map and a time step into a consistency control module, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module, and the consistency control module can ensure that the generated liver cancer enhanced CT image is consistent with the plain scan CT image in anatomical structure; the first result, multiple first encoding results, the pure noise feature map and the time step obtained by the consistency control module are input into an implicit diffusion model, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder, and the implicit representation of the image can be enhanced, that is, the image features are enhanced, and finally the enhanced implicit representation is restored to the target medical image, thereby obtaining a medical image enhanced for the plain scan medical image, that is, it can ensure that the generated enhanced medical image is consistent with the plain scan medical image in anatomical structure, and can improve the enhancement effect of the medical image.

[0013] In some embodiments, the first multi-scale encoder includes multiple groups of cascaded first encoding modules, the multiple groups of first encoding modules have the same structure, each group of the first encoding modules includes multiple cascaded first encoding sub-modules, and the multiple first encoding sub-modules have the same structure;

[0014] The step of inputting the first addition result and the first time encoding result into the first multi-scale encoder to obtain a plurality of first encoding results includes:

[0015] The first temporal encoding result is input into a first starting encoding submodule in a first starting encoding module through a first cross attention connection for encoding, wherein the first starting encoding module is the encoding module arranged at the front of multiple first encoding modules, and the first starting encoding submodule is the encoding submodule arranged at the front of multiple first encoding submodules. The encoding process of the first starting encoding submodule includes:

[0016] Perform SiLU activation function and linear layer operation on the first time encoding result to obtain a first operation result;

[0017] Normalizing, performing SiLU activation function and 3x3 convolution operation on the first addition result to obtain a second operation result;

[0018] Adding the first operation result and the second operation result to obtain a third operation result;

[0019] Perform normalization, SiLU activation function and 3x3 convolution operation on the third operation result to obtain a fourth operation result;

[0020] Adding the fourth operation result to the first addition result to obtain an output result of the first starting encoding submodule;

[0021] Taking the first time encoding result and the output result of the first starting encoding submodule as the input of the next first encoding submodule, iterating sequentially to the last first encoding submodule in the first starting encoding module, and obtaining the first encoding result of the first starting encoding module;

[0022] The first encoding result of the first starting encoding module is used as the input of the next group of first encoding modules, and iterates to the last group of first encoding modules in sequence to obtain multiple first encoding results, wherein a group of first encoding modules corresponds to one first encoding result.

[0023] In some embodiments, the first intermediate module includes a plurality of cascaded residual modules, and the plurality of residual modules have the same structure;

[0024] The step of inputting the last first encoding result of the plurality of first encoding results and the first time encoding result into the first intermediate module to obtain a first result comprises:

[0025] Inputting the last first encoding result of the multiple first encoding results and the first time encoding result into a starting residual module, the starting residual module is the residual module that is the frontmost of the multiple residual modules, and the processing process of the last first encoding result of the multiple first encoding results and the first time encoding result by the starting residual module includes:

[0026] Performing a 3x3 convolution operation on the last first encoding result among the multiple first encoding results to obtain a first convolution result;

[0027] Adding the first convolution result and the first time encoding result to obtain a first added feature result;

[0028] Adding the first added feature result to the last first encoding result among the multiple first encoding results to obtain a second added feature result;

[0029] Perform 3*3 convolution and SiLU activation function operations on the second added feature result to obtain the output result of the starting residual module;

[0030] The output result of the starting residual module is used as the input of the next residual module, and the process is iterated sequentially to the last residual module to obtain a first result.

[0031] In some embodiments, the second multi-scale encoder includes multiple groups of cascaded second encoding modules, the multiple groups of second encoding modules have the same structure, each group of the second encoding modules includes multiple cascaded second encoding sub-modules, and the multiple second encoding sub-modules have the same structure;

[0032] The step of inputting the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain a plurality of second encoding results includes:

[0033] The second temporal encoding result is input into the second starting encoding submodule in the second starting encoding module through the second cross attention connection for encoding, wherein the second starting encoding module is the encoding module arranged at the front of the plurality of second encoding modules, and the second starting encoding submodule is the encoding submodule arranged at the front of the plurality of second encoding submodules, and the encoding process of the second starting encoding submodule includes:

[0034] Perform SiLU activation function and linear layer operation on the second time encoding result to obtain a linear operation result;

[0035] Normalizing, performing SiLU activation function and 3x3 convolution operation on the pure noise feature map to obtain a first convolution operation result;

[0036] Adding the linear operation result and the first convolution operation result to obtain a feature addition result;

[0037] Normalizing, performing SiLU activation function and 3x3 convolution operation on the feature addition result to obtain a second convolution operation result;

[0038] Adding the second convolution operation result to the pure noise feature map to obtain an output result of the second starting coding submodule;

[0039] Using the second time encoding result and the output result of the second starting encoding submodule as the input of the next second encoding submodule, iterating sequentially to the last second encoding submodule in the second starting encoding module, and obtaining the second encoding result of the second starting encoding module;

[0040] The second encoding result of the second starting encoding module is used as the input of the next group of second encoding modules, and iterates to the last group of second encoding modules in sequence to obtain multiple second encoding results, wherein one group of second encoding modules corresponds to one second encoding result.

[0041] In some embodiments, the multi-scale decoder comprises a plurality of groups of cascaded decoding modules, the plurality of groups of decoding modules have the same structure, each group of decoding modules comprises a plurality of cascaded decoding submodules, the plurality of decoding submodules have the same structure, and the plurality of groups of decoding modules have the same scale as the plurality of groups of first encoding modules and the plurality of groups of second encoding modules;

[0042] The step of inputting the second result, the plurality of second encoding results, the second time encoding result, and the plurality of first encoding results into the multi-scale decoder to obtain a decoding feature graph comprises:

[0043] The second result, the second encoding result corresponding to the second encoding module of the same scale as the starting decoding module, the second time encoding result, and the first encoding result corresponding to the first encoding module of the same scale as the starting decoding module are input into the starting decoding module for decoding to obtain a first decoding result. The starting decoding module is the frontmost decoding module in the plurality of groups of decoding modules. The decoding process of the starting decoding module includes:

[0044] Adding the second result to a first encoding result corresponding to a first encoding module of the same scale as the starting decoding module to obtain a first addition output result;

[0045] Input the first addition output result, the second time encoding result and the second encoding module of the same scale as the starting decoding module into the first decoding submodule in the starting decoding module to obtain the output result of the first decoding submodule in the starting decoding module;

[0046] Inputting the output result of the first decoding submodule in the starting decoding module and the second time encoding result into the second decoding submodule in the starting decoding module to obtain the output result of the second decoding submodule in the starting decoding module;

[0047] Inputting the output result of the second decoding submodule in the starting decoding module and the second time encoding result into the third decoding submodule in the starting decoding module to obtain the first decoding result corresponding to the starting decoding module;

[0048] The first decoding result, the second encoding result corresponding to the second encoding module of the same scale as the next group of decoding modules, the second time encoding result and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules are used as inputs of the next group of decoding modules, and the process is iterated sequentially to the last group of decoding modules to obtain a decoding feature map.

[0049] In some embodiments, the first autoencoder includes a first input convolutional layer, a feature extraction part and a first intermediate layer, the feature extraction part includes a plurality of feature extraction modules, each of the feature extraction modules includes a residual module and a downsampling layer, and the first intermediate layer includes a residual module with the same structure as the feature extraction part; encoding the acquired plain scan medical image into a pure noise feature map by the first autoencoder includes:

[0050] Inputting the plain scan medical image into the first input convolution layer to obtain convolved features;

[0051] The convolved features are input into a first feature extraction module in the feature extraction part for processing, and the processing process of the first feature extraction module includes:

[0052] For the residual module in the first feature extraction module, the convolved feature is normalized and convolved twice to obtain a first feature; the first feature is added to the convolved feature to obtain an output result of the residual module;

[0053] Downsampling the output result of the residual module to obtain the output result of the first feature extraction module;

[0054] Using the output result of the first feature extraction module as the input of the next feature extraction module, iterating to the last feature extraction module in turn, and obtaining the output result of the feature extraction part;

[0055] Inputting the output result of the feature extraction part into the residual module in the first intermediate layer to obtain a residual result;

[0056] Perform a self-attention operation on the residual result to obtain a self-attention feature;

[0057] The self-attention features are subjected to residual module processing, normalization, and convolution operations in the first intermediate layer to obtain a pure noise feature map.

[0058] In some embodiments, the first self-decoder includes a second input convolutional layer, a feature restoration part, and a second intermediate layer, the feature restoration part includes a plurality of feature restoration modules, each of the feature restoration modules includes a residual module and an upsampling layer, and the second intermediate layer includes a residual module with the same structure as the feature restoration part; restoring the decoded feature map to a target medical image through the first self-decoder includes:

[0059] Inputting the decoded feature map into the second input convolutional layer to obtain output features of the second input convolutional layer;

[0060] Inputting the output features of the second input convolutional layer into the residual module in the second intermediate layer to obtain an output result of the residual module;

[0061] Perform a self-attention operation on the output result of the residual module to obtain a self-attention output feature;

[0062] The self-attention output feature is processed by the residual module in the second intermediate layer, normalized and convolved to obtain an initial decoding feature;

[0063] The initial decoded features are input into a first feature restoration module in the feature restoration part for processing, wherein the processing process of the first feature restoration module includes:

[0064] For the residual module in the first feature restoration module, the initial decoding feature is normalized and convolved twice to obtain a first decoding feature; the first decoding feature is added to the initial decoding feature to obtain an output result of the residual module;

[0065] Upsampling the output result of the residual module to obtain the output result of the first feature restoration module;

[0066] Using the output result of the first feature restoration module as the input of the next feature restoration module, iterating to the last feature restoration module in turn, and obtaining the output result of the feature restoration part;

[0067] The output result of the feature restoration part is normalized and convolved to obtain a target medical image.

[0068] In a second aspect, an embodiment of the present application further provides a medical image enhancement system, the system comprising:

[0069] A model building unit, used to build an image enhancement model including a first autoencoder, a first autodecoder, a consistency control module and an implicit diffusion model;

[0070] An image encoding unit, configured to encode the acquired plain scan medical image into a pure noise feature map through the first autoencoder;

[0071] A first processing unit is used to input the plain scan medical image, the pure noise feature map and the time step into the consistency control module to obtain a first result, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module; the time step is encoded by the first time encoder to obtain a first time encoding result; the plain scan medical image is encoded by the second autoencoder to obtain an output result of the second autoencoder; the output result of the second autoencoder is added to the pure noise feature map to obtain a first addition result, and the first addition result and the first time encoding result are input into the first multi-scale encoder to obtain multiple first encoding results; the last first encoding result of the multiple first encoding results and the first time encoding result are input into the first intermediate module to obtain a first result;

[0072] a second processing unit, configured to input the first result, the multiple first encoding results, the pure noise feature map and the time step into the implicit diffusion model to obtain a decoding feature map, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder; encode the time step by the second time encoder to obtain a second time encoding result; input the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain multiple second encoding results; add the last second encoding result of the multiple second encoding results to the first result to obtain a second addition result; input the second addition result into the second intermediate module to obtain a second result; input the second result, the multiple second encoding results, the second time encoding result and the multiple first encoding results into the multi-scale decoder to obtain a decoding feature map;

[0073] An image restoration unit is used to restore the decoded feature map into a target medical image through the first self-decoder, and the target medical image is a medical image enhanced from the plain scan medical image.

[0074] In a third aspect, an embodiment of the present application also provides an electronic device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a medical image enhancement method as described above.

[0075] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a medical image enhancement method as described above.

[0076] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0078] Figure 1 is a flowchart of an embodiment of a medical image enhancement method provided by the present application;

[0079] Figure 2 It is a schematic diagram of a denoising iterative structure in a best embodiment of a medical image enhancement method provided by the present application;

[0080] Figure 3 It is a structural schematic diagram of an overall network model in a best embodiment of a medical image enhancement method provided by the present application;

[0081] Figure 4 It is a structural schematic diagram of an embodiment of a medical image enhancement system provided by the present application. DETAILED DESCRIPTION

[0082] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0083] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0084] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0085] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0086] Computed tomography (CT) is an important imaging method for obtaining internal structural information of objects in a non-destructive way. It has many advantages such as high resolution, high sensitivity and multiple levels. It is one of the medical imaging diagnostic equipment with the largest installed capacity in my country and is widely used in various medical clinical examination fields. Enhanced CT is a method of highlighting lesions or other parts in CT images by injecting contrast agents into the patient's body. It has significant significance in the imaging diagnosis of liver cancer.

[0087] However, due to the presence of X-ray radiation during the CT scanning process, as people gradually understand the potential hazards of radiation, the issue of CT radiation dose has received more and more attention. At the same time, since CT contrast agents cannot be used for patients with poor basic conditions, renal failure, severe allergies, etc., enhanced CT has certain limitations in clinical application. However, when only plain scan CT is used for diagnosis, it is very easy to miss and misdiagnose liver lesions. Most current methods are difficult to accurately capture the image features in plain scan CT when generating enhanced CT images, and the image enhancement effect is relatively poor.

[0088] In order to solve the above-mentioned problem of poor image enhancement effect, the present application proposes a medical image enhancement method, system, electronic device and storage medium.

[0089] Reference Figure 1 The present invention provides a method for medical image enhancement, which comprises the following steps:

[0090] Step S100, constructing an image enhancement model including a first autoencoder, a first autodecoder, a consistency control module and an implicit diffusion model;

[0091] Step S200, encoding the acquired plain scan medical image into a pure noise feature map through a first autoencoder;

[0092] Step S300, inputting the plain scan medical image, the pure noise feature map and the time step into the consistency control module to obtain a first result, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module; encoding the time step by the first time encoder to obtain a first time encoding result; encoding the plain scan medical image by the second autoencoder to obtain an output result of the second autoencoder; adding the output result of the second autoencoder to the pure noise feature map to obtain a first addition result, inputting the first addition result and the first time encoding result into the first multi-scale encoder to obtain multiple first encoding results; inputting the last first encoding result of the multiple first encoding results and the first time encoding result into the first intermediate module to obtain the first result;

[0093] Step S400, inputting the first result, multiple first encoding results, a pure noise feature map and a time step into an implicit diffusion model to obtain a decoding feature map, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder; encoding the time step by the second time encoder to obtain a second time encoding result; inputting the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain multiple second encoding results; adding the last second encoding result of the multiple second encoding results to the first result to obtain a second addition result; inputting the second addition result into the second intermediate module to obtain a second result; inputting the second result, multiple second encoding results, the second time encoding result and the multiple first encoding results into the multi-scale decoder to obtain a decoding feature map;

[0094] Step S500: restore the decoded feature map to a target medical image through a first self-decoder, where the target medical image is a medical image enhanced from a plain scan medical image.

[0095] In this embodiment, a first result is obtained by inputting a plain scan medical image, a pure noise feature map and a time step into a consistency control module, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module, and the consistency control module can ensure that the generated liver cancer enhanced CT image is consistent with the plain scan CT image in anatomical structure; the first result, multiple first encoding results, a pure noise feature map and a time step obtained by the consistency control module are input into an implicit diffusion model, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder, and the implicit representation of the image can be enhanced, that is, the image features are enhanced, and finally the enhanced implicit representation is restored to the target medical image, thereby obtaining a medical image enhanced for the plain scan medical image, that is, it can ensure that the generated enhanced medical image is consistent with the plain scan medical image in anatomical structure, and can improve the enhancement effect of the medical image.

[0096] In some embodiments, the first multi-scale encoder includes multiple groups of cascaded first encoding modules, the multiple groups of first encoding modules have the same structure, each group of first encoding modules includes multiple cascaded first encoding sub-modules, and the multiple first encoding sub-modules have the same structure;

[0097] Inputting the first addition result and the first time encoding result into a first multi-scale encoder to obtain a plurality of first encoding results, including:

[0098] The first time encoding result is input into the first starting encoding submodule in the first starting encoding module through the first cross attention connection for encoding, the first starting encoding module is the encoding module arranged at the front of the multiple first encoding modules, the first starting encoding submodule is the encoding submodule arranged at the front of the multiple first encoding submodules, and the encoding process of the first starting encoding submodule includes:

[0099] Perform SiLU activation function and linear layer operation on the first time encoding result to obtain a first operation result;

[0100] The first addition result is normalized, SiLU activation function and 3x3 convolution operation are performed to obtain a second operation result;

[0101] Adding the first operation result and the second operation result to obtain a third operation result;

[0102] Normalize the third operation result, perform SiLU activation function and 3x3 convolution operation, and obtain a fourth operation result;

[0103] Adding the fourth operation result to the first addition result to obtain an output result of the first starting encoding submodule;

[0104] Taking the first time encoding result and the output result of the first starting encoding submodule as the input of the next first encoding submodule, iterating sequentially to the last first encoding submodule in the first starting encoding module, and obtaining the first encoding result of the first starting encoding module;

[0105] The first encoding result of the first starting encoding module is used as the input of the next group of first encoding modules, and iterates to the last group of first encoding modules in sequence to obtain multiple first encoding results, wherein one group of first encoding modules corresponds to one first encoding result.

[0106] In this embodiment, the first addition result and the first time encoding result are input into the first multi-scale encoder, and the first multi-scale encoder includes multiple groups of cascaded first encoding modules, the multiple groups of first encoding modules have the same structure, each group of first encoding modules includes multiple cascaded first encoding sub-modules, and the multiple first encoding sub-modules have the same structure. Through multiple encoding modules and encoding sub-modules with the same structure, feature extraction can be better performed.

[0107] The above-mentioned method of using the first time coding result and the output result of the first starting coding submodule as the input of the next first coding submodule, and iterating sequentially to the last first coding submodule in the first starting coding module to obtain the first coding result of the first starting coding module, can be that the first time coding result and the output result of the first starting coding submodule are used as the input of the next first coding submodule. Since the structure of each first coding submodule is the same, the next first coding submodule is encoded in the same way as the encoding process of the first starting coding submodule, except that the input of the next first coding submodule is the output result of the previous first coding submodule and the first time coding result. The same encoding method is used to iterate sequentially to the last first coding submodule in the first starting coding module to obtain the first coding result of the first starting coding module.

[0108] The above-mentioned process of using the first encoding result of the first starting encoding module as the input of the next group of first encoding modules, iterating sequentially to the last group of first encoding modules, and obtaining multiple first encoding results, can be that the first encoding result of the first starting encoding module is used as the input of the next group of first encoding modules. Since the structure of each first encoding module is the same, the next first encoding module is encoded in the same manner as the encoding process of the first starting encoding module, except that the input of the next first encoding module is the first encoding result of the previous first encoding module. The same encoding method is used, iterating sequentially to the last group of first encoding modules, and obtaining multiple first encoding results.

[0109] In some embodiments, the first intermediate module includes a plurality of cascaded residual modules, and the plurality of residual modules have the same structure;

[0110] Inputting the last first encoding result and the first time encoding result of the plurality of first encoding results into the first intermediate module to obtain a first result, including:

[0111] Inputting the last first encoding result and the first time encoding result of the multiple first encoding results into the starting residual module, the starting residual module is the residual module that is the frontmost of the multiple residual modules, and the processing process of the last first encoding result and the first time encoding result of the multiple first encoding results by the starting residual module includes:

[0112] Performing a 3x3 convolution operation on the last first encoding result among the multiple first encoding results to obtain a first convolution result;

[0113] Adding the first convolution result and the first time encoding result to obtain a first added feature result;

[0114] Adding the first added feature result to the last first encoding result among the plurality of first encoding results to obtain a second added feature result;

[0115] The second added feature result is subjected to 3*3 convolution and SiLU activation function operations to obtain the output result of the starting residual module;

[0116] The output result of the starting residual module is used as the input of the next residual module, and the process is iterated to the last residual module to obtain the first result.

[0117] In this embodiment, multiple residual modules with the same structure are used to enable information interaction between different levels, thereby enhancing the model's ability to integrate global and local information.

[0118] The output result of the starting residual module is used as the input of the next residual module, and it is iterated to the last residual module in sequence to obtain the first result. The output result of the starting residual module can be used as the input of the next residual module. Since the structure of each residual module is the same, the next residual module adopts the same processing process as the starting residual module, except that the input of the next residual module is the output result of the previous residual module. The same residual module processing method is adopted, and it is iterated to the last residual module in sequence to obtain the first result.

[0119] In some embodiments, the second multi-scale encoder includes multiple groups of cascaded second encoding modules, the multiple groups of second encoding modules have the same structure, each group of second encoding modules includes multiple cascaded second encoding sub-modules, and the multiple second encoding sub-modules have the same structure;

[0120] The second time encoding result and the pure noise feature map are input into a second multi-scale encoder to obtain a plurality of second encoding results, including:

[0121] The second time encoding result is input into the second starting encoding submodule in the second starting encoding module through the second cross attention connection for encoding, the second starting encoding module is the encoding module arranged at the front of the plurality of second encoding modules, the second starting encoding submodule is the encoding submodule arranged at the front of the plurality of second encoding submodules, and the encoding process of the second starting encoding submodule includes:

[0122] Perform SiLU activation function and linear layer operation on the second time encoding result to obtain a linear operation result;

[0123] The pure noise feature map is normalized, SiLU activated and 3x3 convolution is performed to obtain the result of the first convolution operation;

[0124] Add the linear operation result and the first convolution operation result to obtain a feature addition result;

[0125] The feature addition result is normalized, SiLU activation function and 3x3 convolution operation are performed to obtain the result of the second convolution operation;

[0126] Adding the second convolution operation result to the pure noise feature map to obtain the output result of the second starting coding submodule;

[0127] Using the second time encoding result and the output result of the second starting encoding submodule as the input of the next second encoding submodule, iterating sequentially to the last second encoding submodule in the second starting encoding module, and obtaining the second encoding result of the second starting encoding module;

[0128] The second encoding result of the second starting encoding module is used as the input of the next group of second encoding modules, and iterates to the last group of second encoding modules in sequence to obtain multiple second encoding results, wherein one group of second encoding modules corresponds to one second encoding result.

[0129] In this embodiment, the second time encoding result and the pure noise feature map are input into the second multi-scale encoder, and the second multi-scale encoder includes multiple groups of cascaded second encoding modules, the multiple groups of second encoding modules have the same structure, each group of second encoding modules includes multiple cascaded second encoding sub-modules, and the multiple second encoding sub-modules have the same structure. Through multiple encoding modules and encoding sub-modules with the same structure, better feature extraction can be performed, which can lay a good data foundation for later enhanced image effects.

[0130] The above-mentioned method of using the second time coding result and the output result of the second starting coding submodule as the input of the next second coding submodule, and iterating sequentially to the last second coding submodule in the second starting coding module to obtain the second coding result of the second starting coding module, can be that the second time coding result and the output result of the second starting coding submodule are used as the input of the next second coding submodule. Since the structure of each second coding submodule is the same, the next second coding submodule is encoded in the same way as the encoding process of the second starting coding submodule, except that the input of the next second coding submodule is the output result of the previous second coding submodule and the second time coding result. The same encoding method is used to iterate sequentially to the last second coding submodule in the second starting coding module to obtain the second coding result of the second starting coding module.

[0131] The above-mentioned process of using the second encoding result of the second starting encoding module as the input of the next group of second encoding modules, iterating sequentially to the last group of second encoding modules, and obtaining multiple second encoding results, can be that the second encoding result of the second starting encoding module is used as the input of the next group of second encoding modules. Since the structure of each second encoding module is the same, the next second encoding module is encoded in the same manner as the encoding process of the second starting encoding module, except that the input of the next second encoding module is the second encoding result of the previous second encoding module. The same encoding method is used, iterating sequentially to the last group of second encoding modules, and obtaining multiple second encoding results.

[0132] In some embodiments, the multi-scale decoder comprises a plurality of groups of cascaded decoding modules, the plurality of groups of decoding modules have the same structure, each group of decoding modules comprises a plurality of cascaded decoding submodules, the plurality of decoding submodules have the same structure, and the plurality of groups of decoding modules have the same scale as the plurality of groups of first encoding modules and the plurality of groups of second encoding modules;

[0133] Inputting the second result, the plurality of second encoding results, the second time encoding result, and the plurality of first encoding results into a multi-scale decoder to obtain a decoding feature map, including:

[0134] The second result, the second encoding result corresponding to the second encoding module of the same scale as the starting decoding module, the second time encoding result, and the first encoding result corresponding to the first encoding module of the same scale as the starting decoding module are input into the starting decoding module for decoding to obtain a first decoding result. The starting decoding module is the frontmost decoding module in the multiple groups of decoding modules. The decoding process of the starting decoding module includes:

[0135] Adding the second result to the first encoding result corresponding to the first encoding module of the same scale as the starting decoding module to obtain a first addition output result;

[0136] Inputting the first addition output result, the second time encoding result and the second encoding module of the same scale as the starting decoding module into the first decoding submodule in the starting decoding module to obtain the output result of the first decoding submodule in the starting decoding module;

[0137] Inputting the output result of the first decoding submodule in the starting decoding module and the second time encoding result into the second decoding submodule in the starting decoding module to obtain the output result of the second decoding submodule in the starting decoding module;

[0138] Inputting the output result of the second decoding submodule in the starting decoding module and the second time encoding result into the third decoding submodule in the starting decoding module to obtain the first decoding result corresponding to the starting decoding module;

[0139] The first decoding result, the second encoding result corresponding to the second encoding module of the same scale as the next group of decoding modules, the second time encoding result and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules are taken as inputs of the next group of decoding modules, and iterate in sequence to the last group of decoding modules to obtain a decoding feature map.

[0140] In this embodiment, the second result, multiple second encoding results, second time encoding results and multiple first encoding results are input into a multi-scale decoder to obtain a decoding feature map. Since the scales of the multiple groups of decoding modules are the same as the scales of the multiple groups of first encoding modules and the multiple groups of second encoding modules, the second result, the second encoding result corresponding to the second encoding module with the same scale as the starting decoding module, the second time encoding result and the first encoding result corresponding to the first encoding module with the same scale as the starting decoding module are input into the starting decoding module for decoding to obtain the first decoding result, which can better decode the features extracted by the encoding module. Since the encoding module continuously increases the feature depth when extracting features, it can more accurately capture the features in the plain scan medical image. Therefore, the enhanced image feature representation can be obtained through the decoding module, thereby laying a good data foundation for later restoration of the image.

[0141] The above-mentioned first decoding result, the second encoding result corresponding to the second encoding module of the same scale as the next group of decoding modules, the second time encoding result and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules are used as the input of the next group of decoding modules, and iterates to the last group of decoding modules in sequence to obtain the decoding feature map. The first decoding result, the second encoding result corresponding to the second encoding module of the same scale as the next group of decoding modules, the second time encoding result and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules are used as the input of the next group of decoding modules. Since the structure of each group of decoding modules is the same, the next group of decoding modules is decoded in the same way as the decoding process of the starting decoding module, except that the input of the next group of decoding modules is the first decoding result of the previous group of decoding modules and the second encoding result corresponding to the second encoding module of the same scale as the next group of decoding modules, the second time encoding result and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules. The same encoding method is used to iterate to the last group of decoding modules in sequence to obtain the decoding feature map.

[0142] For example, the first decoding result, the second encoding result corresponding to the second encoding module of the same scale as the next group of decoding modules, the second time encoding result, and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules are used as inputs of the next group of decoding modules; the decoding process of the next group of decoding modules includes:

[0143] Adding the first decoding result and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules to obtain the output result of the next group of decoding modules;

[0144] Inputting the output results of the next group of decoding modules, the second time encoding results, and the second encoding modules of the same scale as the next group of decoding modules into the first decoding submodule in the next group of decoding modules, and obtaining the output results of the first decoding submodule in the next group of decoding modules;

[0145] Inputting the output result of the first decoding submodule in the next group of decoding modules and the second time encoding result into the second decoding submodule in the next group of decoding modules to obtain the output result of the second decoding submodule in the next group of decoding modules;

[0146] The output result of the second decoding submodule in the next group of decoding modules and the second time encoding result are input into the third decoding submodule in the next group of decoding modules to obtain the first decoding result corresponding to the next group of decoding modules.

[0147] In some embodiments, the first autoencoder includes a first input convolutional layer, a feature extraction part and a first intermediate layer, the feature extraction part includes a plurality of feature extraction modules, each feature extraction module includes a residual module and a downsampling layer, and the first intermediate layer includes a residual module with the same structure as the feature extraction part; encoding the acquired plain scan medical image into a pure noise feature map by the first autoencoder includes:

[0148] Input the plain scan medical image into the first input convolution layer to obtain the convolved features;

[0149] The convolved features are input into the first feature extraction module in the feature extraction part for processing. The processing process of the first feature extraction module includes:

[0150] For the residual module in the first feature extraction module, the convolved feature is normalized and convolved twice to obtain the first feature; the first feature is added to the convolved feature to obtain the output result of the residual module;

[0151] Downsampling the output result of the residual module to obtain the output result of the first feature extraction module;

[0152] The output result of the first feature extraction module is used as the input of the next feature extraction module, and the output result of the feature extraction part is obtained by iterating to the last feature extraction module in sequence;

[0153] Input the output result of the feature extraction part into the residual module in the first intermediate layer to obtain a residual result;

[0154] Perform self-attention operation on the residual result to obtain self-attention features;

[0155] The self-attention features are processed by the residual module in the first intermediate layer, normalized and convolved to obtain a pure noise feature map.

[0156] In this embodiment, in order to increase the learning efficiency of the overall model, the plain scan medical image is encoded into an implicit representation through the first autoencoder, and the implicit representation is used as the input of the implicit diffusion model, so that the image features can be better learned, thereby laying a good data foundation for the later implicit diffusion model.

[0157] In the above, the output result of the first feature extraction module is used as the input of the next feature extraction module, and it is iterated to the last feature extraction module in sequence to obtain the output result of the feature extraction part. The output result of the first feature extraction module can be used as the input of the next feature extraction module. Since the structures of multiple feature extraction modules are the same, the next feature extraction module adopts the same processing method as the first feature extraction module, except that the input of the next feature extraction module is the output result of the previous feature extraction module. The same processing method is used to iterate to the last feature extraction module in sequence to obtain the output result of the feature extraction part.

[0158] In some embodiments, the first self-decoder includes a second input convolutional layer, a feature restoration part, and a second intermediate layer, the feature restoration part includes a plurality of feature restoration modules, each feature restoration module includes a residual module and an upsampling layer, and the second intermediate layer includes a residual module with the same structure as the feature restoration part; restoring the decoded feature map to a target medical image through the first self-decoder includes:

[0159] Input the decoded feature map into the second input convolution layer to obtain the output feature of the second input convolution layer;

[0160] Inputting the output features of the second input convolutional layer into the residual module in the second intermediate layer to obtain the output result of the residual module;

[0161] Perform self-attention operation on the output result of the residual module to obtain the self-attention output feature;

[0162] The self-attention output features are processed by the residual module in the second intermediate layer, normalized and convolved to obtain the initial decoding features;

[0163] The initial decoded features are input into the first feature restoration module in the feature restoration part for processing. The processing process of the first feature restoration module includes:

[0164] For the residual module in the first feature restoration module, the initial decoding feature is normalized and convolved twice to obtain a first decoding feature; the first decoding feature is added to the initial decoding feature to obtain an output result of the residual module;

[0165] Upsampling the output result of the residual module to obtain the output result of the first feature restoration module;

[0166] The output result of the first feature restoration module is used as the input of the next feature restoration module, and the output result of the feature restoration part is obtained by iterating to the last feature restoration module in sequence;

[0167] The output results of the feature restoration part are normalized and convolved to obtain the target medical image.

[0168] In this embodiment, since the decoding feature map is a relatively accurate enhanced image feature obtained through multiple cascaded decoding modules and decoding sub-modules, the image enhancement effect can be improved by restoring the accurate enhanced image feature through the first self-decoder.

[0169] The output result of the first feature restoration module is used as the input of the next feature restoration module, and it is iterated to the last feature restoration module in sequence to obtain the output result of the feature restoration part. The output result of the first feature restoration module can be used as the input of the next feature restoration module. Since the structure of each feature restoration module in multiple feature restoration modules is the same, the next feature restoration module adopts the same processing method as the first feature restoration module, except that the input of the next feature restoration module is the output result of the previous feature restoration module. The same processing method is adopted, and it is iterated to the last feature restoration module in sequence to obtain the output result of the feature restoration part.

[0170] To facilitate understanding by those skilled in the art, a set of best embodiments is provided below:

[0171] Computed tomography (CT) is an important imaging method for obtaining internal structural information of objects in a non-destructive way. It has many advantages such as high resolution, high sensitivity and multi-level. It is one of the medical imaging diagnostic equipment with the largest installed capacity and is widely used in various medical clinical examination fields. Enhanced CT is a method of highlighting lesions or other parts in CT images by injecting contrast agents into the patient's body. It has significant significance in the imaging diagnosis of liver cancer. However, due to the presence of X-ray radiation in the CT scanning process, as people gradually understand the potential hazards of radiation, the issue of CT radiation dose has received more and more attention. At the same time, since CT contrast agents cannot be used for patients with poor basic conditions, renal failure, severe allergies, etc., enhanced CT has certain limitations in clinical application. However, when only plain scan CT is used for diagnosis, it is very easy to miss and misdiagnose liver lesions. Therefore, researching and developing new technologies for generating enhanced CT based on plain scan CT of liver cancer has important scientific significance and application prospects in the field of imaging diagnosis of liver cancer.

[0172] Most current methods solve the problem of difficulty in accurately capturing the features of the image in plain scan CT when generating enhanced CT images from plain scan CT. This embodiment uses an image generation model based on a diffusion model to generate enhanced CT images (medical images), and uses the powerful generation capability of the generation model to better highlight the features of enhanced CT. Plain scan CT is introduced into the network design as a priori information, and enhanced CT images with more detailed features are generated while maintaining the consistency of the anatomical structure.

[0173] In order to solve the technical problems that enhanced liver cancer CT scanning has many application limitations and high costs, and that the existing enhanced CT generation technology is difficult to accurately reflect the characteristics of lesions, this embodiment uses an image generation model based on a diffusion model, and fuses plain scan CT as a priori into the denoising process of the diffusion model, thereby improving the expressiveness of image generation while maintaining the consistency of anatomical structure. This embodiment ultimately obtains a generation result that can better reflect the characteristics of the lesions.

[0174] The technical solution adopted in this embodiment is: an image enhancement model using a consistency control module and an implicit diffusion model. The image enhancement model improves the image enhancement effect on liver cancer CT images. Figure 2 , the specific steps are as follows:

[0175] Step 1: Build a consistency control module.

[0176] By encoding the plain scan medical image step by step, the structural information is extracted and used to control the structure of the image generated by the implicit diffusion model. First, an image autoencoder (i.e., the second autoencoder) (the same as the autoencoder structure in step 3) is used to encode the plain scan medical image into a feature map (i.e., the output result of the second autoencoder) that implicitly represents the same size.

[0177] First, a temporal encoding module is constructed to encode the denoised time step t. The first step of temporal encoding is to encode t into a vector of length 320 through trigonometric functions, which is calculated as follows:

[0178]

[0179] The second step is to change the vector length to 1280 through a linear layer, and then pass it through the SiLU activation function and another linear layer to output the time code with a length of 1280. i represents the vector subscript of Position Embedding, and d represents the vector length.

[0180] By encoding the plain image step by step, the structural information is extracted from it, which is used to control the structure of the image generated by the diffusion model. First, the plain image is encoded into a feature map with the same size implicitly represented by an image autoencoder. In order to better extract features, a multi-scale encoder and an intermediate block are used next. The multi-scale encoder is divided into four groups of encoding modules, each of which contains three encoding submodules with the same structure. The result of temporal encoding is input into each encoding submodule in each encoding module through cross attention connection. At the end of encoding of each encoding module, the size of the feature map is reduced by downsampling. In each encoding submodule, the temporal encoding result is firstly subjected to SiLU activation function and linear layer operation, and then added to the features passed in from the upper layer after normalization, SiLU activation function, and 3*3 convolution operation, and then normalized, SiLU activation function, 3*3 convolution operation, and then added to the unprocessed features passed in from the upper layer to obtain the result. Through four sets of repeated encoding modules, the image size is gradually halved, while the feature depth is increased.

[0181] The middle module contains two residual modules. Each residual module first performs a 3*3 convolution on the features passed in from the upper layer, then adds them to the time encoding result, and then adds them to the features passed in from the upper layer without convolution to complete the residual connection. Finally, another 3*3 convolution and SiLU activation function are performed to obtain the result.

[0182] Step 2: Construct the implicit diffusion model backbone.

[0183] In the implicit diffusion model, each denoising process is implemented through a UNet structure. The UNet structure consists of three main parts: encoder, intermediate module and decoder.

[0184] The encoder part is divided into four groups of encoding modules, each of which contains three encoding submodules with the same structure. The temporal encoding result is input into each encoding submodule in each encoding module through cross attention connection. At the end of each encoding module, the size of the feature map is reduced by downsampling. The structure of each encoding submodule is consistent with that of the encoding submodule in step 1. Through four groups of repeated encoding modules, the image size is gradually halved while the feature depth is increased.

[0185] The middle module contains two residual modules. Each residual module first performs a 3*3 convolution on the features passed in from the upper layer, then adds them to the temporal encoding result, and then adds them to the features passed in from the upper layer without convolution to complete the residual connection. Finally, another 3*3 convolution and SiLU activation function are performed to obtain the result. This structure allows the network to exchange information between different layers, enhancing the model's ability to integrate global and local information.

[0186] The decoder part is also divided into four groups of decoding modules, each of which contains three decoding submodules with the same structure. The time encoding result is input into each decoding submodule in each decoding module through cross-attention connection. Each decoding module is fused with the flat-scan image features of the same size extracted from the consistency control module by point-to-point addition, and then fused with the feature map of the corresponding size in the encoding module by splicing, and then the next decoding operation is performed. At the end of each decoding module, the size of the feature map is doubled by upsampling. Each decoding submodule contains a residual module with the same structure as the residual module of the middle layer in step one. Through four sets of repeated decoding modules, the image size is gradually doubled, while the feature depth is reduced.

[0187] Finally, through the encoder, intermediate block and decoder, the implicit representation of the noisy image at T time steps is restored to the implicit representation of the noisy image at T-1 time steps. The total number of denoising steps in this method is 50, that is, after 50 denoising operations of this UNet, the final implicit representation of the enhanced image under the control of the consistency of the plain scan image is obtained, and then decoded into the target enhanced image (i.e., the target medical image) through the image self-decoder (see step 3).

[0188] Step 3: Build the image implicit codec.

[0189] In order to increase the learning efficiency of the overall model, the autoencoder (AE) is used to encode the plain scan medical image into an implicit representation (latent code) (i.e., pure noise feature map), and then the implicit representation is used as the input in the implicit diffusion model backbone. In the consistency control module, this encoder is also used to encode the plain scan image (i.e., plain scan medical image) into an implicit representation. After denoising is completed, the denoised implicit representation is restored to the target medical image through the autodecoder.

[0190] The autoencoder contains an input convolution layer, a feature extraction part, and an intermediate layer. The feature extraction part contains four modules, of which the four modules contain 1, 2, 4, and 4 residual modules respectively, and each module contains a downsampling layer. The residual module here first normalizes and convolves the input features twice, then adds the unprocessed input features to the processed features (i.e., the features obtained after two normalization and convolution operations), and downsamples the result of the addition (i.e., the output result of the residual module). In the intermediate layer, the features are first processed by the same residual module as the feature extraction part, followed by a self-attention operation, and finally processed by the residual module, normalized, and convolved once more to obtain an implicit representation of the image.

[0191] The structure of the auto-decoder is exactly the opposite of that of the auto-encoder, including an input convolutional layer, an intermediate layer, and a feature restoration part. The structure of the intermediate layer is exactly the same as that in the auto-encoder. The feature restoration part contains four modules, where the four modules contain 4, 4, 2, and 1 residual modules respectively, and each module contains an upsampling layer. The structure of the residual module is the same as that in the auto-encoder. Finally, the restored image is obtained through normalization and convolution operations.

[0192] Step 4: Construct the framework of the overall network model (i.e., the image enhancement model).

[0193] Refer to Figure 3 , which is the structural schematic diagram of the overall network model. The image enhancement model fully extracts the anatomical structure features of the fused plain scan CT images of liver cancer (i.e., plain scan medical images) according to the consistency control module designed in Step 1; the pure noise implicit representation is point-to-point added and fused with the feature to be decoded (i.e., the feature obtained from the encoding part) in the intermediate module and the decoding part in Step 2, and the final denoised implicit representation (i.e., the decoded feature map) is obtained after 50 UNet structure denoising steps; then, the denoised implicit representation is restored to the required enhanced CT image of liver cancer (i.e., the target medical image) through the auto-decoder in Step 3.

[0194] Step 5: Design the loss function of the image enhancement model.

[0195] Given the training dataset D = {(x1, y1), (x2, y2), …, (x n , y n )}, where x = {x1, x2,..., x n} are the image patches extracted from the plain scan CT images, and y = {y1, y2,..., y n} are the images extracted from the enhanced CT images, and n is the total number of training samples;

[0196] For each training pair, randomly select the time step t, 0 < t ≤ 50, and the loss function of this image enhancement model can be expressed in the form of mean square error:

[0197]

[0198] Among them, represents the expectation (here it means averaging over all possible inputs), z0 represents the initial latent representation, that is, the pure noise implicit representation (i.e., the pure noise feature map), t represents the time step, ∈ represents the result after adding t times of noise to the enhanced CT image, and ∈ θ represents the plain scan CT image with the model parameters parameterized as θ and the prediction denoised iterated 50 - t times, and c f represents the multi-scale features of the plain scan CT of liver cancer for consistency control.

[0199] Step 6: For the overall network framework designed above, use Adam optimizer to optimize the loss function.

[0200] Step 7: The image pairs and random time steps extracted from the dataset of liver cancer plain scan CT images and enhanced CT images are used as input of the image enhancement model.

[0201] Step 8: Train the denoising UNet network (i.e., implicit diffusion model) and consistency control module to obtain the image enhancement model. By setting the time step t from 50 to 1, the corresponding enhanced liver cancer CT image (i.e., target medical image) is gradually generated from pure noise under the control of the plain scan image.

[0202] Compared with the prior art, the technical solution of this embodiment has the following beneficial effects:

[0203] This embodiment enhances the lesion features of the generated liver cancer enhanced CT image by using a diffusion model-based image generation model with stronger expressiveness, and at the same time ensures that the generated liver cancer enhanced CT image is consistent with the plain scan CT image in terms of anatomical structure through a consistency control module.

[0204] Reference Figure 4 The embodiment of the present application further provides a medical image enhancement system, which includes a model building unit 100, an image encoding unit 200, a first processing unit 300, a second processing unit 400 and an image restoration unit 500, wherein:

[0205] A model construction unit 100, for constructing an image enhancement model including a first autoencoder, a first autodecoder, a consistency control module and an implicit diffusion model;

[0206] An image encoding unit 200, configured to encode the acquired plain scan medical image into a pure noise feature map through a first autoencoder;

[0207] The first processing unit 300 is used to input the plain scan medical image, the pure noise feature map and the time step into the consistency control module to obtain a first result, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module; encode the time step by the first time encoder to obtain a first time encoding result; encode the plain scan medical image by the second autoencoder to obtain an output result of the second autoencoder; add the output result of the second autoencoder to the pure noise feature map to obtain a first addition result, input the first addition result and the first time encoding result into the first multi-scale encoder to obtain multiple first encoding results; input the last first encoding result of the multiple first encoding results and the first time encoding result into the first intermediate module to obtain the first result;

[0208] The second processing unit 400 is used to input the first result, multiple first encoding results, the pure noise feature map and the time step into the implicit diffusion model to obtain a decoding feature map, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder; encode the time step by the second time encoder to obtain a second time encoding result; input the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain multiple second encoding results; add the last second encoding result of the multiple second encoding results to the first result to obtain a second addition result; input the second addition result into the second intermediate module to obtain the second result; input the second result, multiple second encoding results, the second time encoding result and the multiple first encoding results into the multi-scale decoder to obtain a decoding feature map;

[0209] The image restoration unit 500 is used to restore the decoded feature map into a target medical image through a first self-decoder, and the target medical image is a medical image enhanced from the plain scan medical image.

[0210] It should be noted that, since the medical image enhancement system in this embodiment and the medical image enhancement method described above are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to the system embodiment and will not be described in detail here.

[0211] An embodiment of the present application also provides an electronic device, comprising: at least one control processor and a memory for communicating with the at least one control processor.

[0212] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0213] The non-transient software program and instructions required to implement the medical image enhancement method of the above embodiment are stored in the memory. When executed by the processor, the medical image enhancement method of the above embodiment is executed, for example, the above described Figure 1 The method comprises steps S100 to S500.

[0214] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0215] The present application also provides a computer-readable storage medium storing computer-executable instructions, which are executed by one or more control processors to enable the one or more control processors to perform a medical image enhancement method in the above method embodiment, for example, to perform the above described Figure 1 The functions of method steps S100 to S500 in the method.

[0216] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0217] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.

Claims

1. A medical image enhancement method, characterized in that: The method comprises: Constructing an image enhancement model including a first autoencoder, a first autodecoder, a consistency control module, and an implicit diffusion model; encoding the acquired plain scan medical image into a pure noise feature map by the first autoencoder; Input the plain scan medical image, the pure noise feature map and the time step into the consistency control module to obtain a first result, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module; encode the time step by the first time encoder to obtain a first time encoding result; encode the plain scan medical image by the second autoencoder to obtain an output result of the second autoencoder; add the output result of the second autoencoder to the pure noise feature map to obtain a first addition result, input the first addition result and the first time encoding result into the first multi-scale encoder to obtain multiple first encoding results; input the last first encoding result of the multiple first encoding results and the first time encoding result into the first intermediate module to obtain a first result; Input the first result, the multiple first encoding results, the pure noise feature map and the time step into the implicit diffusion model to obtain a decoding feature map, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder; encode the time step by the second time encoder to obtain a second time encoding result; input the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain multiple second encoding results; add the last second encoding result of the multiple second encoding results to the first result to obtain a second addition result; input the second addition result into the second intermediate module to obtain a second result; input the second result, the multiple second encoding results, the second time encoding result and the multiple first encoding results into the multi-scale decoder to obtain a decoding feature map; The decoded feature map is restored to a target medical image through the first self-decoder, and the target medical image is a medical image enhanced from the plain scan medical image.

2. The medical image enhancement method according to claim 1, characterized in that: The first multi-scale encoder comprises a plurality of groups of cascaded first encoding modules, the plurality of groups of the first encoding modules have the same structure, each group of the first encoding modules comprises a plurality of cascaded first encoding sub-modules, the plurality of the first encoding sub-modules have the same structure; The step of inputting the first addition result and the first time encoding result into the first multi-scale encoder to obtain a plurality of first encoding results includes: The first temporal encoding result is input into a first starting encoding submodule in a first starting encoding module through a first cross attention connection for encoding, wherein the first starting encoding module is the encoding module arranged at the front of multiple first encoding modules, and the first starting encoding submodule is the encoding submodule arranged at the front of multiple first encoding submodules. The encoding process of the first starting encoding submodule includes: Perform SiLU activation function and linear layer operation on the first time encoding result to obtain a first operation result; Normalizing, performing SiLU activation function and 3x3 convolution operation on the first addition result to obtain a second operation result; Adding the first operation result and the second operation result to obtain a third operation result; Normalize, apply SiLU activation function and 3x3 convolution operation to the third operation result to obtain a fourth operation result; Adding the fourth operation result to the first addition result to obtain an output result of the first starting encoding submodule; Taking the first time encoding result and the output result of the first starting encoding submodule as the input of the next first encoding submodule, iterating sequentially to the last first encoding submodule in the first starting encoding module, and obtaining the first encoding result of the first starting encoding module; The first encoding result of the first starting encoding module is used as the input of the next group of first encoding modules, and iterates to the last group of first encoding modules in sequence to obtain multiple first encoding results, wherein a group of first encoding modules corresponds to one first encoding result.

3. The medical image enhancement method according to claim 1, characterized in that: The first intermediate module includes a plurality of cascaded residual modules, and the plurality of residual modules have the same structure; The step of inputting the last first encoding result of the plurality of first encoding results and the first time encoding result into the first intermediate module to obtain a first result comprises: Inputting the last first encoding result of the multiple first encoding results and the first time encoding result into a starting residual module, the starting residual module is the residual module that is the frontmost of the multiple residual modules, and the processing process of the last first encoding result of the multiple first encoding results and the first time encoding result by the starting residual module includes: Performing a 3x3 convolution operation on the last first encoding result among the multiple first encoding results to obtain a first convolution result; Adding the first convolution result and the first time encoding result to obtain a first added feature result; Adding the first added feature result to the last first encoding result among the multiple first encoding results to obtain a second added feature result; Perform 3*3 convolution and SiLU activation function operations on the second added feature result to obtain the output result of the starting residual module; The output result of the starting residual module is used as the input of the next residual module, and the process is iterated sequentially to the last residual module to obtain a first result.

4. The medical image enhancement method according to claim 1, characterized in that: The second multi-scale encoder comprises a plurality of groups of cascaded second encoding modules, the plurality of groups of the second encoding modules have the same structure, each group of the second encoding modules comprises a plurality of cascaded second encoding sub-modules, the plurality of the second encoding sub-modules have the same structure; The step of inputting the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain a plurality of second encoding results includes: The second temporal encoding result is input into the second starting encoding submodule in the second starting encoding module through the second cross attention connection for encoding, wherein the second starting encoding module is the encoding module arranged at the front of the plurality of second encoding modules, and the second starting encoding submodule is the encoding submodule arranged at the front of the plurality of second encoding submodules, and the encoding process of the second starting encoding submodule includes: Perform SiLU activation function and linear layer operation on the second time encoding result to obtain a linear operation result; Normalizing, performing SiLU activation function and 3x3 convolution operation on the pure noise feature map to obtain a first convolution operation result; Adding the linear operation result and the first convolution operation result to obtain a feature addition result; Normalizing, performing SiLU activation function and 3x3 convolution operation on the feature addition result to obtain a second convolution operation result; Adding the second convolution operation result to the pure noise feature map to obtain an output result of the second starting coding submodule; Using the second time encoding result and the output result of the second starting encoding submodule as the input of the next second encoding submodule, iterating sequentially to the last second encoding submodule in the second starting encoding module, and obtaining the second encoding result of the second starting encoding module; The second encoding result of the second starting encoding module is used as the input of the next group of second encoding modules, and iterates to the last group of second encoding modules in sequence to obtain multiple second encoding results, wherein one group of second encoding modules corresponds to one second encoding result.

5. The medical image enhancement method according to claim 1, characterized in that: The multi-scale decoder comprises a plurality of groups of cascaded decoding modules, the structures of the plurality of groups of decoding modules are the same, each group of the decoding modules comprises a plurality of cascaded decoding sub-modules, the structures of the plurality of decoding sub-modules are the same, and the plurality of groups of decoding modules have the same scale as the plurality of groups of first encoding modules and the plurality of groups of second encoding modules; The step of inputting the second result, the plurality of second encoding results, the second time encoding result, and the plurality of first encoding results into the multi-scale decoder to obtain a decoding feature graph comprises: The second result, the second encoding result corresponding to the second encoding module of the same scale as the starting decoding module, the second time encoding result, and the first encoding result corresponding to the first encoding module of the same scale as the starting decoding module are input into the starting decoding module for decoding to obtain a first decoding result. The starting decoding module is the frontmost decoding module in the plurality of groups of decoding modules. The decoding process of the starting decoding module includes: Adding the second result to a first encoding result corresponding to a first encoding module of the same scale as the starting decoding module to obtain a first addition output result; Input the first addition output result, the second time encoding result and the second encoding module of the same scale as the starting decoding module into the first decoding submodule in the starting decoding module to obtain the output result of the first decoding submodule in the starting decoding module; Inputting the output result of the first decoding submodule in the starting decoding module and the second time encoding result into the second decoding submodule in the starting decoding module to obtain the output result of the second decoding submodule in the starting decoding module; Inputting the output result of the second decoding submodule in the starting decoding module and the second time encoding result into the third decoding submodule in the starting decoding module to obtain the first decoding result corresponding to the starting decoding module; The first decoding result, the second encoding result corresponding to the second encoding module of the same scale as the next group of decoding modules, the second time encoding result and the first encoding result corresponding to the first encoding module of the same scale as the next group of decoding modules are used as inputs of the next group of decoding modules, and the process is iterated sequentially to the last group of decoding modules to obtain a decoding feature map.

6. The medical image enhancement method according to claim 1, characterized in that: The first autoencoder includes a first input convolutional layer, a feature extraction part and a first intermediate layer, the feature extraction part includes a plurality of feature extraction modules, each of the feature extraction modules includes a residual module and a downsampling layer, and the first intermediate layer includes a residual module with the same structure as the feature extraction part; The step of encoding the acquired plain scan medical image into a pure noise feature map by the first autoencoder includes: Inputting the plain scan medical image into the first input convolution layer to obtain convolved features; The convolved features are input into a first feature extraction module in the feature extraction part for processing, and the processing process of the first feature extraction module includes: For the residual module in the first feature extraction module, the convolved feature is normalized and convolved twice to obtain a first feature; the first feature is added to the convolved feature to obtain an output result of the residual module; Downsampling the output result of the residual module to obtain the output result of the first feature extraction module; Using the output result of the first feature extraction module as the input of the next feature extraction module, iterating to the last feature extraction module in sequence, and obtaining the output result of the feature extraction part; Inputting the output result of the feature extraction part into the residual module in the first intermediate layer to obtain a residual result; Perform a self-attention operation on the residual result to obtain a self-attention feature; The self-attention features are subjected to residual module processing, normalization, and convolution operations in the first intermediate layer to obtain a pure noise feature map.

7. The medical image enhancement method according to claim 1, characterized in that: The first autodecoder includes a second input convolutional layer, a feature restoration part, and a second intermediate layer, the feature restoration part includes a plurality of feature restoration modules, each of the feature restoration modules includes a residual module and an upsampling layer, and the second intermediate layer includes a residual module with the same structure as the feature restoration part; The step of restoring the decoded feature map to a target medical image by the first self-decoder includes: Inputting the decoded feature map into the second input convolutional layer to obtain output features of the second input convolutional layer; Inputting the output features of the second input convolutional layer into the residual module in the second intermediate layer to obtain an output result of the residual module; Perform a self-attention operation on the output result of the residual module to obtain a self-attention output feature; The self-attention output feature is processed by the residual module in the second intermediate layer, normalized and convolved to obtain an initial decoding feature; The initial decoded features are input into a first feature restoration module in the feature restoration part for processing, wherein the processing process of the first feature restoration module includes: For the residual module in the first feature restoration module, the initial decoding feature is normalized and convolved twice to obtain a first decoding feature; the first decoding feature is added to the initial decoding feature to obtain an output result of the residual module; Upsampling the output result of the residual module to obtain the output result of the first feature restoration module; Using the output result of the first feature restoration module as the input of the next feature restoration module, iterating to the last feature restoration module in turn, and obtaining the output result of the feature restoration part; The output result of the feature restoration part is normalized and convolved to obtain a target medical image.

8. A medical image enhancement system, characterized in that: The system comprises: A model building unit, used to build an image enhancement model including a first autoencoder, a first autodecoder, a consistency control module and an implicit diffusion model; An image encoding unit, configured to encode the acquired plain scan medical image into a pure noise feature map through the first autoencoder; A first processing unit is used to input the plain scan medical image, the pure noise feature map and the time step into the consistency control module to obtain a first result, wherein the consistency control module includes a first time encoder, a second autoencoder, a first multi-scale encoder and a first intermediate module; the time step is encoded by the first time encoder to obtain a first time encoding result; the plain scan medical image is encoded by the second autoencoder to obtain an output result of the second autoencoder; the output result of the second autoencoder is added to the pure noise feature map to obtain a first addition result, and the first addition result and the first time encoding result are input into the first multi-scale encoder to obtain multiple first encoding results; the last first encoding result of the multiple first encoding results and the first time encoding result are input into the first intermediate module to obtain a first result; a second processing unit, configured to input the first result, the multiple first encoding results, the pure noise feature map and the time step into the implicit diffusion model to obtain a decoding feature map, wherein the implicit diffusion model includes a second time encoder, a second intermediate module, a second multi-scale encoder and a multi-scale decoder corresponding to the second multi-scale encoder; encode the time step by the second time encoder to obtain a second time encoding result; input the second time encoding result and the pure noise feature map into the second multi-scale encoder to obtain multiple second encoding results; add the last second encoding result of the multiple second encoding results to the first result to obtain a second addition result; input the second addition result into the second intermediate module to obtain a second result; input the second result, the multiple second encoding results, the second time encoding result and the multiple first encoding results into the multi-scale decoder to obtain a decoding feature map; An image restoration unit is used to restore the decoded feature map into a target medical image through the first self-decoder, and the target medical image is a medical image enhanced from the plain scan medical image.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the medical image enhancement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the medical image enhancement method according to any one of claims 1 to 7.

Images