Methods and equipment for optimizing prostate MRI images for clinical surgery

Abstract

This invention discloses a method and device for optimizing prostate MRI images for clinical surgery, belonging to the field of prostate MRI image processing technology. The optimization method includes: retrieving an initial MRI image; retrieving a trained data fitting network; and adjusting the initial MRI image using the data fitting network to generate an optimized, improved MRI image. The data fitting network includes an information mining module and an enhancement residual module. The information mining module is used to mine image features from the initial MRI image, and the enhancement residual module processes the feature map preceding the information mining module to generate a residual feature map. Then, the residual feature map is used to register the feature map following the information mining module. This invention uses the enhancement residual module to perform dual-domain constrained enhancement adjustment on the preceding feature data to generate a residual feature map, which can greatly enhance the network's targeted optimization effect on the target region and avoid the distortion of image detail information.

Description

Technical Field

[0001] This invention belongs to the field of prostate MRI image processing technology, specifically relating to a method and device for optimizing prostate MRI images for clinical surgery. Background Technology

[0002] Magnetic resonance imaging (MRI) is widely used to support the diagnosis and treatment of diseases due to its excellent soft tissue resolution. However, the quality of MRI images is affected by factors such as long signal acquisition time and patient movement, often resulting in artifacts and loss of detail. In clinical surgery, accurately locating the position and region of prostate lesions is crucial.

[0003] In recent years, deep learning technology has made significant progress in the field of medical image processing, providing new ideas for optimizing MRI image quality. While existing deep learning-based MRI image reconstruction methods can effectively remove artifacts, they generally suffer from poor visual performance and severe distortion of image detail information after optimization. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for optimizing prostate MRI images for clinical surgery, in order to solve the above-mentioned technical problems.

[0005] The technical solution is as follows: a method for optimizing prostate MRI images for clinical surgery, comprising the following steps:

[0006] An initial MRI image is retrieved, a trained data fitting network is retrieved, and the initial MRI image is adjusted using the data fitting network to generate an optimized and improved MRI image.

[0007] The data fitting network includes an information mining module and an enhancement residual module. The information mining module is used to mine the image features of the initial MRI image, and the enhancement residual module is used to process the feature map in front of the information mining module to generate a residual feature map, and then use the residual feature map to register the feature map behind the information mining module.

[0008] Furthermore, the residual feature map and the feature map behind the information mining module are registered by adding corresponding elements.

[0009] Furthermore, the data fitting network includes a pre-operator, multiple information mining modules, and multiple enhancement residual modules. The pre-operator is positioned before the information mining modules and the enhancement residual modules. After performing a first operation on the initial MRI image, the pre-operator generates a pre-feature map. The multiple information mining modules are arranged sequentially, and the enhancement residual modules are correspondingly set with the information mining modules. The first information mining module is used to perform a second operation on the pre-feature map, and subsequent information mining modules are used to perform a second operation on the previously registered feature map. The first enhancement residual module is used to perform a third operation on the pre-feature map, and subsequent enhancement residual modules are used to perform a third operation on the previously registered feature map and the previous residual feature map.

[0010] Furthermore, the internal operation process of the pre-operator includes:

[0011] The initial MRI image is sequentially subjected to a first convolution calculation and a first activation calculation to generate the pre-feature map.

[0012] Furthermore, the internal computation process of the information mining module includes:

[0013] Multiple convolutional components with different receptive field sizes are used to calculate the feature maps of the input information mining module, generating multiple first convolutional feature maps;

[0014] Multiple first convolutional feature maps are fused to generate a second convolutional feature map;

[0015] Each first convolutional feature map is subtracted from the second convolutional feature map to generate multiple third convolutional feature maps.

[0016] Each of the third convolutional feature maps is activated to generate multiple fourth convolutional feature maps;

[0017] Multiple fourth convolutional feature maps are multiplied by Hadamard to produce a fifth convolutional feature map.

[0018] The second convolutional feature map is fused with the fifth convolutional feature map to generate the sixth convolutional feature map for the output information mining module.

[0019] Furthermore, the internal calculation process of the enhanced residual module includes:

[0020] Global average pooling is performed on the first and second feature maps of the input enhancement residual module in the channel direction to generate the first pooling feature map and the second pooling feature map, respectively.

[0021] Add the corresponding elements of the first pooling feature map and the second pooling feature map to generate the third pooling feature map;

[0022] The activated third pooling feature map is fused with one of the feature maps to generate the residual feature map;

[0023] Specifically, for the first enhanced residual module, both the first feature map and the second feature map are the previous feature maps; for subsequent enhanced residual modules, the first feature map is the feature map after the previous registration, and the second feature map is the previous residual feature map.

[0024] Furthermore, the data fitting network is provided with an image generation module at the end. The image generation module performs a fourth operation on the last registered feature map and all the third pooling feature maps to generate the improved MRI image.

[0025] Furthermore, the internal computation process of the image generation module includes:

[0026] All the third pooling feature maps are concatenated to generate the first improved feature map;

[0027] The first improved feature map is sequentially subjected to a second convolution calculation and a second activation calculation to generate a second improved feature map;

[0028] The second improved feature map is fused with the last registered feature map to generate the third improved feature map;

[0029] The improved MRI image is generated by sequentially performing a third convolution calculation and a third activation calculation on the third improved feature map.

[0030] The image generation module effectively integrates residual features and deep features after multi-level constraints in the network, reconstructs details of key regions, and improves the visual performance of the image in the human eye.

[0031] The present invention also provides a prostate MRI image optimization device for clinical surgery, comprising a computer program / instructions that, when executed, implement the method described above.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] In existing convolutional neural networks used for image processing, residual connections are typically used to prevent gradient vanishing and accelerate model training convergence, directly fusing the feature maps before and after convolution. However, for prostate MRI image optimization tasks, simple residual connections can lead to the suppression and blurring of features in the image region of interest, making it difficult to achieve the desired clinical application effect. By using the enhanced residual module provided in this invention, gradient vanishing is avoided, and residual feature maps are generated through dual-domain constraint enhancement adjustment of the preceding feature data. This can greatly enhance the network's targeted optimization effect on the target region and avoid the distortion of image detail information. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the process for generating the last registered feature map using the data fitting network of this invention.

[0035] Attached Figure

[0036] 1-Initial MRI image, 2-Pre-operator, 3-Information mining module, 4-Enhancement residual module, 5-Feature map after last registration. Detailed Implementation

[0037] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0038] This embodiment provides a method for optimizing prostate MRI images for clinical surgery, such as... Figure 1 As shown, it includes the following steps:

[0039] The initial MRI image 1 is retrieved, and the trained data fitting network is retrieved. The data fitting network is used to adjust the initial MRI image 1 to generate an optimized and improved MRI image.

[0040] The data fitting network in this embodiment includes a pre-operator 2, an information mining module 3, an enhancement residual module 4, and an image generation module. The pre-operator 2 is positioned before the information mining module 3 and the enhancement residual module 4. After performing a first operation on the initial MRI image 1, the pre-operator 2 generates a pre-feature map. The pre-operator is used to obtain shallow feature information of the initial MRI image 1. The number of channels in the pre-feature map can be set to 24, 48, or 64, etc. Specifically, the first operation inside the pre-operator 2 includes: performing a first convolution calculation and a first activation calculation on the initial MRI image 1 to generate the pre-feature map. During the first convolution calculation, the stride is usually 1, and the first activation calculation can be performed using commonly used activation functions such as ReLU.

[0041] Information mining module 3 is used to mine deep image features of the initial MRI image 1. Enhancement residual module 4 is used to process the feature map in front of (corresponding) information mining module 3 to generate residual feature map. Then, the residual feature map is added to the corresponding elements of the feature map behind information mining module 3 (the size of the residual feature map and the feature map behind information mining module 3 are equal) to realize the registration of the feature map behind information mining module 3.

[0042] As an exemplary implementation, the data fitting network has four information mining modules 3 arranged from front to back, and the reinforcement residual module 4 is set up one-to-one with the information mining modules 3. The first information mining module 3 is used to perform a second operation on the previous feature map, and the subsequent information mining modules 3 are used to perform a second operation on the previous registered feature map (that is, the feature map generated by adding the corresponding elements of the feature map output by the previous information mining module and the residual feature map output by the corresponding reinforcement residual module).

[0043] As is known to those skilled in the art, numerous readily available feature extraction modules exist for mining deep image features from MRI images. In some possible implementations, existing algorithm modules can be directly migrated and applied to the information mining module of this invention. To better match the prostate MRI image optimization task, the internal computation process of the information mining module 3 of this invention is preferably the following:

[0044] Multiple convolutional components with different receptive field sizes are used to calculate the feature map of the input information mining module 3 (the feature map input to the first information mining module is the pre-registered feature map; for subsequent information mining modules, the input feature map is the previously registered feature map), generating multiple first convolutional feature maps of equal size. For example, convolutional components 1, 2, and 3 can be used to calculate the feature map of the input information mining module, generating three first convolutional feature maps. Each of the three convolutional components includes sequentially set convolutional layers and activation layers. The kernel size of the convolutional layer of convolutional component 1 is 1*1, the kernel size of the convolutional layer of convolutional component 2 is 3*3, and the kernel size of the convolutional layer of convolutional component 3 is 5*5. The activation layers of the three convolutional components can each independently select activation functions such as ReLU and tanh.

[0045] All first convolutional feature maps (within the same information mining module) are fused to generate a second convolutional feature map. The fusion of multiple first convolutional feature maps can be achieved using existing methods. In one possible implementation, the process of fusing multiple first convolutional feature maps includes: firstly, concatenating all the first convolutional feature maps to generate a combined feature map; then, sequentially performing convolution and activation (e.g., PReLU activation) calculations on the combined feature map to generate a second convolutional feature map. The size of the second convolutional feature map can be equal to the size of the first convolutional feature maps.

[0046] Each first convolutional feature map is subtracted from the second convolutional feature map to generate multiple third convolutional feature maps. Specifically, each first convolutional feature map is subtracted from the second convolutional feature map to generate multiple corresponding third convolutional feature maps.

[0047] Each third convolutional feature map is activated separately to generate multiple fourth convolutional feature maps of equal size; specifically, existing activation functions such as ReLU, tanh, or sigmoid can be independently selected to activate each third convolutional feature map.

[0048] Multiple fourth convolutional feature maps are multiplied by Hadamard to produce a fifth convolutional feature map.

[0049] The second and fifth convolutional feature maps are fused to generate the sixth convolutional feature map for the output information mining module. Existing methods can be directly used to fuse the second and fifth convolutional feature maps. In one possible implementation, the second and fifth convolutional feature maps can be fused sequentially through concatenation, convolution, and activation (e.g., PReLU activation).

[0050] After the above-mentioned calculation process is adopted, the data fitting network of the information mining module 3 has a good ability to distinguish the numerous non-uniform blurred image information in the prostate MRI image, and can better perceive the deep spatial features, thereby achieving a better feature information mining effect.

[0051] The first enhanced residual module 4 is used to perform a third operation on the preceding feature map, and subsequent enhanced residual modules 4 are used to perform a third operation on the previously registered feature map and the previous (output of the enhanced residual module 4) residual feature map. In one possible implementation, the internal operation process of the enhanced residual module 4 includes:

[0052] Global average pooling is performed on the one-way and two-way feature maps of the input enhancement residual module 4 in the channel direction to generate a first pooling feature map and a second pooling feature map with 1 channel respectively.

[0053] Add the corresponding elements of the first pooling feature map and the second pooling feature map to generate the third pooling feature map;

[0054] The third pooling feature map, after activation (e.g., sigmoid function activation), is multiplied with the corresponding element of one feature map to produce a residual feature map.

[0055] Specifically, for the first enhanced residual module 4, both the first and second feature maps are pre-registered feature maps; for subsequent enhanced residual modules 4, the first feature map is the feature map after the previous registration, and the second feature map is the previous residual feature map.

[0056] The image generation module performs a fourth operation on the last registered feature map 5 and the third pooling feature maps generated in all preceding enhancement residual modules 4 to generate an improved MRI image. In one possible implementation, the fourth operation within the image generation module includes:

[0057] Extract all the third pooling feature maps generated in the previous enhanced residual module 4, and stitch these third pooling feature maps together to generate the first improved feature map.

[0058] The first improved feature map is sequentially subjected to the second convolution and the second activation calculations to generate a second improved feature map with 1 channel. During the second convolution calculation, the kernel size can be 3*3 and the stride is usually 1. The second activation calculation can be completed using commonly used activation functions such as ReLU.

[0059] The second improved feature map is multiplied by the corresponding elements of the last registered feature map 5 to generate the third improved feature map. In this embodiment, the last registered feature map 5 is the feature map generated by adding the corresponding elements of the sixth convolutional feature map output by the fourth information mining module and the residual feature map output by the fourth enhancement residual module.

[0060] The third improved feature map is sequentially subjected to third convolution and third activation calculations to produce an improved MRI image with 3 channels. During the third convolution calculation, the kernel size can be 3*3, and the stride is typically 1. The third activation calculation can be performed using common activation functions such as ReLU.

[0061] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. Those skilled in the art, under the guidance of the present invention, can make various similar representations without departing from the spirit and claims of the present invention, and such modifications all fall within the protection scope of the present invention.

Claims

1. A method for prostate MRI image optimization for clinical procedures, characterized in that, The method comprises the following steps: retrieve an initial MRI image, retrieve a trained data fitting network, adjust the initial MRI image by using the data fitting network, and generate an improved MRI image after optimization; The data fitting network is provided with an information mining module and a strengthened residual module. The information mining module is used for mining image features of the initial MRI image, and the strengthened residual module is used for processing feature maps in front of the information mining module to generate residual feature maps, and then using the residual feature maps to register feature maps behind the information mining module. The internal operation process of the information mining module comprises: a plurality of convolution components with different receptive field sizes are used to calculate the feature maps input into the information mining module to generate a plurality of first convolution feature maps; the first convolution feature maps are fused to generate a second convolution feature map; each first convolution feature map is subtracted from the second convolution feature map to generate a plurality of third convolution feature maps; each third convolution feature map is activated to generate a plurality of fourth convolution feature maps; the fourth convolution feature maps are subjected to Hadamard multiplication to generate a fifth convolution feature map; the second convolution feature map and the fifth convolution feature map are fused to generate a sixth convolution feature map output from the information mining module; The tail of the data fitting network is provided with an image generation module, which generates the improved MRI image based on the last registered feature map.

2. The method of claim 1, wherein, The residual feature map and the feature map behind the information mining module are registered by adding corresponding elements.

3. The method of claim 1, wherein, The data fitting network is provided with a pre-operator, a plurality of information mining modules and a plurality of strengthened residual modules. The pre-operator is arranged in front of the information mining module and the strengthened residual module. After the pre-operator performs a first operation on the initial MRI image, a pre-feature map is generated. The information mining modules are arranged in sequence, and the strengthened residual modules are arranged correspondingly. The first information mining module is used for a second operation on the pre-feature map, the subsequent information mining modules are used for a second operation on the last registered feature map, the first strengthened residual module is used for a third operation on the pre-feature map, and the subsequent strengthened residual modules are used for a third operation on the last registered feature map and the last residual feature map.

4. The prostate MRI image optimization method for clinical surgery according to claim 3, characterized in that, The internal operation process of the pre-operator comprises: the initial MRI image is sequentially subjected to first convolution calculation and first activation calculation to generate the pre-feature map.

5. The prostate MRI image optimization method for clinical surgery according to claim 3, wherein, The internal operation process of the strengthened residual module comprises: the one-way feature map and the two-way feature map input into the strengthened residual module are subjected to global average pooling processing in the channel direction to generate a first pooling feature map and a second pooling feature map, respectively; the first pooling feature map and the second pooling feature map are added to generate a third pooling feature map; the activated third pooling feature map and the one-way feature map are fused to generate the residual feature map; Wherein, for the first reinforced residual module, the one-way feature map and the two-way feature map are the front feature map; for the subsequent reinforced residual module, the one-way feature map is the last registered feature map, and the two-way feature map is the last residual feature map.

6. The prostate MRI image optimization method for clinical surgery of claim 5, wherein, The image generation module generates the improved MRI image after performing fourth operation on the last registered feature map and all the third pooled feature maps.

7. The clinical procedure oriented prostate MRI image optimization method of claim 6, wherein, The internal operation process of the image generation module comprises: Splicing all the third pooled feature maps to generate a first improved feature map; Performing second convolution calculation and second activation calculation on the first improved feature map in sequence to generate a second improved feature map; Fusing the second improved feature map with the last registered feature map to generate a third improved feature map; Performing third convolution calculation and third activation calculation on the third improved feature map in sequence to generate the improved MRI image.

8. A prostate MRI image optimization device for clinical surgery, characterized by, Computer program / instructions are included, when the computer program / instructions are executed, the method as claimed in any one of claims 1-7 is implemented.

Images