Prostate MRI image optimization method and device for clinical operation

Abstract

The invention discloses a prostate MRI (Magnetic Resonance Imaging) image optimization method and equipment for a clinical operation, and belongs to the technical field of prostate MRI image processing. The optimization method comprises the steps that an initial MRI image is called, a trained data fitting network is called, the initial MRI image is adjusted through the data fitting network, and an optimized improved MRI image is generated. An information mining module and an enhanced residual module are arranged in the data fitting network, the information mining module is used for mining image features of the initial MRI image, the enhanced residual module is used for processing a feature map in front of the information mining module to generate a residual feature map, and then the residual feature map is used for registering a feature map behind the information mining module. According to the method, the enhanced residual module is adopted to perform double-domain constraint type enhanced adjustment on the front feature data to generate the residual feature map, so that the targeted optimization effect of the network on the target area can be greatly enhanced, and disorder of image detail information is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of prostate MRI image processing, and in particular relates to a prostate MRI image optimization method and device for clinical surgery. Background Art

[0002] Magnetic resonance imaging (MRI) is widely used to support the diagnosis and treatment of diseases due to its good soft tissue resolution. The quality of MRI images is affected by factors such as long signal acquisition time and patient movement, and often has problems such as artifacts and missing details. In clinical surgery, it is crucial to accurately locate the location and area of ​​prostate lesions.

[0003] In recent years, deep learning technology has made significant progress in the field of medical image processing, providing new ideas for optimizing MRI image quality. Although existing deep learning-based MRI image reconstruction methods can effectively remove artifacts, they generally have problems such as poor visual performance of optimized images and serious confusion of image detail information. Summary of the invention

[0004] In view of this, the present invention provides a prostate MRI image optimization method and device for clinical surgery to solve the above technical problems.

[0005] The technical scheme is as follows: A prostate MRI image optimization method for clinical surgery includes the following steps: Retrieving an initial MRI image, retrieving a trained data fitting network, and using the data fitting network to adjust the initial MRI image to generate an optimized improved MRI image; The data fitting network is provided with an information mining module and an enhanced residual module. The information mining module is used to mine the image features of the initial MRI image, and the enhanced residual module is used to process the feature map in front of the information mining module to generate a residual feature map, and then use the residual feature map to align the feature map behind the information mining module.

[0006] Furthermore, the residual feature map and the feature map after the information mining module are aligned by adding corresponding elements.

[0007] Furthermore, the data fitting network is provided with a pre-operator, a plurality of information mining modules and a plurality of enhanced residual modules, the pre-operator is arranged in front of the information mining module and the enhanced residual module, and the pre-operator generates a pre-feature map after performing a first operation on the initial MRI image; the plurality of information mining modules are arranged in sequence front to back, and the enhanced residual module is arranged corresponding to the information mining module; the first information mining module is used to perform a second operation on the pre-feature map, and the subsequent information mining module is used to perform a second operation on the last registered feature map, the first enhanced residual module is used to perform a third operation on the pre-feature map, and the subsequent enhanced residual module is used to perform a third operation on the last registered feature map and the last residual feature map.

[0008] Furthermore, the internal operation process of the pre-operator includes: The first convolution calculation and the first activation calculation are sequentially performed on the initial MRI image to generate the pre-feature map.

[0009] Furthermore, the internal operation process of the information mining module includes: The feature map of the input information mining module is calculated using a plurality of convolutional components with different receptive field sizes to generate a plurality of first convolutional feature maps; Fusing a plurality of the first convolutional feature maps to generate a second convolutional feature map; Subtracting each first convolution feature map from the second convolution feature map to generate a plurality of third convolution feature maps; Activating each of the third convolutional feature maps respectively to generate a plurality of fourth convolutional feature maps; Perform Hadamard product on multiple fourth convolution feature maps to generate a fifth convolution feature map; The second convolutional feature map is fused with the fifth convolutional feature map to generate a sixth convolutional feature map of the output information mining module.

[0010] Furthermore, the internal operation process of the enhanced residual module includes: Perform global average pooling processing on the first-channel feature map and the second-channel feature map of the input enhanced residual module in the channel direction to generate the first pooling feature map and the second pooling feature map respectively; Adding corresponding elements of the first pooling feature map and the second pooling feature map to generate a third pooling feature map; The activated third pooling feature map is fused with the one-way feature map to generate the residual feature map; Among them, for the first enhanced residual module, the one-way feature map and the two-way feature map are both the previous feature map; for the subsequent enhanced residual module, the one-way feature map is the previous registered feature map, and the two-way feature map is the previous residual feature map.

[0011] Furthermore, an image generation module is provided at the tail of the data fitting network, and the image generation module generates the improved MRI image after performing a fourth operation on the last registered feature map and all the third pooled feature maps.

[0012] Furthermore, the internal operation process of the image generation module includes: All the third pooled feature maps are concatenated to generate the first improved feature map; performing a second convolution calculation and a second activation calculation on the first improved feature map in sequence to generate a second improved feature map; Fusion the second improved feature map with the last registered feature map to generate a third improved feature map; The third improved feature map is sequentially subjected to a third convolution calculation and a third activation calculation to generate the improved MRI image.

[0013] The image generation module effectively integrates the residual features after multi-level constraints in the network with the deep features, reconstructs the details of the key areas, and improves the visual performance of the image in the human eye.

[0014] The present invention also provides a prostate MRI image optimization device for clinical surgery, including a computer program / instruction. When the computer program / instruction is executed, the method described above is implemented.

[0015] Compared with the prior art, the present invention has the following beneficial effects: In the existing convolutional neural networks for image processing, in order to prevent gradient vanishing and accelerate model training convergence, residual connections are usually set to directly fuse the feature maps before and after convolution. However, for the prostate MRI image optimization task, simple residual connections will cause the features of the image region of interest to be suppressed and blurred, making it difficult to achieve the effect of clinical application. After adopting the enhanced residual module provided by the present invention, while avoiding gradient vanishing, the residual feature map is generated by performing dual-domain constrained enhanced adjustment on the front feature data, which can greatly enhance the network's targeted optimization effect on the target area and avoid confusion of image detail information. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a process for generating a final registered feature map for the data fitting network of the present invention. Attached photos

[0017] 1-initial MRI image, 2-pre-operator, 3-information mining module, 4-enhanced residual module, 5-the last feature map after registration. DETAILED DESCRIPTION

[0018] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] This embodiment provides a prostate MRI image optimization method for clinical surgery. Figure 1 As shown, the following steps are included: The initial MRI image 1 is retrieved, the trained data fitting network is retrieved, and the initial MRI image 1 is adjusted using the data fitting network to generate an optimized improved MRI image.

[0020] The data fitting network of this embodiment is provided with a pre-operator 2, an information mining module 3, an enhanced residual module 4 and an image generation module. The pre-operator 2 is arranged in front of the information mining module 3 and the enhanced residual module 4. After the pre-operator 2 performs a first operation on the initial MRI image 1, a pre-feature map is generated. The pre-operator is used to obtain the shallow feature information of the initial MRI image 1. The number of channels of the pre-feature map can be set to 24, 48, 64, etc. Specifically, the first operation inside the pre-operator 2 includes: performing a first convolution calculation and a first activation calculation on the initial MRI image 1 to generate the pre-feature map. During the first convolution calculation, the step size is usually 1, and the first activation calculation can be completed using commonly used activation functions such as ReLU.

[0021] The information mining module 3 is used to mine the deep image features of the initial MRI image 1, and the enhanced residual module 4 is used to process the feature map in front of the (corresponding) information mining module 3 to generate a residual feature map, and then the residual feature map is added to the corresponding elements of the feature map behind the information mining module 3 (the size of the residual feature map is equal to that of the feature map behind the information mining module), so as to realize the alignment of the feature map behind the information mining module 3.

[0022] As an exemplary implementation, four information mining modules 3 are provided in the data fitting network from the front to the back, and the enhanced residual modules 4 are arranged one-to-one corresponding to the information mining modules 3; the first information mining module 3 is used to perform a second operation on the previous feature map, and the subsequent information mining modules 3 are used to perform a second operation on the last registered feature map (that is, a feature map generated by adding the corresponding elements of the residual feature map output by the previous information mining module and the corresponding enhanced residual module).

[0023] It is well known to those skilled in the art that there are currently a large number of ready-made feature extraction modules that can be used to mine deep image features of MRI images. In some possible implementations, the existing algorithm modules can be directly migrated and applied to the information mining module of the present invention. In order to better match the prostate MRI image optimization task, the internal operation process of the information mining module 3 of the present invention is preferably the following process: The feature map of the input information mining module 3 is calculated using multiple convolution components with different receptive field sizes (the feature map of the first information mining module is the pre-feature map, and for the subsequent information mining modules, the input feature map is the feature map after the previous registration) to generate multiple first convolution feature maps of equal size. For example: the feature map of the input information mining module can be calculated using convolution components No. 1, No. 2, and No. 3 respectively to generate three first convolution feature maps. The three convolution components include convolution layers and activation layers arranged in sequence. The convolution kernel size of the convolution layer of convolution component No. 1 is 1*1, the convolution kernel size of the convolution layer of convolution component No. 2 is 3*3, and the convolution kernel size of the convolution layer of convolution component No. 3 is 5*5. The activation layers of the three convolution components can be completed by independently selecting activation functions such as ReLU and tanh.

[0024] All first convolution feature maps (within the same information mining module) are fused to generate a second convolution feature map. The method of fusing multiple first convolution feature maps can directly adopt the existing method. In one possible implementation, the process of fusing multiple first convolution feature maps includes: first splicing all first convolution feature maps to generate a combined feature map, and then performing convolution and activation (such as PReLU function activation) calculations on the combined feature map in turn to generate a second convolution feature map. The size of the second convolution feature map can be equal to the size of the first convolution feature map.

[0025] Subtract each first convolution feature map from the second convolution feature map to generate a plurality of third convolution feature maps. Specifically, each first convolution feature map is subtracted from the second convolution feature map to generate a plurality of corresponding third convolution feature maps.

[0026] Each third convolutional feature map is activated respectively to generate multiple fourth convolutional feature maps of equal size; specifically, existing activation functions such as ReLU, tanh or sigmoid can be independently selected to activate each third convolutional feature map.

[0027] Perform Hadamard product on multiple fourth convolution feature maps to generate a fifth convolution feature map; The second convolution feature map is fused with the fifth convolution feature map to generate the sixth convolution feature map of the output information mining module. The method of fusing the second convolution feature map and the fifth convolution feature map can directly adopt the existing method. In one possible implementation, the second convolution feature map and the fifth convolution feature map can be fused by splicing, convolution and activation (such as PReLU function activation) in sequence.

[0028] After the information mining module 3 adopts the above-mentioned operation process, the data fitting network has a good identification ability for the numerous non-uniform fuzzy image information in the prostate MRI image, and can better perceive the deep spatial features, thereby achieving a better feature information mining effect.

[0029] The first enhanced residual module 4 is used to perform the third operation on the previous feature map, and the subsequent enhanced residual module 4 is used to perform the third operation on the last registered feature map and the last residual feature map (output by the enhanced residual module 4). In one possible implementation, the internal operation process of the enhanced residual module 4 includes: Perform global average pooling processing on the first-channel feature map and the second-channel feature map of the input enhanced residual module 4 in the channel direction, respectively generating a first pooling feature map and a second pooling feature map with channel 1; Add the corresponding elements of the first pooling feature map and the second pooling feature map to generate a third pooling feature map; The third pooled feature map after activation (for example, sigmoid function activation) is multiplied by the corresponding element of the feature map to generate a residual feature map.

[0030] Among them, for the first enhanced residual module 4, the one-way feature map and the two-way feature map are both the previous feature maps; for the subsequent enhanced residual module 4, the one-way feature map is the previous registered feature map, and the two-way feature map is the previous residual feature map.

[0031] The image generation module performs a fourth operation on the last registered feature map 5 and the third pooled feature map generated in all the previous enhanced residual modules 4 to generate an improved MRI image. In one possible implementation, the fourth operation process within the image generation module includes: The third pooling feature maps generated in all the previous enhanced residual modules 4 are extracted, and these third pooling feature maps are spliced ​​together to generate a first improved feature map.

[0032] The second convolution calculation and the second activation calculation are performed on the first improved feature map in sequence to generate a second improved feature map with a channel of 1. During the second convolution calculation, the convolution kernel size can be 3*3, the step size is usually 1, and the second activation calculation can be completed using commonly used activation functions such as ReLU.

[0033] The second improved feature map is multiplied by the corresponding elements of the last registered feature map 5 to generate a third improved feature map; in this embodiment, the last registered feature map 5 is a feature map generated by adding the corresponding elements of the sixth convolutional feature map output by the fourth information mining module and the residual feature map output by the fourth enhanced residual module.

[0034] The third improved feature map is sequentially subjected to the third convolution calculation and the third activation calculation to generate an improved MRI image with a channel of 3. During the third convolution calculation, the convolution kernel size may be 3*3, the step size is usually 1, and the third activation calculation may be completed using commonly used activation functions such as ReLU.

[0035] Finally, it should be noted that the above description is only a preferred embodiment of the present invention. Under the guidance of the present invention, ordinary technicians in this field can make various similar expressions without violating the purpose and claims of the present invention, and such changes all fall within the scope of protection of the present invention.

Claims

1. A prostate MRI image optimization method for clinical surgery, characterized in that: The following steps are involved: Retrieving an initial MRI image, retrieving a trained data fitting network, and using the data fitting network to adjust the initial MRI image to generate an optimized improved MRI image; The data fitting network is provided with an information mining module and an enhanced residual module. The information mining module is used to mine the image features of the initial MRI image, and the enhanced residual module is used to process the feature map in front of the information mining module to generate a residual feature map, and then use the residual feature map to align the feature map behind the information mining module.

2. The prostate MRI image optimization method for clinical surgery according to claim 1, characterized in that: The residual feature map and the feature map after the information mining module are aligned by adding corresponding elements.

3. The prostate MRI image optimization method for clinical surgery according to claim 1, characterized in that: The data fitting network is provided with a pre-operator, a plurality of information mining modules and a plurality of enhanced residual modules, wherein the pre-operator is provided in front of the information mining module and the enhanced residual module, and the pre-operator generates a pre-feature map after performing a first operation on the initial MRI image; The multiple information mining modules are arranged in sequence front to back, and the enhanced residual module is set corresponding to the information mining module; the first information mining module is used to perform a second operation on the previous feature map, and the subsequent information mining module is used to perform a second operation on the previous aligned feature map, the first enhanced residual module is used to perform a third operation on the previous aligned feature map, and the subsequent enhanced residual module is used to perform a third operation on the previous aligned feature map and the previous residual feature map.

4. The prostate MRI image optimization method for clinical surgery according to claim 3, characterized in that: The internal operation process of the pre-operator includes: The first convolution calculation and the first activation calculation are sequentially performed on the initial MRI image to generate the pre-feature map.

5. The prostate MRI image optimization method for clinical surgery according to claim 1 or 3, characterized in that: The internal operation process of the information mining module includes: The feature map of the input information mining module is calculated using a plurality of convolutional components with different receptive field sizes to generate a plurality of first convolutional feature maps; Fusing a plurality of the first convolutional feature maps to generate a second convolutional feature map; Subtracting each first convolution feature map from the second convolution feature map to generate a plurality of third convolution feature maps; Activating each of the third convolutional feature maps respectively to generate a plurality of fourth convolutional feature maps; Perform Hadamard product on multiple fourth convolution feature maps to generate a fifth convolution feature map; The second convolutional feature map is fused with the fifth convolutional feature map to generate a sixth convolutional feature map of the output information mining module.

6. The prostate MRI image optimization method for clinical surgery according to claim 3, characterized in that: The internal operation process of the enhanced residual module includes: Perform global average pooling processing on the first-channel feature map and the second-channel feature map of the input enhanced residual module in the channel direction to generate the first pooling feature map and the second pooling feature map respectively; Adding corresponding elements of the first pooling feature map and the second pooling feature map to generate a third pooling feature map; The activated third pooling feature map is fused with the one-way feature map to generate the residual feature map; Among them, for the first enhanced residual module, the one-way feature map and the two-way feature map are both the previous feature map; for the subsequent enhanced residual module, the one-way feature map is the previous registered feature map, and the two-way feature map is the previous residual feature map.

7. The prostate MRI image optimization method for clinical surgery according to claim 6, characterized in that: An image generation module is provided at the tail of the data fitting network, and the image generation module generates the improved MRI image after performing a fourth operation on the last registered feature map and all the third pooled feature maps.

8. The prostate MRI image optimization method for clinical surgery according to claim 7, characterized in that: The internal operation process of the image generation module includes: All the third pooled feature maps are concatenated to generate the first improved feature map; performing a second convolution calculation and a second activation calculation on the first improved feature map in sequence to generate a second improved feature map; Fusion the second improved feature map with the last registered feature map to generate a third improved feature map; The third improved feature map is sequentially subjected to a third convolution calculation and a third activation calculation to generate the improved MRI image.

9. A prostate MRI image optimization device for clinical surgery, characterized in that: The method comprises a computer program / instruction, and when the computer program / instruction is executed, the method according to any one of claims 1 to 8 is implemented.

Images