MRI hemodynamics-based breast tumor segmentation method, system, equipment and medium

Abstract

The invention discloses a breast tumor segmentation method, system and device based on MRI hemodynamics and a medium. The method comprises the following steps: preprocessing MRI multi-phase image data to be segmented, including resampling, unloading, registering and denoising; inputting the preprocessed MRI multi-phase image data to be segmented into a pre-trained UNet breast region segmentation model for breast region pre-segmentation; carrying out difference value operation on the non-enhancement period image and the enhancement most significant period image to obtain a difference value of the two periods of images; constructing and training a breast MRI tumor segmentation network of a double-branch parallel encoder and decoder; and inputting an image to be segmented into the trained breast MRI tumor segmentation network to obtain a breast tumor area mask of the whole image. According to the method, the breast region pre-segmentation process is designed according to the characteristics of the MRI image, the global features and the local features of the image are effectively extracted through a ViT and CNNs combined network, the influence of normal tissues and organs on tumor segmentation is weakened, and false positive regions are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of tumor image segmentation, and specifically relates to a breast tumor segmentation method, system, device and medium based on MRI hemodynamics. Background Art

[0002] Magnetic resonance imaging (MRI) is one of the most commonly used non-invasive diagnostic and treatment tools in clinical practice. DCE-MRI is an important imaging modality for evaluating breast tumors. By injecting a contrast agent, it can prominently display the blood vessels and capsule morphology in breast tissue, and can provide the morphological and functional characteristics of breast tumors, which is helpful for the detection, localization and staging of breast cancer. For multi-phase images at multiple time points during an MRI scan, the tumor region gradually becomes brighter, while the other normal tissue and organ regions change little or have a small degree of enhancement. By performing a difference operation on the image of the phase with the most significant tumor enhancement and the non-enhanced phase, the influence of normal tissue and organ and artifact regions on tumor segmentation can be eliminated. Combining the network structures of ViT and CNNs can comprehensively extract the global and local features of the image.

[0003] Previous studies have shown that the DCE-MRI tumor segmentation algorithm is... The current research methods are mainly divided into the following two categories: (1) Segmentation algorithms based on convolutional neural networks, such as UNet, VNet, and ResUNet, are methods with robustness in multi-modal and multi-class organ tumor segmentation. In addition, segmentation algorithms designed based on the prior knowledge of DCE-MRI, such as MHL (Hierarchical convolutional neural networks for segmentation of breast tumors in mri with application to radiogenomics.), and tumor detection networks combined with tumor segmentation networks (Breast tumor segmentation in dce-mri with tumor sensitive synthesis.). (2) Segmentation algorithms based on ViT and CNNs, such as UNETR, Swin UNETR, and TransUNet, are medical image segmentation algorithms. However, there is no robust segmentation algorithm for breast tumors.

[0004] The above research methods have the following main disadvantages: (1) There is no standardized and complete image preprocessing process. MRI images do not have a standardized measurement unit like the HU value of CT images, and the gray values of scan result images with different models and different parameter settings in different centers vary significantly.

[0005] (2) There is no effective way to eliminate the false positive regions generated in the segmentation results, such as glands, normal organs, and artifacts; in MRI images, the gray value distributions of some gland or normal organ regions are similar to those of tumor regions. In addition, due to uneven magnetic field strength and scanning environment, image artifacts will be generated, and the gray value distributions of some artifacts are also similar to those of tumor regions.

[0006] (3) It fails to effectively distinguish the tumor region from the surrounding normal tissue and organ regions with similar gray values; the growth location and malignancy degree of the tumor make it difficult to distinguish some tumors from the surrounding tissues and organs. Summary of the Invention

[0007] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a breast tumor segmentation method, system, device and medium based on MRI hemodynamics. By combining ViT and CNNs with multi-phase MRI image information, a new dual-branch network structure for breast tumor segmentation is invented. The network encoder combines ViT and CNNs to fully utilize the global and local information of the image, the backbone network performs accurate tumor segmentation, and the branch network eliminates or weakens the influence of false positive regions.

[0008] To achieve the above purpose, the present invention adopts the following technical solutions: One aspect of the present invention provides a breast tumor segmentation method based on MRI hemodynamics, including the following steps: Preprocess the MRI multi-phase image data to be segmented, including resampling, transfer storage, registration, and noise reduction; Input the preprocessed MRI multi-phase image data to be segmented into a pre-trained UNet breast region segmentation model to segment the breast region of the non-enhanced phase image to obtain a breast region mask, and then remove the other parts of the non-enhanced phase image through the breast region mask, and perform standard normalization on the image within the breast region mask to obtain the processed non-enhanced phase image ; Perform a difference operation on the processed non-enhanced phase image and the most significantly enhanced phase image to obtain the difference between the two-phase images ; Construct and train a breast MRI tumor segmentation network, the breast MRI tumor segmentation network includes a parallel backbone network and a branch network, both the backbone network and the branch network include an encoder and a decoder, and the input images of the backbone network and the branch network are respectively and ; Input the image to be segmented into the trained breast MRI tumor segmentation network to obtain a breast tumor region mask for the entire image.

[0009] As a preferred technical solution, the resampling and transfer storage are specifically as follows: The multi-phase MRI image data to be segmented and the tumor mask are resampled to a resolution of (1, 1, 1), and the original dicom image is transferred and stored as a nii image. Among them, the multi-phase MRI image data to be segmented is resampled using the trilinear interpolation algorithm, and the tumor mask is resampled using the nearest neighbor interpolation algorithm.

[0010] As a preferred technical solution, the registration and noise reduction are specifically as follows: Use the deedsBCV image registration algorithm to register the image with the most significant enhancement phase image in the original multi-phase MRI image as the image to be registered and the non-enhanced phase image as the reference image for image registration;

[0011] As a preferred technical solution, the standard normalization is specifically as follows: ; wherein, is the intensity value of the original image pixel point, is the intensity value of the image pixel point after standard normalization of the MR image, is the mean value of the intensity values of the original image pixel points, is the standard deviation of the intensity values of the original image pixel points.

[0012] As a preferred technical solution, the encoder includes a Conv module and a Swin Transformer module, and the decoder includes an Up block module and an Outblock module; Both the Conv module and the Out block module include three consecutive convolutional layers with a convolutional kernel size of 3×3×3; The Up block module includes three consecutive transposed convolutional layers with a convolutional kernel size of 3×3×3; The Swin Transformer module is a module structure based on the self-attention mechanism. By dividing the image into small blocks, the self-attention of each block is calculated through a sliding window; Among them, the output end of the Up block module in the branch network decoder is correspondingly connected to the input end of the Up block module in the backbone network decoder.

[0013] As a preferred technical solution, the loss function of the breast MRI tumor segmentation network includes the backbone network loss function And the branch network loss function , with a weight ratio of 1:1, as shown in the following formula: ; Among them, the backbone network and the branch network include the dice loss function and the cross-entropy loss function of their respective branches, specifically: ; ; During the training process, 1000 rounds of iterations are performed. The Adam optimizer is used, and the initial learning rate is set to 0.0001. The learning rate adjustment strategy is the cosine annealing algorithm.

[0014] As a preferred technical solution, inputting the image to be segmented into the trained breast MRI tumor segmentation network to obtain the breast tumor region mask of the entire image, specifically: Select image pairs of the input data in the form of a sliding window, with an overlap rate of 0.7 for sliding translation, complete the inference process of the entire image, and obtain the breast tumor region mask of the entire image.

[0015] Another aspect of the present invention also provides a breast tumor segmentation system based on MRI hemodynamics, which is applied to the above-mentioned breast tumor segmentation method based on MRI hemodynamics, and includes a data preprocessing module, a pre-segmentation module, a difference operation module, and a breast MRI tumor segmentation network; The data preprocessing module is used to preprocess the multi-phase MRI image data to be segmented, including resampling, transfer storage, registration, and noise reduction; The pre-segmentation module uses a pre-trained UNet breast region segmentation model to segment the preprocessed multi-phase MRI image data to be segmented, and obtains the breast region and breast region mask of the non-enhanced phase image , and then removes other parts of the non-enhanced phase image through the breast region mask, and performs standard normalization on the image intercepted by the breast region mask; The difference operation module is used to perform a difference operation on the processed non-enhanced phase image and the most significantly enhanced phase image to obtain the difference between the two-phase images ; The breast MRI tumor segmentation network includes a parallel backbone network and branch network. Both the backbone network and the branch network include an encoder and a decoder. The input images of the backbone network and the branch network are respectively and , which are used to perform breast MRI tumor segmentation on the image to be segmented and obtain the breast tumor region mask of the entire image.

[0016] Another aspect of the present invention also provides a breast tumor segmentation device based on MRI hemodynamics. The breast tumor segmentation device based on MRI hemodynamics includes: a memory and at least one processor. Instructions are stored in the memory, and the memory and the at least one processor are interconnected by a line; the at least one processor calls the instructions in the memory to enable the breast tumor segmentation device based on MRI hemodynamics to execute the above-mentioned breast tumor segmentation method based on MRI hemodynamics.

[0017] Another aspect of the present invention also provides a storage medium storing a program, which when executed by a processor, implements the above-mentioned breast tumor segmentation method based on MRI hemodynamics.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) Compared with the prior art, according to the characteristics of MRI images themselves, the present invention designs a set of pre-segmentation processes for the breast region, which can weaken the influence of normal tissue organs such as the heart on tumor segmentation and reduce the occurrence of false positive regions.

[0019] (2) Compared with the prior art, the present invention makes full use of the characteristics that the image intensity values of different regions of multi-phase MRI images based on hemodynamics change differently, greatly weakens the influence of breast fibroglandular and image artifact regions, and reduces false positive regions in the segmentation results.

[0020] (3) The present invention effectively extracts global features and local features of images through a network combining ViT and CNNs. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flowchart of the breast tumor segmentation method based on MRI hemodynamics according to an embodiment of the present invention; Figure 2 is a schematic structural diagram of a breast MRI tumor segmentation network according to an embodiment of the present invention; Figure 3 is a schematic structural diagram of a breast tumor segmentation system based on MRI hemodynamics according to an embodiment of the present invention; Figure 4 is a schematic structural diagram of a storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0023] Embodiment: As Figure 1 shown, this embodiment provides a method for segmenting breast tumors based on MRI (Magnetic Resonance Imaging) hemodynamics, including the following steps: S1. Data preprocessing: Preprocess the multi-phase MRI image data to be segmented, including resampling, conversion and storage, registration, and noise reduction; S11. Sample the multi-phase MRI image data to be segmented and the tumor mask to a resolution of (1, 1, 1), and use the SimpleITK package in the Python language to convert the original dicom image to a nii image. Among them, the multi-phase MRI image data to be segmented is resampled using the trilinear interpolation algorithm, and the tumor mask is resampled using the nearest neighbor interpolation algorithm.

[0024] During the scanning of the patient, due to the influence of the respiratory cycle and the patient's own movement, there will be different degrees of offset in the images of different phases. Therefore, this application uses the deedsBCV image registration algorithm to use the image with the most significant enhancement in the original multi-phase MRI images as the image to be registered , and the non-enhanced phase image as the reference image , and perform image registration to make the difference operation of pixel points between the two-phase images more accurate. The deedsBCV image registration algorithm is a two-stage segmentation algorithm including rigid registration and non-rigid registration.

[0025] Due to the influence of magnetic field inhomogeneity, uneven sensitivity of the receiving coil, and the patient's own conditions, the brightness of the magnetic resonance image is uneven. Therefore, this application uses the N4 correction algorithm to re-model the non-enhanced phase image and the image with the most significant enhancement after registration, and uses the low-frequency information in the image to estimate and correct the bias field, correct the uneven brightness area, make the image clearer, improve the image quality, and achieve noise reduction.

[0026] In one or more preferred embodiments, the non-local means filtering noise reduction algorithm, total variation denoising algorithm, convolutional neural network CNN, and generative adversarial network GAN can also be used to reduce the noise of the registered image.

[0027] S2. Pre-segmentation of the breast region: Input the pre-processed multi-phase MRI image data to be segmented into a pre-trained UNet breast region segmentation model to segment the non-enhanced phase image of the breast region, obtain a breast region mask, and then remove other parts of the non-enhanced phase image through the breast region mask. Standard normalization is performed on the image intercepted by the breast region mask to obtain the processed non-enhanced phase image , where is the intensity value of the original image pixel point, is the mean value of the intensity values of the original image pixel points, is the standard deviation of the intensity values of the original image pixel points, as shown in the following formula: ; Perform a difference operation on the processed non-enhanced phase image obtained after step S2 and the most significantly enhanced phase image to obtain the difference between the two-phase images .

[0028] S3. Construct a breast MRI tumor segmentation network: The breast MRI tumor segmentation network includes a parallel backbone network and a branch network. Both the backbone network and the branch network include an encoder and a decoder (dual-branch parallel encoder and decoder). The input images of the backbone network and the branch network are respectively and , and provides accurate tumor edge information, provides tumor localization information for the tumor segmentation network, and weakens the influence of false positive regions on the tumor segmentation task.

[0029] Furthermore, as Figure 2 shown, the encoder includes a Conv module and a Swin Transformer module, and the decoder includes an Up block module and an Out block module; Both the Conv module and the Out block module include three consecutive convolutional layers with a convolutional kernel size of 3×3×3; The Up block module includes three consecutive deconvolutional layers with a convolutional kernel size of 3×3×3; The Swin Transformer module is a module structure based on the self-attention mechanism. By dividing the image into small blocks, the self-attention of each block is calculated through a sliding window, and it can be calculated at different levels to obtain information of different scales of the image; Among them, there is parameter interaction between the decoders of the backbone network and the branch network, that is, the output end of the Upblock module in the decoder of the branch network is correspondingly connected to the input end of the Up block module in the decoder of the backbone network.

[0030] S4. Training the breast MRI tumor segmentation network: For and phase images, in each round of iterative training, image patches with a size of 96×96×96 at the same position are selected for each image pair, and the ratio of the center points of the image patches inside and outside the tumor area is 1:1 to ensure the balance of positive and negative samples in model training.

[0031] The loss function of the breast MRI tumor segmentation network includes the backbone network loss function and the branch network loss function , and the weight ratio is 1:1, as shown in the following formula: ; Among them, the backbone network and the branch network include the dice loss function and the cross-entropy loss function of their respective branches, specifically: ; ; During the training process, train the attack for 1000 rounds of iteration. The optimizer uses the Adam optimizer, the initial value of the learning rate is set to 0.0001, and the learning rate adjustment strategy is the cosine annealing algorithm.

[0032] S5. Input the image to be segmented into the trained breast MRI tumor segmentation network to obtain the breast tumor region mask of the entire image.

[0033] According to the breast MRI tumor segmentation network trained in step S4, select the image pair of the input data in the form of a sliding window, and the overlapping rate of the sliding translation is 0.7 to finally realize the inference process of the entire image and obtain the breast tumor region mask of the entire image.

[0034] As Figure 3 shown, in another embodiment of the present application, a breast tumor segmentation system based on MRI (magnetic resonance imaging) hemodynamics is provided. The system includes a data preprocessing module, a pre-segmentation module, a difference operation module, and a breast MRI tumor segmentation network; The data preprocessing module is used to preprocess the MRI multi-phase image data to be segmented, including resampling, transfer storage, registration, and noise reduction; The pre-segmentation module uses a pre-trained UNet breast region segmentation model to segment the preprocessed MRI multi-phase image data to be segmented to obtain non-enhanced phase images in the breast region and the breast region mask, and then use the breast region mask to remove other parts of the non-enhanced phase image and perform standard normalization on the image intercepted by the breast region mask; The difference operation module is used to perform a difference operation on the processed non-enhanced phase image and the most significantly enhanced phase image to obtain the difference between the two-phase images ; The breast MRI tumor segmentation network includes a parallel backbone network and a branch network. Both the backbone network and the branch network include an encoder and a decoder. The input images of the backbone network and the branch network are respectively and for performing breast MRI tumor segmentation on the image to be segmented and obtaining the breast tumor region mask of the entire image.

[0035] It should be noted here that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above. This system is applied to the breast tumor segmentation method based on MRI hemodynamics in the above embodiment.

[0036] In another embodiment of the present application, a breast tumor segmentation device based on MRI hemodynamics is further provided. The breast tumor segmentation device based on MRI hemodynamics includes: a memory and at least one processor. Instructions are stored in the memory, and the memory and the at least one processor are interconnected by a line; the at least one processor calls the instructions in the memory so that the breast tumor segmentation device based on MRI hemodynamics executes the breast tumor segmentation method based on MRI hemodynamics as in the above embodiment.

[0037] As Figure 4 shown, in another embodiment of the present application, a storage medium is further provided, storing a program, which when executed by a processor, implements a breast tumor segmentation method based on MRI hemodynamics, specifically: Preprocess the MRI multi-phase image data to be segmented, including resampling, transfer, registration, and noise reduction; Preprocess the MRI multi-phase image data to be segmented, including resampling, transfer, registration, and noise reduction; Input the preprocessed MRI multi-phase image data to be segmented into a pre-trained UNet breast region segmentation model to segment the non-enhanced phase image In the breast area, a breast area mask is obtained, and then the non-enhanced phase image is with other parts removed through the breast area mask, and the image within the breast area mask is subjected to standard normalization to obtain the processed non-enhanced phase image ; The processed non-enhanced phase image and the most significantly enhanced phase image are subjected to a difference operation to obtain the difference between the two-phase images ; Construct and train a breast MRI tumor segmentation network. The breast MRI tumor segmentation network includes a parallel backbone network and a branch network. Both the backbone network and the branch network include an encoder and a decoder. The input images of the backbone network and the branch network are respectively and ; Input the image to be segmented into the trained breast MRI tumor segmentation network to obtain the breast tumor area mask of the entire image.

[0038] It should be understood that each part of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0039] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A breast tumor segmentation method based on MRI hemodynamics, characterized in that: The steps include: Preprocess the multi-phase MRI image data to be segmented, including resampling, transfer, registration and noise reduction; The pre-processed MRI multi-phase image data to be segmented is input into the pre-trained UNet breast region segmentation model to segment the non-enhanced phase images. The breast area is obtained by the breast area mask, and then the non-enhanced image is transformed into The rest of the image is removed, and the image within the breast area mask is normalized to obtain the processed non-enhanced image. ; The processed non-enhanced image and the most significant enhancement period image Perform difference operation to obtain the difference between the two images ; Construct and train a breast MRI tumor segmentation network, wherein the breast MRI tumor segmentation network includes a parallel backbone network and a branch network, wherein the backbone network and the branch network both include an encoder and a decoder, and the input images of the backbone network and the branch network are respectively and ; The image to be segmented is input into the trained breast MRI tumor segmentation network to obtain the breast tumor region mask of the entire image.

2. The method for segmenting breast tumors based on MRI hemodynamics according to claim 1, characterized in that: The resampling and transfer are specifically as follows: The MRI multi-phase image data to be segmented and the tumor mask are resampled to a resolution of (1,1,1), and the original DICOM image is converted into a NII image. The MRI multi-phase image data to be segmented are resampled using a trilinear interpolation algorithm, and the tumor mask is resampled using a nearest neighbor interpolation algorithm.

3. The method for segmenting breast tumors based on MRI hemodynamics according to claim 1, characterized in that: The registration and noise reduction are specifically as follows: The deedsBCV image registration algorithm is used to register the most significantly enhanced image in the original multi-phase MRI image. As the image to be registered , non-enhanced image As a reference image , perform image registration; The registered image is denoised using the N4 correction algorithm, the non-local mean filtering denoising algorithm or the total variation denoising algorithm.

4. The method for segmenting breast tumors based on MRI hemodynamics according to claim 1, characterized in that: The standard normalization is specifically as follows: ； in, is the original image pixel intensity value, is the pixel intensity value of the MR image after standard normalization, is the mean value of the original image pixel intensity, is the standard deviation of the original image pixel intensity value.

5. The method for segmenting breast tumors based on MRI hemodynamics according to claim 1, characterized in that: The encoder includes a Conv module and a Swin Transformer module, and the decoder includes an Up block module and an Outblock module; The Conv module and the Out block module each include three consecutive convolutional layers with a convolution kernel size of 3×3×3; The Up block module includes three consecutive deconvolution layers with a convolution kernel size of 3×3×3; The Swin Transformer module is a module structure based on the self-attention mechanism, which divides the image into small blocks and calculates the self-attention of each block through a sliding window; The output end of the Up block module in the branch network decoder is correspondingly connected to the input end of the Upblock module in the trunk network decoder.

6. The method for segmenting breast tumors based on MRI hemodynamics according to claim 1, characterized in that: The loss function of the breast MRI tumor segmentation network includes the backbone network loss function And the branch network loss function , the weight ratio is 1:1, as shown below: ； Among them, the backbone network and branch network include the dice loss function and cross entropy loss function of their respective branches, specifically: ； ； During the training process, 1000 iterations were performed, the optimizer used the Adam optimizer, the initial value of the learning rate was set to 0.0001, and the learning rate adjustment strategy was the cosine annealing algorithm.

7. The method for segmenting breast tumors based on MRI hemodynamics according to claim 1, characterized in that: The image to be segmented is input into the trained breast MRI tumor segmentation network to obtain the breast tumor region mask of the entire image, specifically: The image pairs of input data are selected in the form of a sliding window, and the overlapping rate of sliding translation is 0.

7. The reasoning process of the entire image is completed to obtain the breast tumor area mask of the entire image.

8. A breast tumor segmentation system based on MRI hemodynamics, characterized in that: A breast tumor segmentation method based on MRI hemodynamics applied to any one of claims 1-7, comprising a data preprocessing module, a pre-segmentation module, a difference operation module and a breast MRI tumor segmentation network; The data preprocessing module is used to preprocess the MRI multi-phase image data to be segmented, including resampling, transfer, registration and noise reduction; The pre-segmentation module uses the pre-trained UNet breast region segmentation model to segment the pre-processed MRI multi-phase image data to obtain the non-enhanced phase image. The breast region and breast region mask are then used to transform the non-enhanced image into The rest of the parts are removed and the image intercepted by the breast area mask is normalized; The difference operation module is used to process the processed non-enhancement period image and the most significant enhancement period image Perform difference operation to obtain the difference between the two images ; The breast MRI tumor segmentation network includes a parallel main network and a branch network, wherein the main network and the branch network both include an encoder and a decoder, and the input images of the main network and the branch network are respectively and , which is used to perform breast MRI tumor segmentation on the image to be segmented and obtain the breast tumor region mask of the entire image.

9. A breast tumor segmentation device based on MRI hemodynamics and hemodynamics, characterized in that: The breast tumor segmentation device based on MRI hemodynamics includes: a memory and at least one processor, the memory storing instructions, the memory and the at least one processor being interconnected through lines; the at least one processor calls the instructions in the memory so that the breast tumor segmentation device based on MRI hemodynamics executes the breast tumor segmentation method based on MRI hemodynamics as described in any one of claims 1-7.

10. A storage medium storing a program, characterized in that: When the program is executed by a processor, the breast tumor segmentation method based on MRI hemodynamics according to any one of claims 1 to 8 is implemented.

Images