A method, system, and medium for segmenting the breast, tumor, and glandular tissue in breast MRI.

Abstract

This invention discloses a method, system, and medium for segmenting the breast, tumor, and glandular tissue in breast MRI. First, multi-phase MRI images of the patient are acquired using dynamic contrast-enhanced magnetic resonance imaging (MRI). Then, the multi-phase MRI images are preprocessed to obtain processed images. Next, a breast MRI segmentation network is constructed based on the UNET architecture, including a coarse segmentation network and a fine segmentation network; both networks include an encoder and a decoder. The processed images are then used to train the breast MRI segmentation network, resulting in a trained network. Finally, the multi-phase MRI images to be predicted are input into the trained network to obtain images of the breast region, tumor region, and glandular tissue region. This invention performs segmentation twice using coarse and fine segmentation networks, simultaneously segmenting the breast, glandular tissue, and tumor regions, thus solving the problems of interference from other organs and excessive computational load associated with directly segmenting tumors.

Description

Technical Field

[0001] This invention belongs to the technical field of digital image processing and neural networks, specifically relating to a method, system, and medium for segmenting the breast, tumor, and glandular tissue in breast MRI. Background Technology

[0002] Currently, in the early detection of breast cancer, magnetic resonance imaging (MRI) is typically used to observe the breast tissue of breast cancer patients to determine the presence, location, and type of breast tumors. MRI, as one of the most widely used non-invasive diagnostic tools in the medical field, plays a crucial role in breast cancer assessment. One key imaging modality is dynamic contrast-enhanced MRI (DCE-MRI), which enhances the contrast of MRI images by injecting contrast agents. This significantly highlights the morphology of blood vessels and capsules within breast tissue, providing crucial information on the appearance and functional characteristics of breast tumors, aiding in the detection, localization, and staging of breast cancer. In a single DCE-MRI scan, through multiple image acquisitions, the tumor region gradually reveals significant features, while changes in other normal tissues and organs are relatively small or slight. To accurately segment non-tumor tissues, such as breast tissue and glandular tissue, multi-phase image analysis is necessary to better distinguish between tumors, breast tissue, and glandular tissue structures. Furthermore, employing a network structure combining the Swin Transformer and multi-scale convolutional neural networks (CNNs) can comprehensively extract global and local features of the images, which is essential for more accurate segmentation of tumors, breast tissue, and glandular tissue.

[0003] Current research methods are mainly divided into the following three categories: (1) segmentation algorithms based on convolutional neural networks, such as UNet, VNet and ResUNet, which are general UNET-type segmentation networks that can be used for multi-class segmentation; (2) medical image segmentation networks based on pure Transformer U-Net, such as Swin-Unet; (3) fusion segmentation algorithms based on ViT and CNNs, such as UNETR, Swin UNETR and TransUNet. However, there are also corresponding problems and disadvantages: First, because the entire breast MRI image is too large, if the entire 3D image is to be put into the model, in order to prevent insufficient video memory, the resolution can generally only be adjusted to a very small size, which will cause the image resolution to be compressed a lot, resulting in glands and tumors having only a few pixels, which cannot even be distinguished by the naked eye, making the model effect very poor; Second, the entire breast MRI image also contains information of other organs, especially the most significant phase in the multi-phase MRI image, which has the greatest interference to the model, because other organs will also be "lit up" at this time. After normalizing the image, the tumor area will not be so "lit up", resulting in poor segmentation results. Current technologies for tumor segmentation typically involve first segmenting the breast tissue using other methods, i.e., segmenting the region of interest before segmenting the tumor or glandular tissue. This often requires running multiple models, as there is no single, effective method to segment the breast tissue, glandular tissue, and tumor using a single model. However, directly using a general multi-classification model without segmenting the region of interest leads to problems such as low resolution and dark tumor regions, resulting in poor segmentation results. Furthermore, few models simulate the way doctors interpret images by using multiple phases of images to segment tumors, thus introducing false positives in tumor segmentation results. In summary, these issues increase the complexity of breast cancer detection, treatment planning, and subsequent management, severely impacting the efficiency and accuracy of clinical practice. Summary of the Invention

[0004] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a method, system, and medium for segmenting the breast, tumor, and glandular tissue in breast MRI. By performing two-stage segmentation using coarse and fine segmentation networks, the breast, glandular tissue, and tumor regions can be segmented simultaneously, making full use of the spatial information of MRI images and improving the performance and accuracy of the model.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The primary objective of this invention is to provide a method for segmenting the breast, tumor, and glandular tissue in breast MRI, comprising the following steps:

[0007] The patient's MRI images were acquired using a dynamic contrast-enhanced magnetic resonance imaging method; the MRI images included non-contrast phase images and most salient phase images.

[0008] The preprocessing of multi-phase MRI images yields processed images; the preprocessing includes resampling, image registration, normalization, and resolution adjustment.

[0009] A breast MRI segmentation network is constructed based on the UNET architecture. The breast MRI segmentation network includes a coarse segmentation network and a fine segmentation network. The coarse segmentation network and the fine segmentation network have the same network structure but different weight parameters. The coarse segmentation network is used to obtain the breast mask. The fine segmentation network is used to obtain the tumor mask and the gland mask. Both the coarse segmentation network and the fine segmentation network include an encoder and a decoder. The encoder is used to extract image features, and the decoder is used to obtain the image mask.

[0010] The processed images were used to train the breast MRI segmentation network, resulting in a well-trained breast MRI segmentation network.

[0011] The multi-phase MRI images to be predicted are input into the trained breast MRI segmentation network to obtain images of the breast region, tumor region, and glandular region of the multi-phase MRI images to be predicted.

[0012] As a preferred technical solution, the preprocessing of multi-phase MRI images to obtain the processed image specifically includes:

[0013] First, the multi-phase MRI images were resampled to a resolution of (1,1,1);

[0014] Then, the deedsBCV image registration algorithm was used to register the most salient phase image in the multi-phase MRI images to the non-enhanced phase image.

[0015] Standard normalization is performed on the non-enhanced images using the following formula:

[0016]

[0017] Where x is the pixel intensity value of the unenhanced image before standard normalization, x scaled σ represents the pixel intensity value of the non-enhanced image after standard normalization, μ represents the mean pixel intensity value of the non-enhanced image before standard normalization, and σ represents the standard deviation of the pixel intensity value of the non-enhanced image before standard normalization.

[0018] The resolution of the non-enhanced phase image and the most salient phase image were adjusted to the specified size using cubic spline interpolation.

[0019] As a preferred technical solution, the encoder is composed of CNNs and ViT; the decoder is composed of deconvolution.

[0020] Both the convolution and deconvolution in the encoder and decoder are three-dimensional convolutions.

[0021] As a preferred technical solution, the encoder includes a convolutional network layer, N+1 hybrid layers, and N downsampling layers; the hybrid layers and downsampling layers are connected alternately in sequence; the convolutional network layer is connected to the first hybrid layer;

[0022] The decoder includes N+1 feature fusion layers and N upsampling layers; the feature fusion layers and upsampling layers are connected alternately in sequence;

[0023] The (N+1)th mixing layer is connected to the first feature fusion layer.

[0024] As a preferred technical solution, the hybrid layer consists of multi-scale attention convolution (MSCA), ResNetBlock, and SwinTransformer;

[0025] The SwinTransformer is connected to ResNetBlock and the multi-scale attention convolutional MSCA, respectively.

[0026] The multi-scale attention convolution (MSCA) is used to extract local features of the input image;

[0027] The ResNetBlock is used to extract global features from the input image;

[0028] The SwinTransformer is used to summarize and fuse local and global features, and output the fused features of the input image.

[0029] As a preferred technical solution, the process of obtaining the trained breast MRI segmentation network is as follows:

[0030] The non-enhanced phase image in the processed image is input into the encoder of the coarse segmentation network to obtain the global field of view and gland information of the entire image to obtain the non-enhanced phase image features;

[0031] The features of the non-enhanced image are fed into the decoder of the coarse segmentation network to segment out the breast mask of the non-enhanced image;

[0032] The most salient phase image in the processed image is normalized and segmented into an image patch containing the tumor region with a set resolution based on the existing tumor label;

[0033] Image patches are fed into the encoder of a fine segmentation network to extract image patch features;

[0034] The non-enhanced image features and image patch features are fused and input into the decoder of the fine segmentation network to obtain the tumor mask and gland mask;

[0035] The loss function is calculated based on the breast mask obtained from the coarse segmentation network, the tumor mask and gland mask obtained from the fine segmentation network, and the corresponding real mask. The parameters of the breast MRI segmentation network are then updated through backpropagation.

[0036] The training continues iteratively until the breast MRI segmentation network converges or reaches the maximum number of iterations, resulting in a well-trained breast MRI segmentation network.

[0037] As a preferred technical solution, the loss function of the breast MRI segmentation network consists of the Dice loss function and the cross-entropy loss function, expressed as follows:

[0038]

[0039] Among them, y mask Image mask predicted by a breast MRI segmentation network. For real mask;

[0040] Loss Dice The Dice loss function is expressed as:

[0041]

[0042] Loss CE The cross-entropy loss function is expressed as:

[0043]

[0044] M represents the number of categories; in a coarse segmentation network, M = 1, and in a fine segmentation network, M = 2; y i,c Let c be the true class of pixel i. When the true class of pixel i is c, y i,c =1, otherwise y i,c =0; p i,c Predict the probability that pixel i belongs to category c for a breast MRI segmentation network.

[0045] As a preferred technical solution, the multi-phase MRI images to be predicted do not contain tumor labels;

[0046] First, the multi-phase MRI images to be predicted are preprocessed;

[0047] The non-enhanced phase image from the preprocessed multi-phase MRI images to be predicted is directly fed into the trained breast MRI segmentation network to obtain the breast mask of the multi-phase MRI images to be predicted. The breast mask is then multiplied with the preprocessed most salient phase image to obtain the breast region image of the most salient phase image in the multi-phase MRI images to be predicted.

[0048] For the most significant phase image in the preprocessed multi-phase MRI images to be predicted, a sliding window method is used to select image blocks. First, the overlap rate of the sliding window is set, and then image blocks of the same size as the sliding window are selected starting from the upper left corner of the most significant image. These blocks are then sequentially placed into the trained breast MRI segmentation network to obtain the tumor mask and gland mask of the multi-phase MRI images to be predicted. The obtained tumor mask and gland mask are then concatenated and multiplied with the preprocessed most significant phase image to obtain the tumor region image and gland region image, respectively.

[0049] The second objective of this invention is to provide a breast, tumor, and glandular segmentation system for breast MRI, comprising an image acquisition module, a preprocessing module, a network construction module, a network training module, and a segmentation output module;

[0050] The image acquisition module is used to acquire multi-phase MRI images of the patient based on dynamic contrast-enhanced magnetic resonance imaging; the multi-phase MRI images include non-contrast phase images and most salient phase images;

[0051] The preprocessing module is used to preprocess multi-phase MRI images to obtain processed images; the preprocessing includes resampling, image registration, normalization, and resolution adjustment.

[0052] The network construction module is used to construct a breast MRI segmentation network based on the UNET structure; the breast MRI segmentation network includes a coarse segmentation network and a fine segmentation network; the coarse segmentation network and the fine segmentation network have the same network structure but different weight parameters; the coarse segmentation network is used to obtain the breast mask; the fine segmentation network is used to obtain the tumor mask and the gland mask; both the coarse segmentation network and the fine segmentation network include an encoder and a decoder; wherein the encoder is used to extract image features, and the decoder is used to obtain the image mask;

[0053] The network training module is used to train the breast MRI segmentation network using the processed images to obtain a trained breast MRI segmentation network.

[0054] The segmentation output module is used to input the multi-phase MRI images to be predicted into the trained breast MRI segmentation network to obtain breast region images, tumor region images, and glandular region images of the multi-phase MRI images to be predicted.

[0055] The third objective of this invention is to provide a computer-readable storage medium storing a program that, when executed by a processor, implements a method for segmenting the breast, tumor, and glandular tissue in breast MRI.

[0056] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0057] 1. Compared with the prior art, the present invention constructs a breast MRI segmentation network based on the UNET architecture, which can simultaneously segment the breast, glandular tissue and tumor regions.

[0058] 2. Compared with the prior art, the network constructed by the present invention makes full use of the three-dimensional spatial information of radiographic images through three-dimensional convolution. Other methods only take two-dimensional planar slices from three-dimensional images as input, without utilizing spatial information or considering the influence of the relationship between slices on segmentation.

[0059] 3. Compared with the prior art, the network of the present invention performs two segments through coarse and fine segmentation networks, which solves the interference caused by other organs to the direct segmentation of tumors. At the same time, it solves the problem of how to input a high-resolution image into the model and avoid excessive computation. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is an overall flowchart of a method for segmenting the breast, tumor, and glandular tissue using breast MRI, as described in an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of the encoder and decoder in an embodiment of the present invention.

[0063] Figure 3 This is a schematic diagram of the structure of the hybrid layer in an embodiment of the present invention.

[0064] Figure 4 This is a schematic diagram of the training of the breast MRI segmentation network in an embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram of the breast, tumor, and glandular segmentation system for breast MRI according to an embodiment of the present invention.

[0066] Figure 6 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation

[0067] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0068] like Figure 1 As shown in this embodiment, a method for segmenting the breast, tumor, and glandular tissue in breast MRI includes the following steps:

[0069] Step 1: Acquire multi-phase MRI images of the patient based on dynamic contrast-enhanced magnetic resonance imaging (MRI); the multi-phase MRI images include non-contrast phase images and most salient phase images;

[0070] Step 2: Preprocess the multi-phase MRI images to obtain the processed images; preprocessing includes resampling, image registration, normalization, and resolution adjustment, etc.

[0071] Step 3: Construct a breast MRI segmentation network based on the UNET structure, including a coarse segmentation network and a fine segmentation network. The coarse and fine segmentation networks have the same network structure but different weight parameters. The coarse segmentation network is used to obtain the breast mask, and the fine segmentation network is used to obtain the tumor mask and gland mask. Both the coarse and fine segmentation networks include an encoder and a decoder. The encoder is used to extract image features, and the decoder is used to obtain the image mask.

[0072] Step 4: Use the processed images to train the breast MRI segmentation network to obtain a trained breast MRI segmentation network;

[0073] Step 5: Input the multi-phase MRI images to be predicted into the trained breast MRI segmentation network to obtain the breast region image, tumor region image and glandular region image of the multi-phase MRI images to be predicted.

[0074] Furthermore, since directly acquired multi-phase MRI images cannot be directly used for the constructed network, and also to ensure the accuracy of network segmentation, preprocessing is required. The specific process is as follows:

[0075] Step 2.1: Since convolutional networks cannot "understand" the distance relationship between pixels, the MRI multi-phase images are first resampled to a resolution of (1,1,1) to ensure the consistency of pixel information between different MRI images.

[0076] Step 2.2: The input of this network is multi-phase images, which use images of the same patient at different phases. However, due to the respiratory cycle and movement of the patient during the scan, the images at different phases will have different degrees of shift. Therefore, in order to ensure that the positions of the images at different phases of the same patient are matched and aligned, the deedsBCV image registration algorithm is used to register the most significant phase image in the multi-phase MRI images to the non-enhanced phase image.

[0077] The deedsBCV image registration algorithm is an efficient 3D discrete deformable registration algorithm for medical images, developed by Mattias P. Heinrich. The algorithm was comprehensively evaluated in 100 abdominal CT scans for inter-patient registration; it achieved the highest accuracy in all 13 anatomical structures, especially difficult-to-handle structures such as the kidneys, stomach and pancreas.

[0078] Step 2.3: Perform standard normalization on the non-enhanced image. The normalization formula is as follows:

[0079]

[0080] Where x is the pixel intensity value of the unenhanced image before standard normalization, x scaled σ represents the pixel intensity value of the non-enhanced image after standard normalization, μ represents the mean pixel intensity value of the non-enhanced image before standard normalization, and σ represents the standard deviation of the pixel intensity value of the non-enhanced image before standard normalization.

[0081] Step 2.4: Use cubic spline interpolation to adjust the size of the non-enhanced period image and the most salient period image to the specified size. In this embodiment, the non-enhanced period image is adjusted to a smaller resolution, for example (96, 96, 96); the most salient period image is adjusted to a size such as (96, 320, 320) simply to unify the size.

[0082] Next, a breast MRI segmentation network is constructed. This application is based on the UNET architecture and includes a coarse segmentation network and a fine segmentation network. The coarse and fine segmentation networks have the same network structure, only the weight parameters are different. Both the coarse and fine segmentation networks include an encoder and a decoder. The encoder is composed of CNNs and ViT. The decoder is composed of deconvolution. Since MRI images are all three-dimensional, the convolution and deconvolution in the encoder and decoder are all three-dimensional convolutions. It should be understood that the three-dimensional convolution mentioned here should also include three-dimensional deconvolution.

[0083] Furthermore, in this embodiment, the encoder includes a convolutional network layer, N+1 hybrid layers, and N downsampling layers; wherein the hybrid layers and downsampling layers are connected alternately in sequence, and the convolutional network layer is connected to the first hybrid layer; the downsampling layer refers to downsampling through convolution while synthesizing features, so that the feature map becomes smaller and the number of channels increases;

[0084] The decoder consists of N+1 feature fusion layers and N upsampling layers; the feature fusion layers and upsampling layers are connected alternately in sequence; the N+1th mixing layer is connected to the first feature fusion layer; the upsampling layer refers to reducing the number of feature channels and amplifying image features by deconvolution.

[0085] Figure 2 A schematic diagram of an encoder and decoder structure is shown, wherein the encoder includes one convolutional network layer, three hybrid layers, and two downsampling layers, and the decoder includes three feature fusion layers and two upsampling layers. Since the network used in this application employs the UNET architecture, skip connections are used to connect the stage results obtained from the feature fusion layers and the encoder in a channel-skipping manner. It should be understood that... Figure 2 This is only one implementation method, and the structure shown is only used to explain the structure of the encoder and decoder. The number of mixing layers, upsampling layers, feature fusion layers and upsampling layers can be adjusted according to actual needs, and it also falls within the scope of protection of this application.

[0086] Furthermore, in this invention, the hybrid layer is mainly composed of multi-scale channel attention (MSCA), ResNetBlock, and SwinTransformer (e.g., Figure 3 As shown, SwinTransformer is connected to ResNetBlock and Multi-Scale Attention Convolution (MSCA) respectively. This hybrid layer captures local features of the input image at different ranges through MSCA, captures global features of the input image through ResNetBlock, and then uses SwinTransformer to summarize and fuse the features obtained by the two methods to output the fused features of the input image.

[0087] Multi-scale channel attention (MSCA) is a type of multi-scale channel attention convolution that captures local information of different ranges through the receptive fields of different convolution kernels of a feature map, and then stacks the results in the channel direction to obtain local features of different ranges of the feature map.

[0088] Furthermore, a breast MRI segmentation network, trained on the processed images, is constructed, such as... Figure 4 As shown, the specific process is as follows:

[0089] First, the non-enhanced image C in the processed image... ne The global field of view and glandular information of the entire image are obtained by inputting the encoder of the coarse segmentation network to obtain the non-enhanced image features E. ne ;

[0090] non-enhanced image features E ne The non-enhancement period image C is segmented by the decoder of the coarse segmentation network. ne Breast mask;

[0091] The most salient period image C in the processed image peak Normalization is performed, and the image is segmented into an image block containing the tumor region with a set resolution based on the existing tumor label; in this embodiment, the image block resolution is (96, 96, 96).

[0092] Image patches are fed into the encoder of a fine segmentation network to extract image patch features E. peak ;

[0093] non-enhanced image features E ne and image patch features E peak Perform feature fusion to allow E ne For E peak It provides a global view of the entire image and gland information to help segment tumors and glands. The fusion result is then input into the decoder of the fine segmentation network to obtain tumor masks and gland masks.

[0094] The loss function is calculated based on the breast mask obtained from the coarse segmentation network, the tumor mask and gland mask obtained from the fine segmentation network, and the corresponding real mask. The parameters of the breast MRI segmentation network are then updated through backpropagation.

[0095] The training continues iteratively until the breast MRI segmentation network converges or reaches the maximum number of iterations, resulting in a well-trained breast MRI segmentation network.

[0096] In this embodiment, the loss function of the breast MRI segmentation network consists of the Dice loss function and the cross-entropy loss function, expressed as follows:

[0097]

[0098] Among them, y mask Image mask predicted by a breast MRI segmentation network. For real mask;

[0099] Loss Dice The Dice loss function is expressed as:

[0100]

[0101] Loss CE The cross-entropy loss function is expressed as:

[0102]

[0103] M represents the number of categories; in a coarse segmentation network, M = 1, and in a fine segmentation network, M = 2; y i,c Let c be the true class of pixel i. When the true class of pixel i is c, y i,c =1, otherwise y i,c =0; p i,c Predict the probability that pixel i belongs to category c for a breast MRI segmentation network.

[0104] Finally, the multi-phase MRI images to be predicted are input into the trained breast MRI segmentation network for prediction. First, the multi-phase MRI images to be predicted are preprocessed.

[0105] Then, for the non-enhanced phase image in the preprocessed multi-phase MRI images to be predicted, it is directly put into the trained breast MRI segmentation network to obtain the breast mask of the multi-phase MRI images to be predicted. Then, the breast mask is multiplied with the preprocessed most significant phase image to obtain the breast region image of the most significant phase image in the multi-phase MRI images to be predicted.

[0106] For the most salient phase image in the preprocessed multi-phase MRI images to be predicted, due to the network training process focusing on the most salient phase image C... peak Tumor regions were segmented into smaller blocks, but for the multi-phase MRI images to be predicted, there were no tumor labels, and the location of the tumor was unknown. Therefore, this application uses a sliding window method to select image blocks from the most significant phase image in the preprocessed multi-phase MRI images to be predicted: First, the overlap rate of the sliding window is set, and then image blocks of the same size as the sliding window are selected starting from the upper left corner of the preprocessed most significant image. These blocks are then sequentially placed into the trained breast MRI segmentation network to obtain the tumor mask and gland mask of the multi-phase MRI images to be predicted. Since the sliding window method is used to process the preprocessed most significant phase image, i.e., one image is segmented into multiple image blocks, the obtained tumor mask and gland mask need to be concatenated. After concatenation, the complete tumor mask and gland mask of the most significant image are obtained. Finally, these are multiplied by the preprocessed most significant phase image to obtain the tumor region image and gland region image, respectively.

[0107] In this embodiment, the overlap rate of the sliding window is set to 0.7, and the size of the sliding window is 96×96×96.

[0108] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0109] Based on the same idea as the breast, tumor, and gland segmentation method for breast MRI in the above embodiments, the present invention also provides a breast, tumor, and gland segmentation system for breast MRI, which can be used to perform the above-described breast, tumor, and gland segmentation method for breast MRI. For ease of explanation, the schematic diagram of an embodiment of the breast, tumor, and gland segmentation system for breast MRI only shows the parts related to the embodiments of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0110] like Figure 5 As shown, another embodiment of the present invention provides a breast, tumor and gland segmentation system for breast MRI, including an image acquisition module, a preprocessing module, a network construction module, a network training module and a segmentation output module;

[0111] The image acquisition module is used to acquire multi-phase MRI images of patients based on dynamic contrast-enhanced magnetic resonance imaging; the multi-phase MRI images include non-contrast phase images and most salient phase images;

[0112] The preprocessing module is used to preprocess multi-phase MRI images to obtain processed images; preprocessing includes resampling, image registration, normalization, and resolution adjustment, etc.

[0113] The network construction module is used to build a breast MRI segmentation network based on the UNET structure, including a coarse segmentation network and a fine segmentation network. The coarse segmentation network and the fine segmentation network have the same network structure but different weight parameters. The coarse segmentation network is used to obtain the breast mask; the fine segmentation network is used to obtain the tumor mask and gland mask. Both the coarse segmentation network and the fine segmentation network include an encoder and a decoder. The encoder is used to extract image features, and the decoder is used to obtain the image mask.

[0114] The network training module is used to train the breast MRI segmentation network using the processed images to obtain a trained breast MRI segmentation network.

[0115] The segmentation output module is used to input the multi-phase MRI images to be predicted into the trained breast MRI segmentation network to obtain images of the breast region, tumor region, and glandular region of the multi-phase MRI images to be predicted.

[0116] The breast, tumor, and gland segmentation system for breast MRI of the present invention corresponds one-to-one with the breast, tumor, and gland segmentation method for breast MRI of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the breast, tumor, and gland segmentation method for breast MRI are applicable to the embodiment of the breast, tumor, and gland segmentation system for breast MRI. For details, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here.

[0117] Furthermore, in the above embodiment of a breast MRI breast, tumor, and gland segmentation system, the logical division of each program module is merely illustrative. In practical applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or software implementation convenience. That is, the internal structure of the breast, tumor, and gland segmentation system of the above-described breast MRI is divided into different program modules to complete all or part of the functions described above.

[0118] like Figure 6 As shown, in one embodiment, a computer-readable storage medium is provided, storing a program in a memory. When the program is executed by a processor, it implements the aforementioned method for segmenting the breast, tumor, and glandular tissue in breast MRI, specifically as follows:

[0119] The method of dynamic contrast-enhanced magnetic resonance imaging was used to acquire multi-phase MRI images of patients, including non-contrast phase images and most salient phase images;

[0120] Preprocessing is performed on multi-phase MRI images to obtain processed images; preprocessing includes resampling, image registration, normalization, and resolution adjustment;

[0121] A breast MRI segmentation network based on the UNET architecture is constructed, including a coarse segmentation network and a fine segmentation network. The coarse segmentation network and the fine segmentation network have the same network structure but different weight parameters. The coarse segmentation network is used to obtain the breast mask, and the fine segmentation network is used to obtain the tumor mask and gland mask. Both the coarse segmentation network and the fine segmentation network include an encoder and a decoder. The encoder is used to extract image features, and the decoder is used to obtain the image mask.

[0122] The processed images were used to train the breast MRI segmentation network, resulting in a well-trained breast MRI segmentation network.

[0123] The multi-phase MRI images to be predicted are input into the trained breast MRI segmentation network to obtain images of the breast region, tumor region, and glandular region of the multi-phase MRI images to be predicted.

[0124] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0125] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for segmenting the breast, tumor, and glandular tissue in breast MRI, characterized in that, Includes the following steps: The patient's MRI images were acquired using a dynamic contrast-enhanced magnetic resonance imaging method; the MRI images included non-contrast phase images and most salient phase images. The preprocessing of multi-phase MRI images yields processed images; the preprocessing includes resampling, image registration, normalization, and resolution adjustment. A breast MRI segmentation network is constructed based on the UNET architecture. The breast MRI segmentation network includes a coarse segmentation network and a fine segmentation network. The coarse segmentation network and the fine segmentation network have the same network structure but different weight parameters. The coarse segmentation network is used to obtain the breast mask. The fine segmentation network is used to obtain the tumor mask and the gland mask. Both the coarse segmentation network and the fine segmentation network include an encoder and a decoder. The encoder is used to extract image features, and the decoder is used to obtain the image mask. The encoder includes a convolutional network layer, N+1 hybrid layers, and N downsampling layers; the hybrid layers and downsampling layers are alternately connected in sequence; the convolutional network layer is connected to the first hybrid layer; the decoder includes N+1 feature fusion layers and N upsampling layers; the feature fusion layers and upsampling layers are alternately connected in sequence; the N+1th hybrid layer is connected to the first feature fusion layer; The hybrid layer consists of a multi-scale attention convolution (MSCA), a ResNet block, and a Swin Transformer. The Swin Transformer is connected to both the ResNet block and the multi-scale attention convolution (MSCA). The multi-scale attention convolution (MSCA) is used to extract local features of the input image. The ResNet block is used to extract global features of the input image. The Swin Transformer is used to summarize and fuse the local and global features to output the fused features of the input image. The processed images were used to train the breast MRI segmentation network, resulting in a well-trained breast MRI segmentation network. The process was as follows: The non-enhanced phase image in the processed image is input into the encoder of the coarse segmentation network to obtain the global field of view and gland information of the entire image to obtain the non-enhanced phase image features; The features of the non-enhanced image are fed into the decoder of the coarse segmentation network to segment out the breast mask of the non-enhanced image; The most salient phase image in the processed image is normalized and segmented into an image patch containing the tumor region with a set resolution based on the existing tumor label; Image patches are fed into the encoder of a fine segmentation network to extract image patch features; The non-enhanced image features and image patch features are fused and input into the decoder of the fine segmentation network to obtain the tumor mask and gland mask; The loss function is calculated based on the breast mask obtained from the coarse segmentation network, the tumor mask and gland mask obtained from the fine segmentation network, and the corresponding real mask. The parameters of the breast MRI segmentation network are then updated through backpropagation. The training continues iteratively until the breast MRI segmentation network converges or reaches the maximum number of iterations, resulting in a well-trained breast MRI segmentation network. The multi-phase MRI images to be predicted are input into the trained breast MRI segmentation network to obtain images of the breast region, tumor region, and glandular region of the multi-phase MRI images to be predicted.

2. The method for segmenting the breast, tumor, and glandular tissue in breast MRI according to claim 1, characterized in that, The preprocessing of multi-phase MRI images to obtain the processed images specifically involves: First, the multi-phase MRI images were resampled to a resolution of (1,1,1). Then, the deedsBCV image registration algorithm was used to register the most salient phase image in the multi-phase MRI images to the non-enhanced phase image. Standard normalization is performed on the non-enhanced images using the following formula: ， Where x is the pixel intensity value of the unenhanced image before standard normalization, x scaled σ represents the pixel intensity value of the non-enhanced image after standard normalization, μ represents the mean pixel intensity value of the non-enhanced image before standard normalization, and σ represents the standard deviation of the pixel intensity value of the non-enhanced image before standard normalization. The resolution of the non-enhanced phase image and the most salient phase image were adjusted to the specified size using cubic spline interpolation.

3. The method for segmenting the breast, tumor, and glandular tissue in breast MRI according to claim 1, characterized in that, The encoder consists of CNNs and ViT; the decoder consists of deconvolution. Both the convolution and deconvolution in the encoder and decoder are three-dimensional convolutions.

4. The method for segmenting the breast, tumor, and glandular tissue in breast MRI according to claim 1, characterized in that, The loss function of the breast MRI segmentation network consists of the Dice loss function and the cross-entropy loss function, expressed as follows: ， Among them, y mask Image mask predicted by a breast MRI segmentation network. For real mask; Loss Dice The Dice loss function is expressed as: ， Loss CE The cross-entropy loss function is expressed as: ， M represents the number of categories; M=1 in a coarse segmentation network and M=2 in a fine segmentation network; y i,c Let c be the true class of pixel i. When the true class of pixel i is c, y i,c =1, otherwise y i,c =0; p i,c Predict the probability that pixel i belongs to category c for a breast MRI segmentation network.

5. A method for segmenting the breast, tumor, and glandular tissue in breast MRI according to claim 1, characterized in that, The multi-phase MRI images to be predicted do not contain tumor labels; First, the multi-phase MRI images to be predicted are preprocessed; The non-enhanced phase image from the preprocessed multi-phase MRI images to be predicted is directly fed into the trained breast MRI segmentation network to obtain the breast mask of the multi-phase MRI images to be predicted. The breast mask is then multiplied with the preprocessed most salient phase image to obtain the breast region image of the most salient phase image in the multi-phase MRI images to be predicted. For the most significant phase image in the preprocessed multi-phase MRI images to be predicted, a sliding window method is used to select image blocks. First, the overlap rate of the sliding window is set, and then image blocks of the same size as the sliding window are selected starting from the upper left corner of the preprocessed most significant phase image. These blocks are then sequentially placed into the trained breast MRI segmentation network to obtain the tumor mask and gland mask of the multi-phase MRI images to be predicted. The obtained tumor mask and gland mask are then concatenated and multiplied with the preprocessed most significant phase image to obtain the tumor region image and gland region image, respectively.

6. A breast MRI system for segmenting the breast, tumor, and glandular tissue, characterized in that, A method for segmenting breast, tumor and glandular tissue in breast MRI according to any one of claims 1-5, comprising an image acquisition module, a preprocessing module, a network construction module, a network training module and a segmentation output module; The image acquisition module is used to acquire multi-phase MRI images of the patient based on dynamic contrast-enhanced magnetic resonance imaging; the multi-phase MRI images include non-contrast phase images and most salient phase images; The preprocessing module is used to preprocess multi-phase MRI images to obtain processed images; the preprocessing includes resampling, image registration, normalization, and resolution adjustment. The network construction module is used to construct a breast MRI segmentation network based on the UNET structure; the breast MRI segmentation network includes a coarse segmentation network and a fine segmentation network; the coarse segmentation network and the fine segmentation network have the same network structure but different weight parameters; the coarse segmentation network is used to obtain the breast mask; the fine segmentation network is used to obtain the tumor mask and the gland mask; both the coarse segmentation network and the fine segmentation network include an encoder and a decoder; wherein the encoder is used to extract image features, and the decoder is used to obtain the image mask; The network training module is used to train the breast MRI segmentation network using the processed images to obtain a trained breast MRI segmentation network. The segmentation output module is used to input the multi-phase MRI images to be predicted into the trained breast MRI segmentation network to obtain breast region images, tumor region images, and glandular region images of the multi-phase MRI images to be predicted.

7. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the breast, tumor and gland segmentation method of breast MRI as described in any one of claims 1-5.

Images