Breast lesion segmentation method and device based on dynamic contrast-enhanced magnetic resonance image

Abstract

The application discloses a breast lesion segmentation method and device based on dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), which comprises the following steps: after pre-processing the DCE-MRI image, the breast region is segmented first, and then the breast cancer lesion region is segmented based on the segmented breast region; when the breast cancer lesion region is segmented, a U-shaped network comprising two encoders and a decoder is adopted, the pre-processed contrast-enhanced late-phase image, the pre-processed contrast-enhanced early-phase image and the pre-processed breast prediction image are spliced respectively and input into an encoder for breast cancer lesion region segmentation to obtain a breast cancer prediction image, and an attention module is adopted between the two encoders to comprehensively process the contrast multi-period information. Compared with the prior art, the application can alleviate the class imbalance problem and exclude the interference of the regions outside the breast, and has high segmentation precision for the segmentation of the breast cancer lesion.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to a method and apparatus for segmenting breast lesions based on dynamic contrast-enhanced magnetic resonance imaging. Background Technology

[0002] Early detection and treatment can significantly improve the survival rate of breast cancer patients. Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) screening is generally recommended. DCE-MRI has high sensitivity and can provide information on breast lesions in terms of hemodynamics, morphology, and other aspects, which helps in the detection, localization, and characterization of lesions, and has been widely recognized by radiologists.

[0003] Artificial intelligence technology can greatly assist radiologists in analyzing DCE-MRI images, reducing their workload. Accurate segmentation of breast cancer areas is a crucial step in this process.

[0004] The location, size, and shape of breast cancer vary from patient to patient, and a standard DCE-MRI scan includes not only the entire breast tissue but also areas such as the thoracic cavity, sternum, and axilla, where the lesion volume is much smaller than the overall scan volume. Severe class imbalance exists when processing DCE-MRI images, and high-contrast areas such as blood vessels and organs also interfere with the segmentation of breast cancer regions, resulting in low segmentation accuracy of existing segmentation models.

[0005] Therefore, existing technologies need to be improved and enhanced. Summary of the Invention

[0006] The main objective of this invention is to provide a method, device, smart terminal, and computer-readable storage medium for segmenting breast lesions based on dynamic contrast-enhanced magnetic resonance imaging, aiming to solve the problem of low segmentation accuracy in existing segmentation models.

[0007] To achieve the above objectives, the present invention provides a method for segmenting breast lesions based on dynamic contrast-enhanced magnetic resonance imaging, the segmentation method comprising:

[0008] Perform the first preprocessing on the image before contrast enhancement;

[0009] The preprocessed contrast-enhanced image is input into the first network model for breast region segmentation to obtain a breast prediction map.

[0010] A second preprocessing was performed on the pre-contrast enhancement image, the post-contrast enhancement image, and the breast prediction map;

[0011] The preprocessed contrast-enhanced post-image and the preprocessed contrast-enhanced pre-image are respectively stitched together with the preprocessed breast cancer prediction map and then input into an encoder of the second network model for breast cancer lesion region segmentation to obtain the breast cancer prediction map. The second network model is based on a U-shaped network and has two encoders and one decoder. The corresponding encoding modules of the two encoders share parameters and have an attention module. The feature maps output by the two encoders and the feature maps output by the attention module are all skip connections to the decoder.

[0012] Optionally, the step of inputting the preprocessed contrast-enhanced pre-image into the first network model for breast region segmentation to obtain a breast prediction map includes:

[0013] The pre-processed contrast-enhanced image is divided into several patch blocks of a set size;

[0014] All the patch blocks are input into the first network model to segment the breast region, and the prediction result of each patch block is obtained;

[0015] All prediction results are merged and the size of the merged image is adjusted to be the same as the size of the pre-contrast enhancement image to obtain the breast prediction map.

[0016] Optionally, the steps of performing the first preprocessing and the second preprocessing on the image include:

[0017] Obtain the edge regions in the image where all voxel intensities are zero, and remove the edge regions from the image;

[0018] The excised image is resampled to reduce the image resolution;

[0019] The resampled image is standardized to obtain a preprocessed image.

[0020] Optionally, the resolution of the resampled image in the second preprocessing is greater than the resolution of the resampled image in the first preprocessing.

[0021] Optionally, the step of the attention module outputting the feature map includes:

[0022] Channel recalibration was performed on the feature maps of the later and earlier stages of contrast enhancement, respectively.

[0023] The difference feature map is obtained by subtracting the recalibrated early-contrast-enhanced feature map from the recalibrated late-contrast-enhanced feature map.

[0024] After concatenating the feature map of the later stage of contrast enhancement with the feature map of the earlier stage of contrast enhancement, channel recalibration is performed, followed by convolution operation to obtain the first weighted feature map corresponding to the feature map of the earlier stage of contrast enhancement and the second weighted feature map corresponding to the feature map of the later stage of contrast enhancement.

[0025] After standardizing the first weighted feature map and the second weighted feature map, multiply them by the feature map before contrast enhancement and the feature map after contrast enhancement, respectively, and calculate the weighted sum to obtain the summed feature map;

[0026] The difference feature map and the summation feature map are fused to obtain a fused feature map, which is then output.

[0027] Optionally, the steps for channel recalibrating the feature map include:

[0028] The feature map is input into a global average pooling layer to obtain a compressed feature map;

[0029] The compressed feature map is input into the fully connected layer to obtain the weighted feature map;

[0030] Multiply the weighted feature map with the feature map to obtain the channel-recalibrated feature map.

[0031] Optionally, the steps for training the second network model include:

[0032] The training samples of the dynamic contrast-enhanced magnetic resonance images are randomly sampled at a preset size to obtain the first patch block;

[0033] A region is defined based on the labels of the training samples, and random sampling of a preset size is performed within the region to obtain a second patch block;

[0034] The first patch block and the second patch block are input into the second network model for training, and the loss function used is a weighted sum of cross-entropy loss and Dice loss.

[0035] To achieve the above objectives, the present invention also provides a breast lesion segmentation device based on dynamic contrast-enhanced magnetic resonance imaging, the device comprising:

[0036] The preprocessing module is used to perform a first preprocessing on the pre-contrast enhancement image and a second preprocessing on the pre-contrast enhancement image, the post-contrast enhancement image, and the breast prediction map;

[0037] The breast region segmentation module is used to input the preprocessed contrast-enhanced pre-image into the first network model to segment the breast region and obtain a breast prediction map.

[0038] The breast cancer lesion region segmentation module is used to stitch the preprocessed contrast-enhanced post-image and the preprocessed contrast-enhanced pre-image with the preprocessed breast cancer prediction map, and then input each of them into an encoder of the second network model to segment the breast cancer lesion region and obtain the breast cancer prediction map. The second network model is based on a U-shaped network and has two encoders and one decoder. The corresponding encoding modules of the two encoders share parameters and have an attention module. The feature maps output by the two encoders and the feature maps output by the attention module are all skip-connected to the decoder.

[0039] To achieve the above objectives, the present invention also provides a smart terminal, which includes a memory, a processor, and a breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging stored in the memory and executable on the processor. When the breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging is executed by the processor, it implements any of the steps of the breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance imaging.

[0040] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging (MRI), wherein when the breast lesion segmentation program based on dynamic contrast-enhanced MRI is executed by a processor, it implements any of the steps of the aforementioned breast lesion segmentation method based on dynamic contrast-enhanced MRI.

[0041] As can be seen from the above, this invention preprocesses DCE-MRI images, first segmenting the breast region, and then segmenting the breast cancer lesion region based on the segmented breast region. This can alleviate the class imbalance problem and eliminate interference from regions outside the breast. When segmenting the breast cancer lesion region, a U-shaped network including two encoders and one decoder is used. The preprocessed contrast-enhanced post-image and the preprocessed contrast-enhanced pre-image are respectively stitched together with the preprocessed breast prediction map, and each is input into one encoder. An attention module is used between the two encoders to comprehensively compare information from multiple phases, further improving the segmentation accuracy of breast cancer lesions. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1This is a schematic diagram of the breast lesion segmentation framework based on dynamic contrast-enhanced magnetic resonance imaging provided by the present invention;

[0044] Figure 2 This is a flowchart illustrating an embodiment of the breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance imaging provided by the present invention.

[0045] Figure 3 yes Figure 2 A schematic diagram of the network architecture of the first network model in the embodiment;

[0046] Figure 4 yes Figure 2 A schematic diagram of the network architecture of the second network model in the embodiment;

[0047] Figure 5 yes Figure 2 A flowchart illustrating step S200 in the embodiment;

[0048] Figure 6 This is a flowchart illustrating an embodiment of the attention module output feature map provided by the present invention;

[0049] Figure 7 yes Figure 2 A schematic diagram of the channel recalibration framework in the embodiment;

[0050] Figure 8 yes Figure 6 A schematic diagram of the framework of the embodiment;

[0051] Figure 9 This is a schematic flowchart of an embodiment of the training model for breast cancer lesion segmentation provided by the present invention;

[0052] Figure 10 This is a schematic diagram of the breast lesion segmentation device based on dynamic contrast-enhanced magnetic resonance imaging provided in an embodiment of the present invention;

[0053] Figure 11 This is a block diagram illustrating the internal structure of a smart terminal provided in an embodiment of the present invention. Detailed Implementation

[0054] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0055] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0056] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0057] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0058] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0061] Dynamic contrast-enhanced magnetic resonance imaging (fMRI) is highly sensitive in the detection of breast cancer. It can provide information on breast lesions in terms of hemodynamics, morphology, and other aspects, which helps in the detection, localization, and characterization of lesions. It has been widely recognized by radiologists.

[0062] Computer vision recognition technology can greatly assist radiologists in analyzing DCE-MRI images, reducing their workload. Accurate segmentation of breast cancer is the first and crucial step.

[0063] Similar to general tumor segmentation tasks, the location, size, and shape of breast cancer vary from patient to patient. Furthermore, standard DCE-MRI scans include not only complete breast tissue but also areas such as the thoracic cavity, sternum, and axilla, where the lesion volume is much smaller than the overall scan volume. Severe class imbalance exists when processing DCE-MRI images, and high-contrast areas such as blood vessels and organs also interfere with breast cancer segmentation, resulting in low segmentation accuracy of existing models and their inability to accurately segment the lesion.

[0064] To improve the segmentation accuracy of breast cancer, such as Figure 1 As shown, this invention proposes a two-step segmentation framework. To mitigate class imbalance and eliminate interference from regions outside the breast, the first step involves segmenting the breast region. Since breast cancer lesions only appear in the breast region, and interfering information outside the breast region, such as blood vessels and organs, inevitably affects the performance of the breast cancer segmentation model, breast region segmentation is crucial. The second step segments the breast cancer lesions based on the segmented breast region. To fully utilize the multi-phase information from DCE-MRI, this invention also designs a novel attention module that comprehensively compares multi-phase information to improve the accuracy of the model in segmenting breast cancer lesions.

[0065] Exemplary methods

[0066] like Figure 2 As shown, this embodiment of the invention provides a method for segmenting breast lesions based on dynamic contrast-enhanced magnetic resonance imaging, deployed on a smart terminal. Specifically, the segmentation method includes the following steps:

[0067] Step S100: Perform the first preprocessing on the contrast enhancement image;

[0068] Specifically, dynamic contrast-enhanced MRI includes pre-contrast images and multi-phase post-contrast images. Generally, the sequence format for DCE-MRI is 1+5 or 1+7 phases, meaning one pre-contrast image and five or seven post-contrast images. When segmenting the breast region in the first step, using pre-contrast images is more effective. This is because in post-contrast images, after contrast enhancement, lesions may be excluded during breast segmentation due to insufficient model accuracy. This is more noticeable when the number of breast annotations is insufficient; sufficient data can avoid this problem. Furthermore, to improve segmentation efficiency and accuracy, pre-contrast images undergo initial preprocessing.

[0069] In this embodiment, the first preprocessing step includes: cropping, resampling, and standardization. Since the image data of dynamic contrast-enhanced MRI is in a three-dimensional format, to reduce computational overhead, regions with voxel intensities of 0 are cropped out, i.e., regions where all voxel intensity values ​​are 0 are removed. For example, the breast region is in the middle of the image, so the upper and lower parts of the image are cropped. Because DCE-MRI images generally have high resolutions (commonly 896×896×120, 704×704×160, 512×512×220, 336×336×300, etc.), resampling to a lower resolution (e.g., 256×256×128) can increase the receptive field of the breast segmentation model without increasing the number of parameters, and can improve the convergence speed and region segmentation accuracy during breast segmentation model training. During standardization, the average intensity value of the pre-contrast-enhanced image is subtracted from each voxel intensity value, and then divided by the standard deviation of the pre-contrast-enhanced image. Specifically, the mean intensity of the voxel intensity of all pixels in the pre-contrast enhancement image is calculated to obtain the average intensity value of the pre-contrast enhancement image, and the standard deviation of the voxel intensity of all pixels in the pre-contrast enhancement image is calculated to obtain the standard deviation of the pre-contrast enhancement image.

[0070] It should be noted that other commonly used techniques in this field can also be used to perform the first preprocessing on the contrast enhancement image.

[0071] Step S200: Input the preprocessed contrast-enhanced early image into the first network model for breast region segmentation to obtain a breast prediction map;

[0072] Specifically, for ease of description, the first network model will be referred to as the breast segmentation model below. The specific network architecture of the first network model is not limited; any network architecture capable of semantic segmentation of 3D images can be used. This embodiment adopts the 3D Unet architecture, most commonly used in the field of semantic segmentation, which includes an encoder and a decoder. The encoder consists of n encoding modules, and the decoder consists of n-1 decoding modules. n is generally between 4 and 6, and can be adjusted according to the segmentation accuracy requirements and computational resource consumption; in this embodiment, n is set to 5. Each encoding module includes two convolution, normalization, and activation function operations, doubling the number of channels of the input feature map after passing through the encoding module, and downsampling the size of the input features to half the original size through a convolution with a stride of 2. Each decoding module also includes two convolution, normalization, and activation function operations, upsampling the size of the input feature map to twice the original size, doubling the number of channels, and then concatenating the upsampled feature map with the corresponding encoder output feature map before performing the decoding operation. The network architecture of the first network model is as follows: Figure 3As shown, the preprocessed contrast-enhanced image is input into the breast segmentation model for breast region segmentation, and the feature map output by the decoder is the breast prediction map.

[0073] Step S300: Perform a second preprocessing on the pre-contrast enhancement image, the post-contrast enhancement image, and the breast prediction map;

[0074] Specifically, similar to the image data processing steps in the breast segmentation model, the second preprocessing also includes cropping, resampling, and standardization of the image data. The difference lies in the fact that, considering the generally small volume occupied by lesions, downsampling to a lower resolution during the second preprocessing could lead to a loss of detail in the segmented breast cancer lesions, or even complete loss of the target. To balance resolution and the model's receptive field, the resolution of the resampled image in the second preprocessing is higher than that in the first preprocessing. For example, the input image data—pre-contrast enhancement image, post-contrast enhancement image, and breast prediction image—is resampled to approximately 400×300×100 resolution. Since the breast prediction image output by the breast segmentation model has already been standardized, it is not necessary to perform standardization on the breast segmentation prediction image again during the second preprocessing.

[0075] Step S400: The preprocessed contrast-enhanced post-image and the preprocessed contrast-enhanced pre-image are respectively stitched together with the preprocessed breast cancer prediction map and then input into one encoder of the second network model for breast cancer lesion region segmentation to obtain the breast cancer prediction map. The second network model is based on a U-shaped network and has two encoders and one decoder. The corresponding encoding modules of the two encoders share parameters and have an attention module. The feature maps output by the two encoders and the feature maps output by the attention module are all skip connections to the decoder.

[0076] Specifically, for ease of description, the second network model will be referred to as the breast cancer lesion segmentation model below. The image data input to the breast cancer lesion segmentation model includes: pre-processed contrast-enhanced pre-image, pre-processed contrast-enhanced post-image, and pre-processed breast prediction map. Multiple-phase contrast-enhanced post-images can also be input.

[0077] The backbone network of the breast cancer lesion segmentation model is a U-shaped network, such as 3D-UNet or VNet. To simultaneously learn feature information from both pre-contrast-enhanced and post-contrast-enhanced images, this embodiment improves the encoder-decoder structure, as shown below. Figure 4As shown, there are two encoders and one decoder. Each encoder includes five encoding modules (the number of encoding modules can be adjusted according to segmentation accuracy requirements and computational resource consumption). The corresponding encoding modules of the two encoders share parameters. The purpose of sharing parameters is to ensure that the output feature maps of the two encoders have the same distribution and to reduce computational resource consumption. To better extract multi-phase information from DCE-MRI, an attention module is set between the corresponding encoding modules of the two encoders. This module extracts the spatial weights of the pre-contrast-enhanced image and the post-contrast-enhanced image, as well as the feature map after feature weight fusion. By using the attention module to compare multi-phase information, the segmentation accuracy is improved. The attention module can be constructed using attention mechanisms commonly used in this field.

[0078] In this embodiment, the post-contrast enhancement image and the breast cancer prediction image are stitched together, as are the pre-contrast enhancement image and the breast cancer prediction image. The stitched images are then input into one encoder of the breast cancer lesion segmentation model. The feature maps output by each encoding module in both encoders are downsampled and passed to the decoding module of the decoder. When the decoding module upsamples the input feature map, it performs skip connections with the feature maps output by the corresponding encoding modules and attention modules of the two encoders. In other words, the decoding module stitches the upsampled feature map with the corresponding feature maps output by the encoding and attention modules before proceeding to the next step. Finally, the decoder outputs the breast cancer prediction image. A description of upsampling and skip connections can be found in the corresponding descriptions within the breast cancer segmentation model.

[0079] As can be seen from the above, this embodiment preprocesses the DCE-MRI images, first segmenting the breast region and then segmenting the breast cancer lesion region, which can alleviate the class imbalance problem and eliminate interference from regions outside the breast. The breast prediction map is combined with the preprocessed contrast-enhanced post-image and the preprocessed contrast-enhanced pre-image to improve the segmentation performance of breast cancer lesions. The breast cancer lesion segmentation model is an improvement on the U-shaped network, employing two encoders to simultaneously process the image stitched together from the breast prediction map and the preprocessed contrast-enhanced post-image, as well as the image stitched together from the breast prediction map and the preprocessed contrast-enhanced post-image. Parameters are shared between the two encoders, and an attention module is used to comprehensively compare information from multiple phases, improving the segmentation accuracy of breast cancer lesions.

[0080] In one embodiment, such as Figure 5 As shown, the process of obtaining the breast prediction map in step S200 specifically includes the following steps:

[0081] Step S210: Divide the preprocessed contrast enhancement image into several patch blocks of a set size;

[0082] Step S220: Input all patch blocks into the first network model to segment the breast region and obtain the prediction result for each patch block;

[0083] Step S230: Merge all prediction results and adjust the size of the merged image to be the same as the size of the pre-contrast enhancement image to obtain a breast prediction map.

[0084] Specifically, considering that the pre-contrast enhancement image is in 3D format and limited by GPU memory, the pre-contrast enhancement image is divided into patch blocks during training of the breast segmentation model and during breast region segmentation by the model. This reduces computational resource consumption and improves computational efficiency. During training, several pre-defined fixed-size patch blocks are randomly selected from the pre-processed pre-contrast enhancement image. All patch blocks are input into the breast segmentation model to obtain the prediction result for each patch block. The loss function used during training is a weighted sum of cross-entropy loss and Dice loss. The training epochs... max The number of epochs depends on the training data, but is generally set to 1000. The optimizer for the breast segmentation model has no special requirements; Adam or SGD are generally acceptable. The initial learning rate (initial_lr) is set to 0.01, and the learning rate strategy is "poly LR". The learning rate is gradually reduced according to the training rounds.

[0085] When segmenting the breast region, the pre-contrast enhancement image is first preprocessed. Then, using a sliding window approach, patches of the same size as those used during training are selected, and predictions are performed on each patch sequentially. The step size of the sliding window is set to half the patch size. Finally, the prediction results of each patch are merged, and nearest neighbor interpolation is used to restore the size of the merged image to the same size as the pre-contrast enhancement image, thus obtaining the breast prediction map.

[0086] As shown above, dividing the pre-contrast enhancement image into patch blocks can reduce the computational resource consumption of the breast segmentation model and improve its efficiency.

[0087] In one embodiment, such as Figure 6 As shown, the specific steps for the attention module to output the feature map include:

[0088] Step S410: Perform channel recalibration on the feature map of the later stage of contrast enhancement and the feature map of the earlier stage of contrast enhancement, respectively;

[0089] Specifically, channel recalibration can learn the relationships between feature channels, transforming the importance of each feature channel to the target task into learnable parameters, and using the learned parameters to enhance useful feature channels and suppress feature channels that contribute little.

[0090] The specific steps for channel recalibration include:

[0091] Step S411: Input the feature map into the global average pooling layer to obtain the compressed feature map;

[0092] Step S412: Input the compressed feature map into the fully connected layer to obtain the weighted feature map;

[0093] Step S413: Multiply the weighted feature map with the feature map to obtain the feature map after channel recalibration.

[0094] Specifically, such as Figure 7 As shown, channel recalibration includes compression, activation, and multiplication. The compression part uses a global average pooling operation to compress the input feature map along the channel dimension. If the original feature map has a dimension of HWC, where H is height, W is width, and C is the number of channels, compression reduces HWC to 11C, effectively compressing HW into one dimension. After compression, this dimension gains the global receptive field of the previous H*W, resulting in a wider receptive field. The compressed feature map is then input into a fully connected layer to predict the importance of each channel, obtaining weighted feature maps representing the importance of different channels. Finally, a multiplication operation applies (activates) these weights to the corresponding channels of the original feature map. The activation output weights represent the importance of each feature channel after feature selection, and are then multiplied channel-wise and weighted onto the input feature map, completing the recalibration of the original features along the channel dimension.

[0095] Channel recalibration can increase the channel weights in the feature map that are useful for segmentation and decrease the channel weights that are not very useful for segmentation, making the feature map more discriminative.

[0096] Step S420: Subtract the recalibrated pre-contrast enhancement feature map from the recalibrated post-contrast enhancement feature map to obtain the difference feature map;

[0097] Step S430: After concatenating the feature map of the later stage of contrast enhancement with the feature map of the earlier stage of contrast enhancement, channel recalibration is performed, and then convolution operation is performed to obtain the first weighted feature map corresponding to the feature map of the earlier stage of contrast enhancement and the second weighted feature map corresponding to the feature map of the later stage of contrast enhancement.

[0098] Step S440: Standardize the first weighted feature map and the second weighted feature map, multiply them by the feature map before contrast enhancement and the feature map after contrast enhancement respectively, and calculate the weighted sum to obtain the summed feature map;

[0099] Step S450: Fuse the difference feature map and the summation feature map to obtain the fused feature map and output it.

[0100] Specifically, the attention module is applied between each corresponding encoding module of the two encoders, mainly including three steps: calculating the difference feature map, calculating the summation feature map, and calculating the fusion feature map. For example... Figure 8 As shown, firstly, the feature maps of the later and earlier stages of contrast enhancement are recalibrated. Then, the recalibrated earlier and later stages of contrast enhancement feature maps are subtracted to explicitly calculate the difference feature map. The difference between the later and earlier stages of contrast enhancement is used to generate the subtraction image. Next, the later and earlier stages of contrast enhancement images are stitched together, recalibrated, and then subjected to two convolution operations (1*1*1 kernel size, 1 output channel) to obtain two weighted feature maps, representing the spatial weights of the earlier and later stages of contrast enhancement feature maps, respectively. Finally, the two weighted feature maps are standardized using a softmax function, and a weighted sum is calculated with the input earlier and later stages of contrast enhancement feature maps to obtain a summed feature map. This step is used to comprehensively compare multiple stages of DCE-MRI images. Finally, the difference feature map and the summed feature map are added to obtain a fused feature map. The fused feature map has stronger discriminative power for breast cancer lesions, helping the breast cancer lesion segmentation model to segment lesions more accurately.

[0101] As described above, this embodiment employs a novel attention module that can fully leverage multi-phase information from DCE-MRI, resulting in higher segmentation accuracy for the breast cancer lesion segmentation model.

[0102] In one embodiment, such as Figure 9 As shown, the steps for training a breast cancer lesion segmentation model include:

[0103] Step A100: Randomly sample the training samples of the dynamic contrast-enhanced magnetic resonance images to obtain the first patch block;

[0104] Step A200: Delineate the region based on the labels of the training samples, and perform random sampling of a preset size within the region to obtain the second patch block;

[0105] Step A300: Input the first patch block and the second patch block into the breast cancer lesion segmentation model for training. The loss function used is a weighted sum of cross-entropy loss and Dice loss.

[0106] Specifically, the training steps for the breast cancer lesion segmentation model are basically the same as those for the breast segmentation model. However, because the volume of a breast cancer lesion is much smaller than the volume of the area scanned by DCE-MRI, most of the randomly selected patches in the segmentation model are negative samples (not containing lesions), resulting in class imbalance. This makes it difficult for the segmentation model to converge to optimal performance during training. Therefore, this embodiment adopts an oversampling strategy: two patches are selected in each iteration. The training samples of the dynamic contrast-enhanced MRI image are randomly sampled at a preset size to obtain the first patch. Then, a small region is defined according to the label of the training sample, and patches are randomly sampled within this region to obtain the second patch. The first and second patches are input as a mini-batch into the breast cancer lesion segmentation model for training. This ensures that the patches input to the breast cancer lesion segmentation model contain lesion information, thereby solving the class imbalance problem. This strategy can improve the convergence speed and segmentation performance of the model training. The learning rate, optimizer, etc., used during training are the same as those used when training the breast segmentation model; please refer to the corresponding description in step S230.

[0107] As described above, by first randomly dividing patch blocks globally, and then randomly dividing patch blocks within the region containing the label according to the label, the patch blocks input during training are guaranteed to contain lesion information, thus solving the class imbalance problem.

[0108] Exemplary device

[0109] like Figure 10 As shown, corresponding to the breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance imaging, this embodiment of the invention also provides a breast lesion segmentation device based on dynamic contrast-enhanced magnetic resonance imaging. Specifically, the device includes:

[0110] The preprocessing module 600 is used to perform a first preprocessing on the pre-contrast enhancement image and a second preprocessing on the pre-contrast enhancement image, the post-contrast enhancement image, and the breast prediction map.

[0111] The breast region segmentation module 610 is used to input the preprocessed contrast-enhanced pre-image into the first network model for breast region segmentation to obtain a breast prediction map.

[0112] The breast cancer lesion region segmentation module 620 is used to stitch the preprocessed contrast-enhanced post-image and the preprocessed contrast-enhanced pre-image with the preprocessed breast cancer prediction map and then input them into an encoder of the second network model to segment the breast cancer lesion region and obtain the breast cancer prediction map. The second network model is based on a U-shaped network and has two encoders and one decoder. The corresponding encoding modules of the two encoders share parameters and have an attention module. The feature maps output by the two encoders and the feature maps output by the attention module are all skip-connected to the decoder.

[0113] In this embodiment, the breast lesion segmentation device based on dynamic contrast-enhanced magnetic resonance imaging described above can be referred to the corresponding description in the breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance imaging, and will not be repeated here.

[0114] Based on the above embodiments, the present invention also provides a smart terminal, the principle block diagram of which can be as follows: Figure 11 As shown. The aforementioned intelligent terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging (MRI). The internal memory provides an environment for the operation of the operating system and the breast lesion segmentation program based on MRI in the non-volatile storage medium. The network interface of the intelligent terminal is used for communication with external terminals via a network connection. When the processor executes the breast lesion segmentation program based on MRI, it implements the steps of any of the aforementioned breast lesion segmentation methods based on MRI. The display screen of the intelligent terminal can be a liquid crystal display (LCD) or an e-ink display.

[0115] Those skilled in the art will understand that Figure 11 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the smart terminal to which the present invention is applied. A specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0116] In one embodiment, a smart terminal is provided, the smart terminal including a memory, a processor, and a breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging stored in the memory and executable on the processor. When the breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging is executed by the processor, it performs the following operation instructions:

[0117] Perform the first preprocessing on the image before contrast enhancement;

[0118] The preprocessed contrast-enhanced image is input into the first network model for breast region segmentation to obtain a breast prediction map.

[0119] A second preprocessing was performed on the pre-contrast enhancement image, the post-contrast enhancement image, and the breast prediction map;

[0120] The preprocessed contrast-enhanced post-image and the preprocessed contrast-enhanced pre-image are respectively stitched together with the preprocessed breast cancer prediction map and then input into an encoder of the second network model for breast cancer lesion region segmentation to obtain the breast cancer prediction map. The second network model is based on a U-shaped network and has two encoders and one decoder. The corresponding encoding modules of the two encoders share parameters and have an attention module. The feature maps output by the two encoders and the feature maps output by the attention module are all skip connections to the decoder.

[0121] Optionally, the step of inputting the preprocessed contrast-enhanced pre-image into the first network model for breast region segmentation to obtain a breast prediction map includes:

[0122] The pre-processed contrast-enhanced image is divided into several patch blocks of a set size;

[0123] All the patch blocks are input into the first network model to segment the breast region, and the prediction result of each patch block is obtained;

[0124] All prediction results are merged and the size of the merged image is adjusted to be the same as the size of the pre-contrast enhancement image to obtain the breast prediction map.

[0125] Optionally, the steps of performing the first preprocessing and the second preprocessing on the image include:

[0126] Obtain the edge regions in the image where all voxel intensities are zero, and remove the edge regions from the image;

[0127] The excised image is resampled to reduce the image resolution;

[0128] The resampled image is standardized to obtain a preprocessed image.

[0129] Optionally, the resolution of the resampled image in the second preprocessing is greater than the resolution of the resampled image in the first preprocessing.

[0130] Optionally, the step of the attention module outputting the feature map includes:

[0131] Channel recalibration was performed on the feature maps of the later and earlier stages of contrast enhancement, respectively.

[0132] The difference feature map is obtained by subtracting the recalibrated early-contrast-enhanced feature map from the recalibrated late-contrast-enhanced feature map.

[0133] After concatenating the feature map of the later stage of contrast enhancement with the feature map of the earlier stage of contrast enhancement, channel recalibration is performed, followed by convolution operation to obtain the first weighted feature map corresponding to the feature map of the earlier stage of contrast enhancement and the second weighted feature map corresponding to the feature map of the later stage of contrast enhancement.

[0134] After standardizing the first weighted feature map and the second weighted feature map, multiply them by the feature map before contrast enhancement and the feature map after contrast enhancement, respectively, and calculate the weighted sum to obtain the summed feature map;

[0135] The difference feature map and the summation feature map are fused to obtain a fused feature map, which is then output.

[0136] Optionally, the steps for channel recalibrating the feature map include:

[0137] The feature map is input into a global average pooling layer to obtain a compressed feature map;

[0138] The compressed feature map is input into the fully connected layer to obtain the weighted feature map;

[0139] Multiply the weighted feature map with the feature map to obtain the channel-recalibrated feature map.

[0140] Optionally, the steps for training the second network model include:

[0141] The training samples of the dynamic contrast-enhanced magnetic resonance images are randomly sampled at a preset size to obtain the first patch block;

[0142] A region is defined based on the labels of the training samples, and random sampling of a preset size is performed within the region to obtain a second patch block;

[0143] The first patch block and the second patch block are input into the second network model for training, and the loss function used is a weighted sum of cross-entropy loss and Dice loss.

[0144] This invention also provides a computer-readable storage medium storing a breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging. When the breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance imaging is executed by a processor, it implements the steps of any of the breast lesion segmentation methods based on dynamic contrast-enhanced magnetic resonance imaging provided in this invention.

[0145] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0147] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0149] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of the above modules or units is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0150] If the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0151] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not mean that the essence of the corresponding technical solutions deviates from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for breast lesion segmentation based on dynamic contrast-enhanced magnetic resonance images, characterized in that, The segmentation method comprises: performing first preprocessing on the pre-contrast-enhanced image; inputting the pre-processed pre-contrast-enhanced image into a first network model to segment the breast region and obtain a breast prediction map; performing second preprocessing on the pre-contrast-enhanced image, the post-contrast-enhanced image and the breast prediction map; splicing the pre-processed post-contrast-enhanced image, the pre-processed pre-contrast-enhanced image and the pre-processed breast prediction map respectively, and then inputting them into an encoder of a second network model to segment the breast cancer lesion region and obtain a breast cancer prediction map, wherein the second network model is established based on a U-shaped network, two encoders and a decoder are arranged in the second network model, the parameter sharing modules corresponding to the two encoders are shared and provided with an attention module, and the feature maps output by the two encoders and the feature map output by the attention module are all connected to the decoder in a skip connection mode; the step of outputting the feature map by the attention module comprises: respectively performing channel re-labeling on the post-contrast-enhanced feature map and the pre-contrast-enhanced feature map; subtracting the re-labeled pre-contrast-enhanced feature map from the re-labeled post-contrast-enhanced feature map to obtain a difference feature map; splicing the post-contrast-enhanced feature map and the pre-contrast-enhanced feature map, performing channel re-labeling and then performing convolution to obtain a first weight feature map corresponding to the pre-contrast-enhanced feature map and a second weight feature map corresponding to the post-contrast-enhanced feature map; standardizing the first weight feature map and the second weight feature map, multiplying them with the pre-contrast-enhanced feature map and the post-contrast-enhanced feature map respectively, calculating the weighted sum to obtain a sum feature map; fusing the difference feature map and the sum feature map to obtain a fusion feature map and outputting the fusion feature map; the step of training the second network model comprises: performing random sampling of a preset size on the training sample of the dynamic contrast-enhanced magnetic resonance image to obtain a first patch block; performing random sampling of a preset size in a region demarcated according to the label of the training sample to obtain a second patch block; inputting the first patch block and the second patch block into the second network model for training, and using a weighted sum of cross-entropy loss and Dice loss as the loss function.

2. The breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance images of claim 1, wherein, The step of inputting the pre-processed pre-contrast-enhanced image into the first network model to segment the breast region and obtain the breast prediction map comprises: dividing the pre-processed pre-contrast-enhanced image into a plurality of patch blocks of a preset size; inputting all the patch blocks into the first network model to segment the breast region and obtain the prediction result of each patch block; merging all the prediction results and adjusting the size of the merged image to the same size as the pre-contrast-enhanced image to obtain the breast prediction map.

3. The breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance images of claim 1, wherein, The steps of performing the first preprocessing and the second preprocessing on the image comprise: cutting out an edge region in which the intensity of all voxels is zero from the image; resampling the cut-out image to reduce the resolution of the image; performing standardization processing on the resampled image to obtain the pre-processed image.

4. The breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance images of claim 3, wherein, The resolution of the second pre-processed image after resampling is greater than the resolution of the first pre-processed image after resampling.

5. The breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance images of claim 1, wherein, The step of performing channel recalibration on the feature map comprises: inputting the feature map into a global average pooling layer to obtain a compressed feature map; inputting the compressed feature map into a fully connected layer to obtain a weight feature map; multiplying the weight feature map with the feature map to obtain a channel recalibrated feature map.

6. An apparatus for breast lesion segmentation based on dynamic contrast-enhanced magnetic resonance images, characterized by The device is used to implement the steps of the breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance images according to any one of claims 1-5, and the device comprises: a preprocessing module configured to perform first preprocessing on the pre-contrast-enhanced image and second preprocessing on the pre-contrast-enhanced image, the post-contrast-enhanced image, and the breast prediction map; a breast region segmentation module configured to input the pre-processed pre-contrast-enhanced image into a first network model to perform breast region segmentation and obtain a breast prediction map; a breast cancer lesion region segmentation module configured to splice the pre-processed post-contrast-enhanced image, the pre-processed pre-contrast-enhanced image, and the pre-processed breast prediction map respectively, and then input them into an encoder of a second network model to perform breast cancer lesion region segmentation and obtain a breast cancer prediction map, wherein the second network model is established based on a U-shaped network, the second network model comprises two encoders and a decoder, the parameter sharing and attention modules are arranged between the corresponding encoding modules of the two encoders, and the feature maps output by the two encoders and the feature map output by the attention module are all connected to the decoder in a skip connection manner.

7. The intelligent terminal, characterized by The intelligent terminal comprises a memory, a processor, and a breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance images stored on the memory and executable on the processor, and the breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance images is executed by the processor to implement the steps of the breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance images according to any one of claims 1-5.

8. A computer readable storage medium, characterized in that, The computer readable storage medium stores a breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance images, and the breast lesion segmentation program based on dynamic contrast-enhanced magnetic resonance images is executed by the processor to implement the steps of the breast lesion segmentation method based on dynamic contrast-enhanced magnetic resonance images according to any one of claims 1-5.

Images