Image segmentation method, computer device and storage medium

Abstract

The present application relates to an image segmentation method, a computer device, and a storage medium. The method includes: performing a blocking process on a fat-suppressed magnetic resonance image, and performing a contrast-limited adaptive histogram equalization process on each image block as a unit to obtain a processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes a region of interest; inputting the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtaining a mask image of the region of interest through the segmentation network; and obtaining a segmentation result of the region of interest in the fat-suppressed magnetic resonance image according to the fat-suppressed magnetic resonance image and the mask image. By using this method, the segmentation result of the region of interest in the fat-suppressed magnetic resonance image can be accurately obtained.

Description

Technical Field

[0001] This application relates to the field of medical image technology, and particularly to an image segmentation method, a computer device, and a storage medium. Background Art

[0002] The segmentation of the lesion area in medical images is of great significance in a computer-aided diagnosis system. Through the computer-aided diagnosis system, the lesions segmented in medical images can be accurately detected. For example, the early detection and diagnosis of breast cancer can effectively improve the cure rate of breast cancer. In a computer-aided diagnosis system based on three-dimensional magnetic resonance images, the segmentation of breast tissue and glandular tissue is of great significance.

[0003] Currently, most segmentation methods are based on non-fat-suppressed magnetic resonance images to achieve the segmentation of tissues and lesions, and it is difficult to accurately segment tissues and lesions in fat-suppressed magnetic resonance images. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide an image segmentation method, a computer device, and a storage medium that can accurately segment tissues and lesions in fat-suppressed magnetic resonance images.

[0005] An image segmentation method, the method includes:

[0006] Perform a block processing on the fat-suppressed magnetic resonance image, and perform a limited contrast adaptive histogram equalization processing on each image block as a unit to obtain a processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes a region of interest;

[0007] Input the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtain a mask image of the region of interest through the segmentation network;

[0008] According to the fat-suppressed magnetic resonance image and the mask image, obtain a segmentation result of the region of interest in the fat-suppressed magnetic resonance image.

[0009] In one embodiment, the segmentation network includes: an encoding unit and a decoding unit; the step of inputting the processed fat-suppressed magnetic resonance image into a preset segmentation network and obtaining a mask image of the region of interest through the segmentation network includes:

[0010] Input the processed fat-suppressed magnetic resonance image into the encoding unit, perform boundary padding on the processed fat-suppressed magnetic resonance image by using padding convolution, and perform downsampling on the fat-suppressed magnetic resonance image after boundary padding to obtain a downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image;

[0011] Input the downsampled feature map into the decoding unit, perform boundary padding on the downsampled feature map using the padding convolution, and upsample the downsampled feature map after boundary padding to obtain the mask image of the region of interest.

[0012] In one embodiment, the encoding unit includes a plurality of convolutional blocks, each convolutional block is composed of a convolutional layer and a pooling layer, and each convolutional layer is connected to the normalization layer and the activation function layer of the segmentation network; the convolutional layer includes the padding convolution; the padding convolution is used to perform boundary padding on the fat-suppressed magnetic resonance image and increase the size of the image matrix corresponding to the fat-suppressed magnetic resonance image; the decoding unit includes a plurality of deconvolutional blocks, each deconvolutional block is composed of a convolutional layer and a deconvolutional layer, and each convolutional layer is connected to the normalization layer and the activation function layer; wherein, the number of convolutional blocks is the same as the number of deconvolutional blocks; the function adopted by the normalization layer is the instance normalization function; the function adopted by the activation function layer is the leaky rectified linear unit function.

[0013] In one embodiment, the segmentation network further includes a connection unit; the encoding unit is connected to the decoding unit through the connection unit; the step of inputting the downsampled feature map into the decoding unit and upsamping the downsampled feature map to obtain the mask image of the region of interest includes:

[0014] Connect the downsampled feature map output by the first convolutional block and the upsampled feature map output by the first deconvolutional block through the connection unit to obtain the connected feature map; the first deconvolutional block is the deconvolutional block corresponding to the first convolutional block;

[0015] Input the connected feature map into the second deconvolutional block for upsampling to obtain the mask image of the region of interest; the second deconvolutional block is the next deconvolutional block adjacent to the first deconvolutional block.

[0016] In one embodiment, the number of convolutional blocks and deconvolutional blocks is determined according to the size of the fat-suppressed magnetic resonance image, the size of the upsampled feature map, and the size of the downsampled feature map.

[0017] In one embodiment, before performing block processing on the fat-suppressed magnetic resonance image and performing contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image, the method further includes:

[0018] Resample the fat-suppressed magnetic resonance image according to the size of the fat-suppressed magnetic resonance image and the spacing between voxels of the fat-suppressed magnetic resonance image to obtain a resampled magnetic resonance image.

[0019] In one embodiment, the training process of the segmentation network includes:

[0020] Obtain a sample fat-suppressed magnetic resonance image and a gold standard image corresponding to the sample fat-suppressed magnetic resonance image; the sample fat-suppressed magnetic resonance image includes a sample region of interest.

[0021] Perform a blocking process on the sample fat-suppressed magnetic resonance image, and perform contrast-limited adaptive histogram equalization processing on each sample image block to obtain a processed sample fat-suppressed magnetic resonance image.

[0022] Input the processed sample fat-suppressed magnetic resonance image into an initial segmentation network, and obtain a sample mask image of the sample region of interest through the initial segmentation network.

[0023] According to the sample mask image and the gold standard image, obtain the value of the loss function of the initial segmentation network; the loss function of the initial segmentation network includes a Dice loss function and a cross-entropy loss function; the value of the loss function of the initial segmentation network is the sum of the value of the Dice loss function and the value of the cross-entropy loss function.

[0024] Train the initial segmentation network according to the value of the loss function of the initial segmentation network to obtain the segmentation network.

[0025] An image segmentation method, the method includes:

[0026] Perform a blocking process on a fat-suppressed breast image containing a breast, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain a processed fat-suppressed breast image.

[0027] Input the processed fat-suppressed breast image into a preset breast segmentation network, and obtain a breast mask image through the breast segmentation network.

[0028] Obtain a breast segmentation image according to the breast mask image and the fat-suppressed breast image.

[0029] Input the breast segmentation image into a preset gland segmentation network, and obtain a gland mask image through the gland segmentation network.

[0030] Obtain a gland segmentation image according to the gland mask image and the fat-suppressed breast image, or according to the gland mask image and the breast segmentation image;

[0031] Obtain the breast density corresponding to the fat-suppressed breast image according to the breast segmentation image and the gland segmentation image.

[0032] An image segmentation device, the device includes:

[0033] A first acquisition module, configured to perform block processing on a fat-suppressed magnetic resonance image, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain a processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes a region of interest;

[0034] A first segmentation module, configured to input the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtain a mask image of the region of interest through the segmentation network;

[0035] A second acquisition module, configured to obtain a segmentation result of the region of interest in the fat-suppressed magnetic resonance image according to the fat-suppressed magnetic resonance image and the mask image.

[0036] An image segmentation device, the device includes:

[0037] A first acquisition module, configured to perform block processing on a fat-suppressed breast image including a breast, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain a processed fat-suppressed breast image;

[0038] A first segmentation module, configured to input the processed fat-suppressed breast image into a preset breast segmentation network, and obtain a breast mask image through the breast segmentation network;

[0039] A second acquisition module, configured to obtain a breast segmentation image according to the breast mask image and the fat-suppressed breast image;

[0040] A second segmentation module, configured to input the breast segmentation image into a preset gland segmentation network, and obtain a gland mask image through the gland segmentation network;

[0041] A third acquisition module, configured to obtain a gland segmentation image according to the gland mask image and the fat-suppressed breast image, or according to the gland mask image and the breast segmentation image;

[0042] A fourth acquisition module, configured to obtain the breast density corresponding to the fat-suppressed breast image according to the breast segmentation image and the gland segmentation image.

[0043] A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0044] Perform block processing on the fat-suppressed magnetic resonance image, and perform limited contrast adaptive histogram equalization processing for each image block to obtain the processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes a region of interest;

[0045] Input the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtain a mask image of the region of interest through the segmentation network;

[0046] According to the fat-suppressed magnetic resonance image and the mask image, obtain the segmentation result of the region of interest in the fat-suppressed magnetic resonance image.

[0047] A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0048] Perform block processing on the fat-suppressed breast image containing the breast, and perform limited contrast adaptive histogram equalization processing for each image block to obtain the processed fat-suppressed breast image;

[0049] Input the processed fat-suppressed breast image into a preset breast segmentation network, and obtain a breast mask image through the breast segmentation network;

[0050] According to the breast mask image and the fat-suppressed breast image, obtain a breast segmentation image;

[0051] Input the breast segmentation image into a preset gland segmentation network, and obtain a gland mask image through the gland segmentation network;

[0052] According to the gland mask image and the fat-suppressed breast image, or according to the gland mask image and the breast segmentation image, obtain a gland segmentation image;

[0053] According to the breast segmentation image and the gland segmentation image, obtain the breast density corresponding to the fat-suppressed breast image.

[0054] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0055] The fat-suppressed magnetic resonance images are segmented, and contrast-limited adaptive histogram equalization processing is performed on each image block as a unit to obtain the processed fat-suppressed magnetic resonance images; the fat-suppressed magnetic resonance images include regions of interest;

[0056] The processed fat-suppressed magnetic resonance images are input into a preset segmentation network, and a mask image of the region of interest is obtained through the segmentation network;

[0057] According to the fat-suppressed magnetic resonance images and the mask images, a segmentation result of the region of interest in the fat-suppressed magnetic resonance images is obtained.

[0058] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the following steps are implemented:

[0059] The fat-suppressed breast images containing breasts are segmented, and contrast-limited adaptive histogram equalization processing is performed on each image block as a unit to obtain the processed fat-suppressed breast images;

[0060] The processed fat-suppressed breast images are input into a preset breast segmentation network, and a breast mask image is obtained through the breast segmentation network;

[0061] According to the breast mask images and the fat-suppressed breast images, breast segmentation images are obtained;

[0062] The breast segmentation images are input into a preset gland segmentation network, and gland mask images are obtained through the gland segmentation network;

[0063] According to the gland mask images and the fat-suppressed breast images, or according to the gland mask images and the breast segmentation images, gland segmentation images are obtained;

[0064] According to the breast segmentation images and the gland segmentation images, the breast density corresponding to the fat-suppressed breast images is obtained.

[0065] The above image segmentation method, device, computer device, and storage medium can perform block processing on the fat-suppressed magnetic resonance image, and can perform limited contrast adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image. Since the limited contrast adaptive histogram equalization processing is performed, the contrast of the fat-suppressed magnetic resonance image can be enhanced, and the noise points of the fat-suppressed magnetic resonance image can be reduced. In this way, when the processed fat-suppressed magnetic resonance image is input into a preset segmentation network, the segmentation network can accurately segment the processed fat-suppressed magnetic resonance image, and accurately obtain the mask image of the region of interest in the fat-suppressed magnetic resonance image, improving the accuracy of obtaining the mask image of the region of interest. Therefore, according to the fat-suppressed magnetic resonance image and the obtained mask image of the region of interest, the segmentation result of the region of interest in the fat-suppressed magnetic resonance image can be accurately obtained. Description of the Drawings

[0066] Figure 1 It is an application environment diagram of the image segmentation method in an embodiment;

[0067] Figure 2 It is a schematic flowchart of the image segmentation method in an embodiment;

[0068] Figure 2a It is a schematic diagram of image comparison of the image preprocessed by CLAHE in an embodiment;

[0069] Figure 3 It is a schematic flowchart of the image segmentation method in an embodiment;

[0070] Figure 3a It is a schematic structural diagram of the segmentation network in an embodiment;

[0071] Figure 4 It is a schematic flowchart of the image segmentation method in an embodiment;

[0072] Figure 5 It is a schematic flowchart of the image segmentation method in an embodiment;

[0073] Figure 6 It is a schematic flowchart of the image segmentation method in an embodiment;

[0074] Figure 6a It is a schematic diagram of the segmentation method of breast fat-suppressed mammary gland images in an embodiment;

[0075] Figure 7 It is a schematic diagram of the segmentation results of the breast and gland in an embodiment;

[0076] Figure 8 It is a schematic diagram of the correlation between the segmentation result volume and the gold standard volume in an embodiment;

[0077] Figure 9 is a structural block diagram of an image segmentation device in an embodiment;

[0078] Figure 10 is a structural block diagram of an image segmentation device in an embodiment. Specific embodiments

[0079] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0080] The image segmentation method provided by the embodiments of the present application can be applicable to a Figure 1 computer device as shown. The computer device includes a processor and a memory connected through a system bus. A computer program is stored in the memory. When the processor executes the computer program, it can execute the steps of the following method embodiments. Optionally, the computer device may further include a network interface, a display screen, and an input device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. Optionally, the computer device may be a server, a personal computer, a personal digital assistant, or other terminal devices, such as a tablet computer, a mobile phone, etc., or may be a cloud or a remote server. The embodiments of the present application do not limit the specific form of the computer device.

[0081] Currently, for the segmentation of breast and glandular tissues in three-dimensional magnetic resonance images, both manual segmentation and semi-automatic segmentation methods that require user assistance are relatively complicated, have low segmentation efficiency, and there are large differences between observers and within observers themselves. In the segmentation methods based on deep learning, the segmentation of tissue and lesion regions depends on manual annotation by doctors, which is time-consuming and labor-intensive. Moreover, currently, most methods are based on non-fat-suppressed magnetic resonance images to achieve tissue and lesion segmentation. During the clinical breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) scan, the images before and after injection of the contrast agent are usually obtained using fat-suppression imaging technology. The obtained fat-suppressed magnetic resonance images have a high noise level, low contrast, and uneven fat suppression. Traditional segmentation methods cannot accurately segment tissues and lesions in fat-suppressed magnetic resonance images. Therefore, it is necessary to provide an image segmentation method to accurately segment tissues and lesions in fat-suppressed magnetic resonance images.

[0082] In one embodiment, as Figure 2 shown, an image segmentation method is provided. Taking the computer device in Figure 1 as an example, the method includes the following steps:

[0083] S201. Perform block processing on the fat-suppressed magnetic resonance image, and perform contrast-limited adaptive histogram equalization processing on each image block as a unit to obtain the processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes the region of interest.

[0084] Among them, the fat-suppressed magnetic resonance image is an image obtained by injecting a contrast agent into the scanned object and suppressing the fat in the scanned part of the scanned object. It should be noted that the fat-suppressed magnetic resonance image in this application is a fat-suppressed dynamic enhanced magnetic resonance image.

[0085] Specifically, the computer device performs block processing on the obtained fat-suppressed magnetic resonance image including the region of interest, and performs contrast-limited adaptive histogram equalization (CLAHE) processing on each of the divided image blocks as a unit to obtain the processed fat-suppressed magnetic resonance image. Here, it should be noted that performing contrast-limited adaptive histogram equalization processing on the obtained fat-suppressed magnetic resonance image in units of blocks is a preprocessing operation added in view of the relatively high noise level, low intensity contrast, and uneven fat suppression of the fat-suppressed magnetic resonance image. Optionally, the computer device can perform block processing on the fat-suppressed magnetic resonance image, perform histogram equalization on each image block as a unit, and then use linear interpolation to determine the voxel values between blocks, so as to enhance the contrast and suppress the noise at the same time, thereby improving the accuracy of segmentation. Exemplarily, the comparison diagram of the image processed by CLAHE can be as Figure 2a shown, and it can be seen from Figure 2a that the contrast of the image processed by CLAHE is enhanced and the noise points are reduced. Optionally, the region of interest can be the breast, or the region of the scanned object including a mass, for example, the abdomen, lymph, etc. Optionally, the computer device can obtain the fat-suppressed magnetic resonance image from the magnetic resonance device, or can obtain the fat-suppressed magnetic resonance image from a PACS (Picture Archiving and Communication Systems) server.

[0086] S202. Input the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtain a mask image of the region of interest through the segmentation network.

[0087] Specifically, the computer device inputs the obtained fat-suppressed magnetic resonance image into a preset segmentation network, and obtains a mask image of the region of interest through the segmentation network. It can be understood that the obtained mask image of the region of interest is a binary image. Generally, in the convolutional operation of a neural network, since the voxel points at the edge of the input image do not lie at the center of the convolutional kernel, and the convolutional kernel cannot extend beyond the edge of the input image, the information at the edge of the input image will be omitted, resulting in inconsistent sizes of the input image and the output image. Optionally, the convolutional layer of the segmentation network may include padding convolution, which can perform boundary padding on the input image to increase the size of the image matrix, so that the size of the output image is the same as that of the input image. Optionally, padding convolution is usually filled with "0". When the convolutional kernel operates on the fat-suppressed magnetic resonance image, it can extend to the virtual voxels outside the edge of the fat-suppressed magnetic resonance image, so that the size of the output image of the segmentation network is the same as that of the input fat-suppressed magnetic resonance image.

[0088] S203. Obtain the segmentation result of the region of interest in the fat-suppressed magnetic resonance image according to the fat-suppressed magnetic resonance image and the mask image.

[0089] Specifically, the computer device obtains the segmentation result of the region of interest in the fat-suppressed magnetic resonance image according to the fat-suppressed magnetic resonance image and the mask image of the region of interest. Optionally, the computer device can perform a dot product on each pixel point of the fat-suppressed magnetic resonance image and the mask image of the region of interest to obtain the segmentation result of the region of interest in the fat-suppressed magnetic resonance image; or it can perform superposition on the fat-suppressed magnetic resonance image and the mask image of the region of interest to obtain the segmentation result of the region of interest in the fat-suppressed magnetic resonance image.

[0090] In the above image segmentation method, by performing block processing on the fat-suppressed magnetic resonance image, contrast-limited adaptive histogram equalization processing can be performed on each image block as a unit to obtain the processed fat-suppressed magnetic resonance image. Since the contrast-limited adaptive histogram equalization processing is performed, the contrast of the fat-suppressed magnetic resonance image can be enhanced, and the noise points of the fat-suppressed magnetic resonance image can be reduced. In this way, when the processed fat-suppressed magnetic resonance image is input into a preset segmentation network, the segmentation network can accurately segment the processed fat-suppressed magnetic resonance image, accurately obtain the mask image of the region of interest in the fat-suppressed magnetic resonance image, improve the accuracy of obtaining the mask image of the region of interest, and thus the segmentation result of the region of interest in the fat-suppressed magnetic resonance image can be accurately obtained according to the fat-suppressed magnetic resonance image and the obtained mask image of the region of interest.

[0091] In the scenario where the processed fat-suppressed magnetic resonance image is input into a preset segmentation network and a mask image of the region of interest is obtained through the segmentation network, in one embodiment, the segmentation network includes: an encoding unit and a decoding unit; as Figure 3 shown, the above S202 includes:

[0092] S301, input the processed fat-suppressed magnetic resonance image into the encoding unit, perform boundary padding on the processed fat-suppressed magnetic resonance image using padding convolution, and perform downsampling on the fat-suppressed magnetic resonance image after boundary padding to obtain a downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image.

[0093] Specifically, the computer device inputs the processed fat-suppressed magnetic resonance image into the encoding unit of the segmentation network, performs boundary padding on the processed fat-suppressed magnetic resonance image using padding convolution, and performs downsampling on the fat-suppressed magnetic resonance image after boundary padding to obtain a downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image. Optionally, as Figure 3a shown, the encoding unit includes multiple convolutional blocks, each convolutional block consists of a convolutional layer and a pooling layer, and each convolutional layer is connected to the normalization layer and the activation function layer of the segmentation network. The convolutional layer includes padding convolution; the padding convolution is used to perform boundary padding on the processed fat-suppressed magnetic resonance image to increase the size of the image matrix corresponding to the processed fat-suppressed magnetic resonance image. The function used by the normalization layer is the instance normalization function, and the function used by the activation function layer is the leaky rectified linear unit function. Exemplarily, as an implementable embodiment, each of the above convolutional blocks may consist of two 3×3×3 convolutional layers and one 2×2×2 max pooling layer, and a normalization (Normalization) layer and a rectified linear unit (ReLU) are connected after each convolutional layer. It can be understood that performing boundary padding on the processed fat-suppressed magnetic resonance image using padding convolution is an operation performed by the convolutional layers included in each of the above convolutional blocks. Optionally, the number of times of downsampling performed by the encoding unit on the fat-suppressed magnetic resonance image after boundary padding can be determined according to the size of the fat-suppressed magnetic resonance image, and the sizes of the obtained downsampled feature map and the upsampled feature map. In this embodiment, the number of times of downsampling can be set to 5.

[0094] S302, input the downsampled feature map into the decoding unit, perform boundary padding on the downsampled feature map using padding convolution, and perform upsampling on the downsampled feature map after boundary padding to obtain a mask image of the region of interest.

[0095] Specifically, the computer device inputs the downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image obtained above into the decoding unit of the segmentation network, performs boundary padding on the obtained downsampled feature map using padding convolution, and upsamples the boundary-padded downsampled feature map to obtain the mask image of the above-mentioned region of interest. Optionally, please continue to refer to Figure 3a , the decoding unit includes a plurality of deconvolution blocks, each deconvolution block is composed of a convolutional layer and a deconvolution layer, each convolutional layer is connected to the above-mentioned normalization layer and activation function layer, the function used in the normalization layer is the instance normalization function, and the function used in the activation function layer is the leaky rectified linear unit function. Optionally, the number of convolutional blocks included in the above-mentioned encoding unit is the same as the number of deconvolution blocks included in the above-mentioned decoding unit. That is to say, the number of times of downsampling and upsampling performed by the segmentation network corresponds. Optionally, the number of the above-mentioned convolutional blocks and deconvolution blocks can be determined according to the size of the fat-suppressed magnetic resonance image, the size of the above-mentioned upsampled feature map, and the size of the above-mentioned downsampled special map. It can be understood that the above-mentioned decoding unit is an upsampling process, the decoding unit includes a plurality of deconvolution blocks, and upsampling can be implemented by a 2×2×2 deconvolution. Optionally, the last layer of the decoding unit can map the upsampled feature map to the output layer of the entire segmentation network through a 1×1×1 convolution, and the output image is the classification result of each voxel point of the input image, that is, the classification result of the background voxel or the foreground voxel.

[0096] It should be noted that the function used in the normalization layer in this embodiment is the instance normalization function because the GPU memory of the computer device limits the size of the batch during the training process, so instance normalization (InstanceNormalization, IN) is used instead of batch normalization (Batch Normalization). Since when segmenting the region of interest of the fat-suppressed magnetic resonance image, there are many background voxels (voxel values are approximately 0) in the fat-suppressed magnetic resonance image, and they may become negative values after preprocessing. In the gradient-based learning process, if the ReLU function is used, the gradient parameter of the neuron is always 0 and will not be activated in the subsequent training process, resulting in slow training. Therefore, the leaky rectified linear unit (Leaky ReLU) function is used instead of the ReLU function. When the input voxel value is negative, the gradient parameter of the neuron will not be 0 (but a very small number), which can keep the neuron activated all the time, thereby accelerating the learning speed of the network.

[0097] In this embodiment, the computer device inputs the processed fat-suppressed magnetic resonance image into the encoding unit of the segmentation network. Through this encoding unit, padding convolution can be used to perform boundary padding on the processed fat-suppressed magnetic resonance image, which can increase the size of the matrix corresponding to the processed fat-suppressed magnetic resonance image, ensuring that the information at the edge of the processed fat-suppressed magnetic resonance image will not be missed. After boundary padding, the fat-suppressed magnetic resonance image can completely retain the information of the processed fat-suppressed magnetic resonance image, so that the downsampling can be accurately performed on the fat-suppressed magnetic resonance image after boundary padding, improving the accuracy of obtaining the downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image. Furthermore, the obtained downsampled feature map can be input into the decoding unit of the segmentation network. Through this decoding unit, padding convolution can be used to perform boundary padding on the obtained downsampled feature map, so that the downsampled feature map after boundary padding can completely retain the information of the downsampled feature map, and thus the upsampling can be accurately performed on the downsampled feature map after boundary padding, thereby improving the accuracy of the mask image of the region of interest of the processed fat-suppressed magnetic resonance image obtained.

[0098] In the scenario of inputting the obtained downsampled feature map into the decoding unit of the segmentation network, performing upsampling on the downsampled feature map, and obtaining the mask image of the region of interest of the fat-suppressed image, the above-mentioned segmentation network further includes a connection unit. The above-mentioned encoding unit is connected to the above-mentioned encoding unit through the connection unit. In one embodiment, as Figure 4 shown, the above S302 includes:

[0099] S401, connecting the downsampled feature map output by the first convolutional block and the upsampled feature map output by the first transposed convolutional block through the connection unit to obtain the connected feature map; the first transposed convolutional block is the transposed convolutional block corresponding to the first convolutional block.

[0100] Specifically, the computer device connects the downsampled feature map output by the first convolutional block of the above-mentioned encoding unit and the upsampled feature map output by the first transposed convolutional block of the above-mentioned decoding unit through the connection unit of the segmentation network to obtain the connected feature map. Among them, the above first transposed convolution is the transposed convolutional block corresponding to the above first convolutional block. Exemplarily, please continue to refer to Figure 3a , the connection unit of the segmentation network can be Figure 3a the dotted line shown in Figure 3a the last convolutional block from top to bottom (on the left) during the downsampling process in Figure 3aThe first transposed convolution block from bottom to top during upsampling (on the right side). Optionally, the computer device may splice the downsampled feature map output by the first convolution block and the upsampled feature map output by the first transposed convolution block through a connection unit to obtain a spliced feature map.

[0101] S402. Input the spliced feature map into a second transposed convolution block for upsampling to obtain a mask image of the region of interest; the second transposed convolution block is the next transposed convolution block adjacent to the first transposed convolution block.

[0102] Specifically, the computer device inputs the above-mentioned spliced feature map into the second transposed convolution block for upsampling to obtain a mask image of the region of interest of the above-mentioned fat-suppressed magnetic resonance image. Among them, the above-mentioned second transposed convolution block is the next transposed convolution block adjacent to the above-mentioned first transposed convolution block. Exemplarily, the second transposed convolution block may be Figure 3a The second transposed convolution block from bottom to top during upsampling (on the right side).

[0103] In this embodiment, the computer device connects the downsampled feature map output by the first convolution block of the encoding unit and the upsampled feature map output by the first transposed convolution block of the decoding unit through the connection unit of the segmentation network. The obtained spliced feature map can avoid the loss of detailed information during downsampling and upsampling, and can obtain a spliced feature map with higher accuracy. Thus, the spliced feature map with higher accuracy can be input into the second transposed convolution block of the decoding unit for upsampling to obtain a mask image of the region of interest with higher accuracy, improving the segmentation accuracy of the fat-suppressed image.

[0104] In some scenarios, before performing block processing on the fat-suppressed magnetic resonance image and performing contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image, the computer device may also perform resampling processing on the fat-suppressed magnetic resonance image, perform block processing on the resampled magnetic resonance image. In one embodiment, before the above S201, the method further includes: performing resampling on the fat-suppressed magnetic resonance image according to the size of the fat-suppressed magnetic resonance image and the spacing between voxels of the fat-suppressed magnetic resonance image to obtain a resampled magnetic resonance image.

[0105] Specifically, the computer device resamples the fat-suppressed magnetic resonance image according to the size of the fat-suppressed magnetic resonance image and the spacing between the voxels of the fat-suppressed magnetic resonance image, and obtains the resampled magnetic resonance image. It should be noted that for anisotropic fat-suppressed magnetic resonance images, since the voxel spacings in different directions are different, the direction with higher image resolution can be downsampled first to make its resolution consistent with other directions, and then the anisotropic fat-suppressed magnetic resonance images in all directions can be resampled simultaneously according to the processing method of isotropic images. Optionally, the computer device can use cubic spline interpolation to resample the fat-suppressed magnetic resonance image, or use nearest neighbor interpolation to resample the fat-suppressed magnetic resonance image.

[0106] In this embodiment, the computer device can resample the fat-suppressed magnetic resonance image according to the size of the fat-suppressed magnetic resonance image and the spacing between the voxels of the fat-suppressed magnetic resonance image, and obtain the resampled magnetic resonance image, ensuring the accuracy of the obtained resampled magnetic resonance image.

[0107] In the scenario of inputting the fat-suppressed magnetic resonance image into the segmentation network, the segmentation network needs to be pre-trained. In one embodiment, as Figure 5 shown, the training process of the segmentation network includes:

[0108] S501, obtain a sample fat-suppressed magnetic resonance image and a gold standard image corresponding to the sample fat-suppressed magnetic resonance image; the sample fat-suppressed magnetic resonance image includes a sample region of interest.

[0109] Specifically, the computer device obtains a sample fat-suppressed magnetic resonance image and a gold standard image corresponding to the sample fat-suppressed magnetic resonance image. Among them, the sample fat-suppressed magnetic resonance image includes a sample region of interest. Optionally, the sample region of interest can be the breast, or the region of the scanned object including a mass, such as the abdomen, lymph, etc. Optionally, the computer device can obtain the sample fat-suppressed magnetic resonance image from a magnetic resonance device, or obtain the sample fat-suppressed magnetic resonance image from a PACS (Picture Archiving and Communication Systems) server. Optionally, the computer device can perform data augmentation operations such as mirror transformation, scale stretching transformation, rotation and translation to expand the capacity of the sample fat-suppressed magnetic resonance image and prevent model overfitting.

[0110] S502, perform a block processing on the sample fat-suppressed magnetic resonance image, and perform contrast-limited adaptive histogram equalization processing on each sample image block to obtain the processed sample fat-suppressed magnetic resonance image.

[0111] Specifically, the computer device performs a blocking process on the acquired sample fat-suppressed magnetic resonance image, and performs Contrast Limited Adaptive Histogram Equalization (CLAHE) processing on each sample image block after blocking to obtain the processed sample fat-suppressed magnetic resonance image. It should be noted here that performing the contrast-limited adaptive histogram equalization process on the acquired sample fat-suppressed magnetic resonance image in units of blocks is a preprocessing operation added in view of the characteristics of the sample fat-suppressed magnetic resonance image, such as relatively high noise level, low intensity contrast, and uneven fat suppression. Optionally, the computer device can perform a blocking process on the sample fat-suppressed magnetic resonance image, perform histogram equalization in units of image blocks, and then use linear interpolation to determine the voxel values between blocks, simultaneously enhancing the contrast and suppressing noise, thereby improving the accuracy of segmentation. Optionally, before performing the blocking process on the sample fat-suppressed magnetic resonance image, the computer device can also perform a resampling process on the sample fat-suppressed magnetic resonance image. Optionally, the computer device can use cubic spline interpolation to perform a resampling process on the sample fat-suppressed magnetic resonance image. Optionally, the computer device can also use nearest neighbor interpolation to perform a resampling process on the gold standard image corresponding to the sample fat-suppressed magnetic resonance image.

[0112] S503. Input the processed sample fat-suppressed magnetic resonance image into the initial segmentation network, and obtain the sample mask image of the sample region of interest through the initial segmentation network.

[0113] Specifically, the computer device inputs the processed sample fat-suppressed magnetic resonance image obtained above into the initial segmentation network, and obtains the sample mask image of the sample region of interest through the initial segmentation network. Optionally, the network structure of the initial segmentation network can refer to the description of the above embodiment, and will not be elaborated here in this embodiment.

[0114] S504. Obtain the value of the loss function of the initial segmentation network according to the sample mask image and the gold standard image; the loss function of the initial segmentation network includes a Dice loss function and a cross-entropy loss function; the value of the loss function of the initial segmentation network is the sum of the value of the Dice loss function and the value of the cross-entropy loss function.

[0115] Specifically, the computer device obtains the value of the loss function of the above initial segmentation network based on the obtained sample mask image and the gold standard image corresponding to the sample fat-suppressed magnetic resonance image. Among them, the loss function of the initial segmentation network includes the Dice loss function and the cross-entropy loss function; the value of the loss function of the initial segmentation network is the sum of the value of the Dice loss function and the value of the cross-entropy loss function. It can be understood that the Dice loss function can directly optimize the segmentation similarity and can solve the problem of unbalanced training sample categories. However, its gradient form is complex, and the gradient changes violently during the backpropagation process, resulting in the problem that the network is difficult to converge. The cross-entropy loss function can measure the difference between the gold standard image and the segmentation result in the segmentation task. The smaller the cross-entropy value, the better the segmentation effect, but it cannot solve the problem of class imbalance. Therefore, combining the Dice loss function and the cross-entropy loss function can effectively improve the stability of the training process and the accuracy of segmentation. Optionally, the expressions of the above Dice loss function and cross-entropy loss function can be shown as the following formulas (1) and (2): In the formula, K represents the number of categories (K is 2 in this patent, representing the foreground and the background), I represents the voxel set in each Batch, u represents the Softmax output probability value, and v represents the one-hot encoding value of the gold standard.

[0116] S505. Train the initial segmentation network according to the value of the loss function of the initial segmentation network to obtain a segmentation network.

[0117] Specifically, the computer device trains the initial segmentation network according to the value of the loss function of the initial segmentation network to obtain the above-mentioned segmentation network. Optionally, the computer device may determine the initial segmentation network corresponding to when the value of the loss function of the initial segmentation network reaches a stable value or a minimum value as the above-mentioned segmentation network. Optionally, during the training process of the segmentation network, the computer device may use five-fold cross-validation, randomly divide the training set into 5 equal parts, and randomly select one part as the validation set in each fold of cross-validation process to determine the hyperparameters of the model. After training, the validation results of the five models are fused to determine the hyperparameters of the test model. During the training process, the initial learning rate of the initial segmentation network can be set to 0.01, and the learning rate is gradually decreased exponentially as the iteration continues to make the model more stable. Optionally, the computer device may adopt a strategy of randomly initializing weights with a normal distribution to assign initial values to the initial segmentation network. Optionally, the computer device may also use the Stochastic Gradient Descent method with Nesterov Momentum (parameter set to 0.9) to optimize the loss function. Optionally, in this embodiment, the computer device may define the epoch of the training process as the iterative optimization on 250 Batches, with the maximum value set to 1000, and stop training when the learning rate is lower than 10-6 or exceeds 1000 epochs. Optionally, in the test stage, the computer device may use the sliding window method to predict each test sample, where the overlapping area is set to half of the size of the input image, and the weight of the area close to the center is increased.

[0118] Optionally, in one embodiment, during the training process, preprocessing such as resampling and limited contrast adaptive histogram equalization can be performed on the sample fat-suppressed magnetic resonance images. After the preprocessing of resampling and limited contrast adaptive histogram equalization, the image size of the images is unified to the median size of the images in the training dataset. The batch size (Batch Size) during the training process can be determined according to the GPU memory size of the computer device, giving priority to ensuring that the input size is as large as possible to guarantee the accuracy of the segmentation result, and secondly ensuring that the batch size is at least 2 to prevent gradient noise caused by too few samples. Exemplarily, taking the training of the breast segmentation network and the gland segmentation network as an example, the image input sizes of the breast segmentation and gland segmentation networks can be 40×256×192 and 64×96×224 respectively, and the batch size can be set to 2. To obtain features with sufficient information, the size of the output feature map can be set to at least 4×4×4; to limit the model size, the upper limit of the number of features can be set to 320. Optionally, the number of downsampling times of the sample fat-suppressed magnetic resonance images during the training process can be determined according to the size of the input sample fat-suppressed magnetic resonance images and the size of the feature map. Exemplarily, as a feasible implementation method, the number of downsampling times can be 5.

[0119] In this embodiment, the computer device performs a blocking process on the sample fat-suppressed magnetic resonance image, and can perform limited contrast adaptive histogram equalization processing for each sample image block to obtain the processed sample fat-suppressed magnetic resonance image. Since the limited contrast adaptive histogram equalization processing is performed, the contrast of the sample fat-suppressed magnetic resonance image can be enhanced, and the noise points of the sample fat-suppressed magnetic resonance image can be reduced. In this way, when the processed sample fat-suppressed magnetic resonance image is input into the initial segmentation network, the sample mask image of the sample region of interest can be accurately obtained through the initial segmentation network, so that the value of the loss function of the initial segmentation network can be obtained according to the sample mask image and the gold standard image corresponding to the sample fat-suppressed magnetic resonance image. Furthermore, the initial segmentation network can be trained according to the value of the loss function of the initial segmentation network. Through a large number of sample fat-suppressed magnetic resonance images, the initial segmentation network can be accurately trained, and the accuracy of the obtained segmentation network is improved.

[0120] In one embodiment, as Figure 6 shown, taking a breast image as an example, an image segmentation method is provided. Taking the computer device in Figure 1 as an example for illustration, the method includes the following steps:

[0121] S601, perform a blocking process on the fat-suppressed breast image including the breast, and perform limited contrast adaptive histogram equalization processing for each image block to obtain the processed fat-suppressed breast image.

[0122] Specifically, the computer device performs a block processing on the acquired fat-suppressed breast images, and performs a contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed breast images. It should be noted here that the contrast-limited adaptive histogram equalization processing of the acquired fat-suppressed breast images in units of blocks is a preprocessing operation added in view of the characteristics of the fat-suppressed breast images, such as relatively high noise level, low intensity contrast, and uneven fat suppression. Optionally, the computer device can perform a block processing on the fat-suppressed breast images, perform a histogram equalization on each image block, and then use linear interpolation to determine the voxel values between blocks, so as to enhance the contrast and suppress the noise at the same time, thereby improving the accuracy of segmentation. Optionally, the fat-suppressed breast images acquired by the computer device can be fat-suppressed breast images including both breasts, or fat-suppressed breast images including a single breast. It should be noted that the fat-suppressed magnetic resonance images including the breast in this application can be fat-suppressed dynamic enhanced magnetic resonance images. Optionally, the computer device can acquire the fat-suppressed magnetic resonance images including the breast from a magnetic resonance device, or can acquire the fat-suppressed magnetic resonance images including the breast from a PACS (Picture Archiving and Communication Systems) server.

[0123] S602. Input the processed fat-suppressed breast images into a preset breast segmentation network, and obtain a breast mask image through the breast segmentation network.

[0124] Specifically, the computer device inputs the processed fat-suppressed breast images into a preset breast segmentation network, and obtains a breast mask image through the breast segmentation network. Among them, the convolutional layer of the breast segmentation network includes a padding convolution; the padding convolution is used to perform boundary padding on the above-mentioned fat-suppressed breast images to increase the size of the image matrix corresponding to the above-mentioned fat-suppressed breast images. It should be noted that for the specific description of the network structure of the breast segmentation network, please refer to the description of the above-mentioned segmentation network, and this embodiment will not be elaborated here. Exemplarily, the obtained breast mask image can be the breast mask image exemplified in 7a.

[0125] S603. Obtain a breast segmentation image according to the breast mask image and the fat-suppressed breast images.

[0126] Specifically, the computer device obtains a breast segmentation image based on the obtained breast mask image and the above-mentioned fat-suppressed breast image. Optionally, the computer device may perform a dot product on the breast mask image and the fat-suppressed breast image to obtain the breast segmentation image. Optionally, the computer device may also superimpose the breast mask image and the fat-suppressed breast image to obtain the breast segmentation image.

[0127] S604. Input the breast segmentation image into a preset gland segmentation network, and obtain a gland mask image through the gland segmentation network.

[0128] Specifically, the computer device inputs the breast segmentation image into a preset gland segmentation network, and obtains a gland mask image through the gland segmentation network. Among them, the convolutional layer of the gland segmentation network includes a padded convolution; the padded convolution is used to perform boundary padding on the above-mentioned breast segmentation image to increase the size of the image matrix corresponding to the breast segmentation image. It should be noted that for the specific description of the network structure of the gland segmentation network, please refer to the description of the above-mentioned segmentation network, and this embodiment will not be elaborated here. Exemplarily, the obtained gland mask image may be the gland mask image exemplified in 6a. Optionally, before the computer device inputs the breast segmentation image into the above-mentioned gland segmentation network, it may also perform a block processing on the breast segmentation image, perform a contrast-limited adaptive histogram equalization processing on each image block as a unit, obtain the processed breast segmentation image, and input the processed breast segmentation image into the above-mentioned gland segmentation network.

[0129] S605. Obtain a gland segmentation image based on the gland mask image and the fat-suppressed breast image, or based on the gland mask image and the breast segmentation image.

[0130] Specifically, the computer device obtains a gland segmentation image based on the obtained gland mask image and the above-mentioned fat-suppressed breast image; or, the computer device obtains a gland segmentation image based on the obtained gland mask image and the above-mentioned breast segmentation image. Optionally, the computer device may perform a dot product on the gland mask image and the above-mentioned fat-suppressed breast image to obtain the gland segmentation image, or may also superimpose the gland mask image and the above-mentioned fat-suppressed breast image to obtain the gland segmentation image. Optionally, the computer device may perform a dot product on the gland mask image and the above-mentioned breast segmentation image to obtain the gland segmentation image, or may also superimpose the gland mask image and the above-mentioned breast segmentation image to obtain the gland segmentation image. It can be understood that since there may be certain errors in the obtained breast segmentation image, obtaining the gland segmentation image based on the gland mask image and the breast segmentation image may superimpose errors, resulting in the accuracy of the obtained gland segmentation image being lower than that of the gland segmentation image obtained based on the gland mask image and the fat-suppressed breast image.

[0131] S606. Obtain the breast density corresponding to the fat-suppressed breast image according to the breast segmentation image and the gland segmentation image.

[0132] Specifically, the computer device obtains the breast density corresponding to the fat-suppressed breast image according to the obtained breast segmentation image and the obtained gland segmentation image. Optionally, since both the obtained breast segmentation image and the gland segmentation image are three-dimensional images, the computer device can obtain the volume of the breast and the volume of the bilateral glands according to the voxel spacing of the breast segmentation image and the gland segmentation image and the resolution of the breast segmentation image and the gland segmentation image, so as to obtain the breast density. Optionally, the computer device can multiply the number of voxels of the breast segmentation image and the gland segmentation image and the resolution of the breast segmentation image and the gland segmentation image to obtain the volume of the breast and the volume of the bilateral glands.

[0133] In the above image segmentation method, by performing block processing on the fat-suppressed breast image containing the breast and performing contrast-limited adaptive histogram equalization processing on each image block, since the contrast-limited adaptive histogram equalization processing can enhance the contrast of the fat-suppressed breast image and reduce the noise points of the fat-suppressed breast image, when the processed fat-suppressed breast image is input into the preset segmentation network, the segmentation network can accurately segment the processed fat-suppressed breast image, accurately obtain the breast mask image, improve the accuracy of obtaining the breast mask image, and similarly, improve the accuracy of the obtained gland mask image, thereby improving the accuracy of the obtained breast segmentation image and gland segmentation image. In this way, according to the breast segmentation image and the gland segmentation image, the breast density corresponding to the fat-suppressed breast image can be accurately obtained, and the accuracy of obtaining the breast density corresponding to the fat-suppressed breast image is improved.

[0134] In one embodiment, the effect of the image segmentation method proposed in this application is evaluated. Table 1 gives the results of five evaluation indexes of the segmentation of the fat-suppressed breast image by the segmentation network in this application relative to the gold standard, and the results in this table are all expressed in the form of mean ± standard deviation.

[0135] Table 1 Segmentation Performance Evaluation

[0136]

[0137] As can be seen from Table 1, the DSCs of breast segmentation and gland segmentation are 0.968±0.017 and 0.877 ±0.081 respectively, and the ASDs are 0.201±0.082 mm and 0.310±0.041 mm respectively, indicating that the segmentation model can accurately segment the target area and precisely detect the boundary. Generally, deep learning methods for medical images are prone to produce a high false positive rate (FPR), resulting in over-segmentation. For breast and gland segmentation, the segmentation results using the segmentation network proposed in this application have high sensitivity and specificity, indicating that the segmentation network has a high true positive rate (TPR) and a low false positive rate, and can effectively reduce over-segmentation. Figure 7 For the example of the segmentation result, the DSC of breast segmentation is 0.961 and the ASD is 0.175 mm; the DSC of gland segmentation is 0.922 and the ASD is 0.296 mm. Among them, the correlation between the segmentation result and the gold standard can be as Figure 8 shown Figure 8 where the x-axis represents the physical actual volume of the gold standard and the y-axis represents the physical actual volume of the segmentation result. The correlation coefficients r of breast segmentation and gland segmentation are 0.995 (p-value <0.001) and 0.971 (p-value <0.001) respectively, indicating that the segmentation result has a strong consistency with the gold standard.

[0138] It should be understood that although Figures 2-6 the steps in the flowchart of Figures 2-6 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover,

[0139] In one embodiment, as Figure 9 shown, an image segmentation device is provided, including: a first acquisition module, a first segmentation module, and a second acquisition module, where:

[0140] The first acquisition module is used to perform block processing on the fat-suppressed magnetic resonance image, and perform limited contrast adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes the region of interest.

[0141] The first segmentation module is used to input the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtain a mask image of the region of interest through the segmentation network.

[0142] The second acquisition module is used to obtain the segmentation result of the region of interest in the fat-suppressed magnetic resonance image according to the fat-suppressed magnetic resonance image and the mask image.

[0143] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0144] Based on the above embodiment, optionally, the above segmentation network includes: an encoding unit and a decoding unit; the above first segmentation module includes: a downsampling unit and an upsampling unit, where:

[0145] The downsampling unit is used to input the processed fat-suppressed magnetic resonance image into the encoding unit, perform boundary padding on the processed fat-suppressed magnetic resonance image using padding convolution, and perform downsampling on the fat-suppressed magnetic resonance image after boundary padding to obtain a downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image.

[0146] The upsampling unit is used to input the downsampled feature map into the decoding unit, perform boundary padding on the downsampled feature map using padding convolution, and perform upsampling on the downsampled feature map after boundary padding to obtain a mask image of the region of interest.

[0147] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0148] Based on the above embodiment, optionally, the encoding unit includes multiple convolutional blocks, each convolutional block is composed of a convolutional layer and a pooling layer, and each convolutional layer is connected to the normalization layer and the activation function layer of the segmentation network; the convolutional layer includes padding convolution; the padding convolution is used to perform boundary padding on the fat-suppressed magnetic resonance image to increase the size of the image matrix corresponding to the fat-suppressed magnetic resonance image; the decoding unit includes multiple deconvolutional blocks, each deconvolutional block is composed of a convolutional layer and a deconvolutional layer, and each convolutional layer is connected to the normalization layer and the activation function layer; where the number of convolutional blocks is the same as the number of deconvolutional blocks; the function adopted by the normalization layer is the instance normalization function; the function adopted by the activation function layer is the leaky rectified linear unit function.

[0149] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0150] Based on the above embodiments, optionally, the segmentation network further includes a connection unit; the encoding unit is connected to the decoding unit through the connection unit; the above upsampling unit is specifically configured to connect the downsampled feature map output by the first convolutional block and the upsampled feature map output by the first transposed convolutional block through the connection unit to obtain a connected feature map; the first transposed convolutional block is the transposed convolutional block corresponding to the first convolutional block; input the connected feature map into the second transposed convolutional block for upsampling to obtain a mask image of the region of interest; the second transposed convolutional block is the next transposed convolutional block adjacent to the first transposed convolutional block.

[0151] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0152] Based on the above embodiments, optionally, the number of convolutional blocks and transposed convolutional blocks is determined according to the size of the fat-suppressed magnetic resonance image, the size of the upsampled feature map, and the size of the downsampled feature map.

[0153] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0154] Based on the above embodiments, optionally, the above device further includes: a resampling module, where:

[0155] The resampling module is configured to resample the fat-suppressed magnetic resonance image according to the size of the fat-suppressed magnetic resonance image and the spacing between voxels of the fat-suppressed magnetic resonance image to obtain a resampled magnetic resonance image.

[0156] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0157] Based on the above embodiments, optionally, the above device further includes: a third acquisition module, a processing module, a second segmentation module, a fourth acquisition module, and a training module, where:

[0158] The third acquisition module is configured to acquire a sample fat-suppressed magnetic resonance image and a gold standard image corresponding to the sample fat-suppressed magnetic resonance image; the sample fat-suppressed magnetic resonance image includes a sample region of interest.

[0159] The processing module is configured to perform block processing on the sample fat-suppressed magnetic resonance image and perform contrast-limited adaptive histogram equalization processing on each sample image block to obtain a processed sample fat-suppressed magnetic resonance image.

[0160] The second segmentation module is used to input the processed sample fat-suppressed magnetic resonance image into the initial segmentation network, and obtain the sample mask image of the sample region of interest through the initial segmentation network.

[0161] The fourth acquisition module is used to obtain the value of the loss function of the initial segmentation network according to the sample mask image and the gold standard image; the loss function of the initial segmentation network includes the Dice loss function and the cross-entropy loss function; the value of the loss function of the initial segmentation network is the sum of the value of the Dice loss function and the value of the cross-entropy loss function.

[0162] The training module is used to train the initial segmentation network according to the value of the loss function of the initial segmentation network to obtain the segmentation network.

[0163] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0164] For the specific limitations of the image segmentation device, reference can be made to the limitations on the image segmentation method in the above text, which will not be elaborated here. Each module in the above image segmentation device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in the form of hardware or independent of it, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0165] In one embodiment, as Figure 10 shown, an image segmentation device is provided, including: a first acquisition module, a first segmentation module, a second acquisition module, a second segmentation module, a third acquisition module, and a fourth acquisition module, where:

[0166] The first acquisition module is used to perform block processing on the fat-suppressed breast image containing the breast, and perform limited contrast adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed breast image.

[0167] The first segmentation module is used to input the processed fat-suppressed breast image into the preset breast segmentation network, and obtain the breast mask image through the breast segmentation network.

[0168] The second acquisition module is used to obtain the breast segmentation image according to the breast mask image and the fat-suppressed breast image;

[0169] The second segmentation module is used to input the breast segmentation image into the preset gland segmentation network, and obtain the gland mask image through the gland segmentation network.

[0170] A third acquisition module, configured to acquire a gland segmentation image according to the gland mask image and the fat-suppressed breast image, or according to the gland mask image and the breast segmentation image;

[0171] A fourth acquisition module, configured to acquire the breast density corresponding to the fat-suppressed breast image according to the breast segmentation image and the gland segmentation image.

[0172] The image segmentation device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0173] For the specific limitations of the image segmentation device, reference can be made to the limitations on the image segmentation method in the above text, which will not be elaborated here. Each module in the above image segmentation device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or independent of it, or stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0174] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0175] Perform block processing on the fat-suppressed magnetic resonance image, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes the region of interest;

[0176] Input the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtain a mask image of the region of interest through the segmentation network;

[0177] According to the fat-suppressed magnetic resonance image and the mask image, obtain the segmentation result of the region of interest in the fat-suppressed magnetic resonance image.

[0178] The computer device provided in the above embodiment has an implementation principle and technical effect similar to those of the above method embodiment, which will not be elaborated here.

[0179] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0180] Perform block processing on the fat-suppressed breast image including the breast, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed breast image;

[0181] Input the processed fat-suppressed breast image into a preset breast segmentation network to obtain a breast mask image through the breast segmentation network;

[0182] Obtain a breast segmentation image based on the breast mask image and the fat-suppressed breast image;

[0183] Input the breast segmentation image into a preset gland segmentation network to obtain a gland mask image through the gland segmentation network;

[0184] Obtain a gland segmentation image based on the gland mask image and the fat-suppressed breast image, or based on the gland mask image and the breast segmentation image;

[0185] Obtain the breast density corresponding to the fat-suppressed breast image based on the breast segmentation image and the gland segmentation image.

[0186] The computer device provided in the above embodiment has the same implementation principle and technical effects as the above method embodiment, which will not be elaborated here.

[0187] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0188] Perform block processing on the fat-suppressed magnetic resonance image, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes a region of interest;

[0189] Input the processed fat-suppressed magnetic resonance image into a preset segmentation network to obtain a mask image of the region of interest through the segmentation network;

[0190] Obtain the segmentation result of the region of interest in the fat-suppressed magnetic resonance image based on the fat-suppressed magnetic resonance image and the mask image.

[0191] The computer-readable storage medium provided in the above embodiment has the same implementation principle and technical effects as the above method embodiment, which will not be elaborated here.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0193] Perform block processing on the fat-suppressed breast image containing the breast, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed breast image;

[0194] Input the processed fat-suppressed breast image into a preset breast segmentation network to obtain a breast mask image through the breast segmentation network;

[0195] Obtain a breast segmentation image based on the breast mask image and the fat-suppressed breast image;

[0196] Input the breast segmentation image into a preset gland segmentation network to obtain a gland mask image through the gland segmentation network;

[0197] Obtain a gland segmentation image based on the gland mask image and the fat-suppressed breast image, or based on the gland mask image and the breast segmentation image;

[0198] Obtain the breast density corresponding to the fat-suppressed breast image based on the breast segmentation image and the gland segmentation image.

[0199] The computer-readable storage medium provided in the above embodiments has the same implementation principle and technical effects as the above method embodiments, and will not be described in detail here.

[0200] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0201] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0202] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An image segmentation method, characterized in that, The method includes: Performing a block processing on the fat-suppressed magnetic resonance image, and performing a contrast-limited adaptive histogram equalization processing on each image block as a unit to obtain a processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes a region of interest; Inputting the processed fat-suppressed magnetic resonance image into a preset segmentation network, and obtaining a mask image of the region of interest through the segmentation network; the segmentation network includes an encoding unit and a decoding unit, and the convolutional layer of the segmentation network includes a padding convolution, and the padding convolution is used for performing boundary padding on the processed fat-suppressed magnetic resonance image; Obtaining a segmentation result of the region of interest in the fat-suppressed magnetic resonance image according to the fat-suppressed magnetic resonance image and the mask image; The step of inputting the processed fat-suppressed magnetic resonance image into a preset segmentation network and obtaining a mask image of the region of interest through the segmentation network includes: Inputting the processed fat-suppressed magnetic resonance image into the encoding unit, performing boundary padding on the processed fat-suppressed magnetic resonance image by using the padding convolution, and performing downsampling on the fat-suppressed magnetic resonance image after boundary padding to obtain a downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image; Inputting the downsampled feature map into the decoding unit, performing boundary padding on the downsampled feature map by using the padding convolution, and performing upsampling on the downsampled feature map after boundary padding to obtain a mask image of the region of interest.

2. The method according to claim 1, characterized in that, The encoding unit includes a plurality of convolutional blocks, each convolutional block is composed of a convolutional layer and a pooling layer, and each convolutional layer is connected to the normalization layer and the activation function layer of the segmentation network; the convolutional layer includes the padding convolution; the padding convolution is used for performing boundary padding on the fat-suppressed magnetic resonance image to increase the size of the image matrix corresponding to the fat-suppressed magnetic resonance image; the decoding unit includes a plurality of deconvolutional blocks, each deconvolutional block is composed of a convolutional layer and a deconvolutional layer, and each convolutional layer is connected to the normalization layer and the activation function layer; wherein, the number of convolutional blocks is the same as the number of deconvolutional blocks; the function adopted by the normalization layer is an instance normalization function; the function adopted by the activation function layer is a leaky rectified linear unit function.

3. The method according to claim 2, wherein The segmentation network further includes a connection unit; the encoding unit is connected to the decoding unit through the connection unit; the step of inputting the downsampled feature map into the decoding unit and performing upsampling on the downsampled feature map to obtain a mask image of the region of interest includes: Connecting the downsampled feature map output by the first convolutional block and the upsampled feature map output by the first deconvolutional block through the connection unit to obtain a connected feature map; the first deconvolutional block is the deconvolutional block corresponding to the first convolutional block; Inputting the connected feature map into the second deconvolutional block for upsampling to obtain a mask image of the region of interest; the second deconvolutional block is the next deconvolutional block adjacent to the first deconvolutional block.

4. The method according to claim 2, wherein The number of the convolutional blocks and the deconvolutional blocks is determined according to the size of the fat-suppressed magnetic resonance image, the size of the upsampled feature map, and the size of the downsampled feature map.

5. The method according to claim 1, characterized in that Before performing the block processing on the fat-suppressed magnetic resonance image and performing the contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image, the method further includes: Resampling the fat-suppressed magnetic resonance image according to the size of the fat-suppressed magnetic resonance image and the spacing between voxels of the fat-suppressed magnetic resonance image to obtain a resampled magnetic resonance image.

6. The method according to any one of claims 1 to 5, characterized in that The training process of the segmentation network includes: Obtaining a sample fat-suppressed magnetic resonance image and a gold standard image corresponding to the sample fat-suppressed magnetic resonance image; the sample fat-suppressed magnetic resonance image includes a sample region of interest. Performing block processing on the sample fat-suppressed magnetic resonance image, and performing contrast-limited adaptive histogram equalization processing on each sample image block to obtain a processed sample fat-suppressed magnetic resonance image. Inputting the processed sample fat-suppressed magnetic resonance image into an initial segmentation network, and obtaining a sample mask image of the sample region of interest through the initial segmentation network. Obtaining the value of the loss function of the initial segmentation network according to the sample mask image and the gold standard image; the loss function of the initial segmentation network includes a Dice loss function and a cross-entropy loss function; the value of the loss function of the initial segmentation network is the sum of the value of the Dice loss function and the value of the cross-entropy loss function. Training the initial segmentation network according to the value of the loss function of the initial segmentation network to obtain the segmentation network.

7. An image segmentation method, characterized in that, The method includes: Performing block processing on the fat-suppressed breast image including the breast, and performing contrast-limited adaptive histogram equalization processing on each image block to obtain a processed fat-suppressed breast image. Inputting the processed fat-suppressed breast image into a preset breast segmentation network, and obtaining a breast mask image through the breast segmentation network; the breast segmentation network includes a first encoding unit and a first decoding unit, and the convolutional layer of the breast segmentation network includes a padding convolution, and the padding convolution is used for performing boundary padding on the processed fat-suppressed breast image. Obtaining a breast segmentation image according to the breast mask image and the fat-suppressed breast image. Inputting the breast segmentation image into a preset gland segmentation network, and obtaining a gland mask image through the gland segmentation network; the gland segmentation network includes a second encoding unit and a second decoding unit; the convolutional layer of the gland segmentation network includes a padding convolution, and the padding convolution is used for performing boundary padding on the breast segmentation image. Obtaining a gland segmentation image according to the gland mask image and the fat-suppressed breast image, or according to the gland mask image and the breast segmentation image. Obtaining the breast density corresponding to the fat-suppressed breast image according to the breast segmentation image and the gland segmentation image. Inputting the processed fat-suppressed breast image into a preset breast segmentation network to obtain a breast mask image through the breast segmentation network, including: Inputting the processed fat-suppressed breast image into the first encoding unit, using padding convolution to perform boundary padding on the processed fat-suppressed magnetic resonance image, and performing downsampling on the fat-suppressed breast image after boundary padding to obtain a downsampled feature map corresponding to the processed fat-suppressed breast image; inputting the downsampled feature map into the first decoding unit, using the padding convolution to perform boundary padding on the downsampled feature map, and performing upsampling on the downsampled feature map after boundary padding to obtain the breast mask image; Inputting the breast segmentation image into a preset gland segmentation network to obtain a gland mask image through the gland segmentation network, including: Inputting the breast segmentation image into the second encoding unit, using padding convolution to perform boundary padding on the breast segmentation image, and performing downsampling on the breast segmentation image after boundary padding to obtain a downsampled feature map corresponding to the processed breast segmentation image; inputting the downsampled feature map into the second decoding unit, using the padding convolution to perform boundary padding on the downsampled feature map, and performing upsampling on the downsampled feature map after boundary padding to obtain the gland mask image.

8. An image segmentation device, characterized in that, The apparatus includes: A first acquisition module, configured to perform block processing on the fat-suppressed magnetic resonance image, and perform contrast-limited adaptive histogram equalization processing on each image block to obtain the processed fat-suppressed magnetic resonance image; the fat-suppressed magnetic resonance image includes a region of interest; A first segmentation module, configured to input the processed fat-suppressed magnetic resonance image into a preset segmentation network to obtain a mask image of the region of interest through the segmentation network; the segmentation network includes an encoding unit and a decoding unit, and the convolutional layer of the segmentation network includes padding convolution, and the padding convolution is used to perform boundary padding on the processed fat-suppressed magnetic resonance image; A second acquisition module, configured to obtain a segmentation result of the region of interest in the fat-suppressed magnetic resonance image according to the fat-suppressed magnetic resonance image and the mask image; The first segmentation module includes a downsampling unit and an upsampling unit; The downsampling unit is configured to input the processed fat-suppressed magnetic resonance image into the encoding unit, use padding convolution to perform boundary padding on the processed fat-suppressed magnetic resonance image, and perform downsampling on the fat-suppressed magnetic resonance image after boundary padding to obtain a downsampled feature map corresponding to the processed fat-suppressed magnetic resonance image; The upsampling unit is configured to input the downsampled feature map into the decoding unit, use the padding convolution to perform boundary padding on the downsampled feature map, and perform upsampling on the downsampled feature map after boundary padding to obtain the mask image of the region of interest.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images