MRI image segmentation method and device based on Mask RCNN algorithm, equipment and storage medium

Abstract

The embodiment of the invention discloses an MRI (Magnetic Resonance Imaging) image segmentation method, device and equipment based on a Mask RCNN (Regional Convolutional Neural Network) algorithm and a storage medium. The method comprises the following steps: extracting features of a preprocessed MRI image to obtain a feature map; the RPN is used for generating a final Proposa box, and the RPN is used for generating a final Proposa box; the ROI is selected to be subjected to pooling operation, feature fusion and feature coding in sequence, and deep target features and a feature coding graph are obtained; performing classification and frame regression on the targets in the ROI, and determining a target frame and a classification label of each target; determining a prediction segmentation mask of each target through the segmentation network; carrying out post-treatment; non-maximum suppression processing is carried out in a classification mode, and a final target frame, a classification label and a post-processed mask image are determined. According to the scheme, focuses with different scales and different shapes, sizes and positions can be processed through the Mask R-CNN algorithm, compared with a traditional MRI image segmentation technology, higher accuracy and generalization ability are achieved, the two tasks of target detection and segmentation are achieved at the same time, and doctors are helped to provide more accurate diagnosis and treatment.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an MRI image segmentation method, device, equipment and storage medium based on a Mask RCNN algorithm. Background Art

[0002] MRI image segmentation technology is widely used in the medical field and can be used for the detection and location of lesions, such as diagnosing tumors, neurological diseases, etc. In MRI images, doctors need to detect and locate the lesions of patients, but MRI image data is relatively complex and contains a lot of noise and other interference factors. Traditional MRI image segmentation technology relies on manually designed features, which usually require professional doctors or image processing experts to manually design features for segmentation, requiring high domain expertise and experience. In addition, manually designed features are costly and slow, which poses a great challenge to doctors' diagnosis. At the same time, traditional MRI image segmentation technology often requires manual parameter adjustment and model selection, and has poor adaptability, making it difficult to achieve excellent results in different data sets and application scenarios. Summary of the invention

[0003] Based on this, it is necessary to propose an MRI image segmentation method, device, equipment and medium based on the Mask RCNN algorithm to address the above problems.

[0004] To achieve the above objectives, the present application provides, in a first aspect, a MRI image segmentation method based on the Mask RCNN algorithm, the method comprising:

[0005] Preprocessing the MRI image data to obtain preprocessed MRI image data;

[0006] Using a convolutional neural network ResNet to extract features from the preprocessed MRI image to obtain a feature map;

[0007] Based on the feature map, the final Proposal boxes are generated using the regional generation network RPN;

[0008] Select a region of interest (ROI) in the final proposal boxes, and perform pooling, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and deep target features;

[0009] Using the pre-trained model, classifying and regressing the objects in the region of interest (ROI) according to the deep target features, and determining the target frame and classification label of each target;

[0010] Determining a predicted segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target features;

[0011] Post-processing the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0012] Based on the target frame, classification label, and post-processed mask image of each target, the target frame of each target is subjected to non-maximum suppression processing according to different categories, and the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data are determined.

[0013] In some embodiments, preprocessing the MRI image data to obtain preprocessed MRI image data includes:

[0014] The MRI image data is subjected to Gaussian filtering, histogram equalization, correction and contrast enhancement processing in sequence to obtain the pre-processed MRI image data.

[0015] In some implementations, the generating of the final Proposal boxes using a region generation network RPN based on the feature map includes:

[0016] The feature map is divided into a plurality of regions of the same shape, and a plurality of anchor boxes of different sizes and shapes are preset in each region, each of the anchor boxes representing a position that may contain an object;

[0017] Use the classification network to score each anchor box and evaluate whether each anchor box may contain the target;

[0018] Sort and select each Anchor box according to the score to obtain candidate Proposal boxes;

[0019] The candidate Proposal boxes are screened and merged through the non-maximum suppression NMS algorithm to generate the final Proposal boxes.

[0020] In some embodiments, the selecting a region of interest ROI in the final Proposal boxes, and sequentially performing pooling operations, feature fusion, and feature encoding on the ROI to obtain deep target features includes:

[0021] Among the final Proposal boxes, select the k Proposal boxes with the highest scores as the region of interest ROI that needs to be viewed;

[0022] ROIPooling maps each region of interest ROI to a feature map of a fixed size;

[0023] Perform feature fusion of each fixed-size feature map obtained by ROIPooling with the global image to determine the fused ROI region features;

[0024] The fused ROI regional features are integrated and encoded using a convolutional neural network to obtain a coding feature map and the deep target features.

[0025] In some embodiments, the use of the pre-trained model to classify and regress the objects in the region of interest ROI according to the deep object features to determine the object box and classification label of each object includes:

[0026] Inputting the deep target features into a classification network to determine the classification probability of each target region;

[0027] Performing threshold processing on the classification probability of each target area, and performing category prediction for each target area to determine the target label;

[0028] Multiple ROI regions located on the same target are aggregated to obtain more accurate target positioning, where the target positioning is the target frame.

[0029] In some embodiments, determining a segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target feature includes:

[0030] Upsampling the deep target features to determine an upsampled feature map;

[0031] The encoded feature map and the upsampled feature map are input into the segmentation network model to determine the segmentation mask of each object.

[0032] In some embodiments, the MRI image segmentation method based on the Mask RCNN algorithm further includes:

[0033] Get each MRI image data in the test set;

[0034] Obtaining a predicted segmentation mask for each of the MRI image data;

[0035] Obtaining a true segmentation mask for each of the MRI image data;

[0036] Comparing the predicted segmentation mask of each MRI image data with the true segmentation mask pixel by pixel to determine the similarity;

[0037] determining a target mask score based on the similarity;

[0038] The segmentation network model is optimized and improved according to the object mask score.

[0039] To achieve the above-mentioned object, the second aspect of the present application provides an MRI image segmentation device based on the Mask RCNN algorithm, the device comprising:

[0040] A preprocessing unit, used for preprocessing the MRI image data to obtain preprocessed MRI image data;

[0041] A feature extraction unit, used to extract features from the preprocessed MRI image using a convolutional neural network ResNet to obtain a feature map;

[0042] Proposal boxes generation unit; used to generate final Proposal boxes based on the feature map using a region generation network RPN;

[0043] A deep target feature determination unit is used to select a region of interest ROI in the final Proposal boxes, and perform pooling operations, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and a deep target feature;

[0044] A target classification and bounding box regression unit, used to use a pre-trained model to classify and regress bounding boxes of targets in the region of interest ROI according to the deep target features, and determine a target box and a classification label for each target;

[0045] An object segmentation unit, configured to determine a predicted segmentation mask of each object through a segmentation network model according to the encoded feature map and the deep object features;

[0046] A post-processing unit, configured to post-process the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0047] A result determination unit is used to perform non-maximum suppression processing on the target frame of each target according to different categories based on the target frame, classification label, and post-processed mask image of each target, and determine the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data.

[0048] To achieve the above-mentioned object, the third aspect of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the following steps:

[0049] Preprocessing the MRI image data to obtain preprocessed MRI image data;

[0050] Using a convolutional neural network ResNet to extract features from the preprocessed MRI image to obtain a feature map;

[0051] Based on the feature map, the final Proposal boxes are generated using the regional generation network RPN;

[0052] Select a region of interest (ROI) in the final proposal boxes, and perform pooling, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and deep target features;

[0053] Using the pre-trained model, classifying and regressing the objects in the region of interest (ROI) according to the deep target features, and determining the target frame and classification label of each target;

[0054] Determining a predicted segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target features;

[0055] Post-processing the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0056] Based on the target frame, classification label, and post-processed mask image of each target, the target frame of each target is subjected to non-maximum suppression processing according to different categories, and the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data are determined.

[0057] To achieve the above-mentioned purpose, the fourth aspect of the present application provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0058] Preprocessing the MRI image data to obtain preprocessed MRI image data;

[0059] Using a convolutional neural network ResNet to extract features from the preprocessed MRI image to obtain a feature map;

[0060] Based on the feature map, the final Proposal boxes are generated using the regional generation network RPN;

[0061] Select a region of interest (ROI) in the final proposal boxes, and perform pooling, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and deep target features;

[0062] Using the pre-trained model, classifying and regressing the objects in the region of interest (ROI) according to the deep target features, and determining the target frame and classification label of each target;

[0063] Determining a predicted segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target features;

[0064] Post-processing the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0065] Based on the target frame, classification label, and post-processed mask image of each target, the target frame of each target is subjected to non-maximum suppression processing according to different categories, and the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data are determined.

[0066] The embodiments of the present invention have the following beneficial effects:

[0067] The present invention combines the Mask R-CNN algorithm with the specific needs of MRI image segmentation, integrates target detection, classification and instance segmentation tasks into one framework, and can accurately detect and segment targets in MRI images by using the convolutional neural network ResNet for feature extraction and using RPN to generate high-quality candidate target frames. After generating the region of interest ROI, deep target features are extracted through pooling operations, feature fusion and feature encoding. The deep target features are not only used for target classification and bounding box regression, but also for subsequent segmentation tasks, so that classification, localization and segmentation tasks can share the same feature representation, which improves the efficiency and accuracy of the model, and further improves the deep target The features are upsampled to obtain an upsampled feature map, which is combined with the encoded feature map to generate a predicted segmentation mask through a segmentation network model. Subsequently, the predicted segmentation mask is post-processed to optimize the segmentation result so that the segmentation mask can more accurately reflect the shape and boundary of the target. Finally, according to the target box, classification label and post-processed mask image of each target, the final target box and classification label are determined through non-maximum suppression processing, which helps to eliminate redundant target boxes and improve the accuracy of target detection and segmentation. The present invention automatically learns multi-layer features, which is more flexible and effective. The algorithm can adapt to different data and application scenarios through an adaptive learning process, and has better mobility and universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0069] in:

[0070] Figure 1 1 is a flow chart of an MRI image segmentation method based on the Mask RCNN algorithm in one embodiment;

[0071] Figure 2 is a structural diagram of an MRI image segmentation device based on the Mask RCNN algorithm in one embodiment;

[0072] Figure 3 is a schematic diagram of the structure of a computer device in one embodiment;

[0073] Figure 4 Schematic diagram of the structure of a computer-readable storage medium in one embodiment. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] In the embodiment of the present application, a MRI image segmentation method based on the Mask RCNN algorithm is provided. Figure 1 , Figure 1 4 is a flow chart of an MRI image segmentation method based on the Mask RCNN algorithm in an embodiment. The MRI image segmentation method based on the Mask RCNN algorithm includes steps S1 to S8.

[0076] Step S1, preprocessing the MRI image data to obtain preprocessed MRI image data;

[0077] In some embodiments, preprocessing the MRI image data to obtain preprocessed MRI image data includes:

[0078] The MRI image data is subjected to Gaussian filtering, histogram equalization, correction and contrast enhancement processing in sequence to obtain the pre-processed MRI image data.

[0079] Specifically, the MRI image data are processed in the following order:

[0080] Step S11, Gaussian filtering: MRI images may be interfered by various noises during the acquisition process, such as magnetic field inhomogeneity, motion artifacts during scanning, etc. In order to reduce the interference of these noises on the subsequent segmentation algorithm, the present invention uses a Gaussian filter to smooth the image.

[0081] Step S12, histogram equalization: Sometimes, due to factors such as changes in scanners and scanning conditions, the contrast of MRI images changes or the affected areas become significantly darker or brighter. In order to maintain the overall clarity of the image, the present invention adjusts the image after Gaussian filtering through an image enhancement method using histogram equalization.

[0082] Step S13, correction: MRI images may have geometric errors, such as rotation, displacement, etc., and may also have nonlinear intensity changes. These errors and changes will have a great impact on the correctness of the image segmentation algorithm. Therefore, image correction operations are required. The corrected image has a stable geometric shape and intensity value, thereby improving the accuracy of the segmentation result.

[0083] Step S14, contrast enhancement: The density of each tissue in the local area of ​​the MRI image is similar, which makes it impossible to effectively segment some areas with poor light-dark contrast. Therefore, the present invention uses a contrast enhancement algorithm CL (such as CLAHE) to enhance the light-dark contrast of the image, which will help to effectively extract and segment the target.

[0084] In summary, after MRI image data preprocessing, the noise that affects MRI image segmentation can be removed, the image contrast can be enhanced, and the image geometry can be adjusted, thereby improving the accuracy and stability of MRI image segmentation.

[0085] In some implementations, the MRI image data may be preprocessed using other methods such as spatial transformation and denoising.

[0086] Step S2, using a convolutional neural network ResNet to extract features from the preprocessed MRI image to obtain a feature map;

[0087] Specifically, step S2 provides the feature map required to generate the Anchor box.

[0088] Step S3, based on the feature map, using the region generation network RPN to generate the final proposal boxes;

[0089] In some implementations, generating the final Proposal boxes based on the feature map using a region generation network RPN includes:

[0090] The feature map is divided into a plurality of regions of the same shape, and a plurality of anchor boxes of different sizes and shapes are preset in each region, each of the anchor boxes representing a position that may contain an object;

[0091] Use the classification network to score each anchor box and evaluate whether each anchor box may contain the target;

[0092] Sort and select each Anchor box according to the score to obtain candidate Proposal boxes;

[0093] The candidate Proposal boxes are screened and merged through the non-maximum suppression NMS algorithm to generate the final Proposal boxes.

[0094] Specifically, in RPN, the extracted feature map is first divided into multiple small areas of the same shape, for example, a 3x3 grid, each area corresponds to multiple Anchor boxes, each box represents an area that may contain a target, and the Anchor box is the result of spatial division on the feature map. Each box represents a candidate box of an area that may contain a target. First, by extracting the feature map, the feature map is segmented and divided into multiple areas, each area corresponds to multiple Anchor boxes, and then each Anchor box is scored to evaluate whether each Anchor box may contain a target. According to these scores, the high-scoring Anchor boxes are sorted and selected as candidate Proposal boxes. Finally, these candidate Proposal boxes are further screened and merged through NMS (non-maximum suppression) to obtain the final Proposal boxes, which will cover various target areas in the image. In this process, the Anchor box is the basis for generating Proposal boxes, and the final Proposal boxes are generated and screened through the scoring and NMS process. These final Proposal boxes are important inputs for subsequent target classification and segmentation.

[0095] Further, in some embodiments, the steps of screening and merging the candidate Proposal boxes by the non-maximum suppression NMS algorithm to generate the final Proposal boxes are:

[0096] Step S31, score sorting: First, all Proposal boxes are sorted according to the score of each Proposal box (such as classification score or confidence score). Usually the score is obtained in the output of the RPN network.

[0097] Step S32, select the box with the highest score: starting from the Proposal box with the highest score, mark the box as the final result, and filter out other Proposal boxes whose IoU (intersection over union) with the box is greater than the threshold according to a certain threshold (such as 0.5).

[0098] Step S33, suppress overlapping boxes: For the remaining Proposal boxes, calculate their IoU with the selected final box. If the IoU is greater than the set threshold, remove it from the final Proposal boxes.

[0099] Step S34, repeat the process: Repeat the above process until all Proposal boxes are processed.

[0100] Through the steps of NMS, the final Proposal boxes are obtained, which have the following characteristics:

[0101] First, there will be no proposal boxes with many overlaps, because NMS will suppress the overlapping boxes and only retain the most representative boxes.

[0102] Secondly, it sufficiently covers various target areas in MRI images without any repeated or redundant boxes.

[0103] The final Proposal boxes obtained in this way can be used as input for subsequent target classification and segmentation, providing a basis for subsequent processing.

[0104] It should be noted that Proposal boxes refer to rectangular areas in an image that are predicted by the algorithm to contain target objects. These rectangular areas are used as candidate areas for subsequent processing (such as classification, regression, etc.) to identify specific objects in the image.

[0105] Step S4, selecting a region of interest ROI in the final Proposal boxes, and performing pooling operations, feature fusion, and feature encoding on the ROI in sequence to obtain deep target features;

[0106] In some embodiments, the selecting a region of interest ROI in the final Proposal boxes, and sequentially performing pooling operations, feature fusion, and feature encoding on the ROI to obtain a coded feature map and deep target features, includes:

[0107] Among the final Proposal boxes, select the k Proposal boxes with the highest scores as the region of interest ROI that needs to be viewed;

[0108] ROIPooling maps each region of interest ROI to a feature map of a fixed size;

[0109] Perform feature fusion of each fixed-size feature map obtained by ROIPooling with the global image to determine the fused ROI region features;

[0110] The fused ROI regional features are integrated and encoded using a convolutional neural network to obtain the encoded feature map and the deep target features.

[0111] Specifically, among the final Proposal boxes, k Proposal boxes with the highest scores are selected as the region of interest ROI that needs to be viewed;

[0112] ROIPooling maps each region of interest ROI to a feature map of a fixed size. Specifically, the feature map of the fixed size here can be a size of 2×2 or 3×3.

[0113] Perform feature fusion on each of the fixed-size feature maps obtained by ROIPooling and the global image to determine the fused ROI region features. Specifically, the feature map obtained by ROIPooling is fused with the global image features to retain the local detail information of the ROI region. Here, the global image may be the original MRI image data or the pre-processed MRI image data, that is, the feature map obtained by ROIPooling is fused with the features extracted from the MRI image data. By fusing these two features, the model can comprehensively utilize local and global information to improve the accuracy and robustness of target detection;

[0114] Integrate the fused ROI regional features, use convolutional neural network to encode them, and obtain the coded feature map and the deep target features. It should be noted that "feature encoding" directly produces the coded feature map, and "deep target features" actually refer to the result of the entire processing flow (including ROI selection, ROI pooling, feature fusion and feature encoding). This "deep target feature" not only contains the detailed features of the ROI area, but also undergoes multi-layer abstraction and combination of the network, so it is more suitable for subsequent target classification and positioning tasks. In other words, integrating the fused ROI regional features and encoding them using convolutional neural networks can further extract and enrich target features, improve the accuracy and performance of target detection tasks, and this application records the richer target features as deep target features.

[0115] In summary, the ROI alignment process in step S4 is to make the size and shape of the ROI consistent with the size and shape of the traditional convolutional neural network input, and to make the features of the ROI richer, which is conducive to subsequent classification and segmentation operations.

[0116] Step S5, using the pre-trained model, classifying and regressing the objects in the region of interest ROI according to the deep object features, and determining the object frame and classification label of each object;

[0117] In some embodiments, the use of the pre-trained model to classify and regress the objects in the region of interest ROI according to the deep object features to determine the object box and classification label of each object includes:

[0118] Inputting the deep target features into a classification network to determine the classification probability of each target region;

[0119] Performing threshold processing on the classification probability of each target area, and performing category prediction for each target area to determine the target label;

[0120] Multiple ROI regions located on the same target are aggregated to obtain more accurate target positioning, where the target positioning is the target frame.

[0121] Specifically, the deep target features obtained from step S4 are used as the input of the classification stage to generate classification probabilities for each target region. Many different classifiers can be used here, such as SVM, Softmax, decision trees, etc. By performing threshold processing on the classification probabilities, a category prediction is made for each target region to determine whether it contains the target. After the classification prediction, multiple ROI regions located on the same target are aggregated to obtain more accurate target positioning.

[0122] More specifically, the specific location and classification information of each target that is finally outputted is usually given in the form of a target box (bounding box) and a category label in target detection. In a specific embodiment, it can be the location coordinates and lesion type of the lesion site in the MRI image.

[0123] In summary, step S5 mainly refers to classifying and marking the targets in each ROI area. A variety of classification algorithms can be used, such as SVM, Softmax, etc. At the same time, in order to improve the classification accuracy, some post-processing operations can also be performed in the classification results, such as aggregation and adjustment of the target area.

[0124] Step S6, determining a predicted segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target features;

[0125] In some embodiments, after upsampling the deep target features, determining a segmentation mask for each target through a segmentation network model includes:

[0126] Upsampling the deep target features to determine an upsampled feature map;

[0127] The encoded feature map and the upsampled feature map are input into the segmentation network model to determine the segmentation mask of each object.

[0128] Specifically, an upsampling algorithm is used to enlarge the deep target features to the same size as the original image to retain the detail information of the target area, that is, to determine the upsampling feature map, input the encoded feature map and the upsampled image features into the segmentation network, and output a pixel-level segmentation mask for each target to separate the target area and the background.

[0129] In summary, step S6 is to separate the area containing the target from the background and output a binary foreground mask. Various segmentation algorithms can be used, such as semantic segmentation, instance segmentation, etc. At the same time, in order to achieve better results, post-processing operations can be performed on the segmentation results, such as boundary smoothing, denoising, etc.

[0130] In some embodiments, the MRI image segmentation method based on the Mask RCNN algorithm further includes:

[0131] Get each MRI image data in the test set;

[0132] Obtaining a predicted segmentation mask for each of the MRI image data;

[0133] Obtaining a true segmentation mask for each of the MRI image data;

[0134] Comparing the predicted segmentation mask of each MRI image data with the true segmentation mask pixel by pixel to determine the similarity;

[0135] determining a target mask score based on the similarity;

[0136] The segmentation network model is optimized and improved according to the object mask score.

[0137] Specifically, for each image in the test set, its manually annotated target area first needs to be converted into a ground truth segmentation mask; each image in the test set is processed through steps S1 to S5 and input into the trained segmentation network model, and the generated segmentation mask is used as the predicted segmentation mask; the predicted segmentation mask is compared with the true segmentation mask pixel by pixel, and their similarity is calculated, generally using pixel-level cross entropy loss function or Dice coefficient and other indicators; based on the comparison result, the score of the target mask is calculated, which reflects the performance of the model in target segmentation. Common indicators include pixel-level accuracy, recall rate, F1 value, etc., to understand how the model performs for target segmentation.

[0138] In some embodiments, steps S1 to S8 of the present application can also be understood as a training process of the model, in which case the predicted segmentation mask is the final post-processed segmentation mask of each target in the MRI image data described in step S8. The target mask scoring process shown above is performed in a similar manner, and the scores of these target masks can be used to guide the adjustment and optimization of model training, such as hyperparameter adjustment, network structure improvement, data enhancement strategy, etc. based on the scoring results. When the score reaches a certain pre-set standard, it can be considered that the model has a certain segmentation capability, so the target mask score is part of the model performance evaluation and can be used as an important reference to guide model improvement and optimization, so as to improve the accuracy of the MRI image segmentation method based on the Mask RCNN algorithm in this application.

[0139] Step S7, post-processing the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0140] Specifically, the predicted segmentation mask of each target is visualized as a segmented mask image, and the segmented mask image is post-processed to obtain a post-processed mask image of each target.

[0141] In some embodiments, the specific steps of the post-processing are:

[0142] Step S71, denoising: the segmented mask image is processed by a denoising algorithm to remove noise points and improve the image quality. Depending on the specific situation, a filter based on image brightness (such as a median filter, a Gaussian filter, etc.) or a filter based on image gradient (such as a bilateral filter, a non-local mean filter, etc.) can be used for denoising.

[0143] Step S72, smoothing: Use a smoothing algorithm to smooth the segmentation result to reduce the jaggedness and noise points after segmentation. Commonly used smoothing algorithms include mean filters based on pixel value averages, Gaussian filters based on pixel value weighted averages, etc. Through layer-by-layer convolution or convolution kernel edge padding, etc., sharp edge structures are made smoother.

[0144] Step S73, edge optimization processing: post-processing and optimizing the segmentation results through morphological operations. For elliptical or circular objects, some secondary morphological operations (such as dilation, erosion, etc.) can improve the edges representing their contours, thereby improving the segmentation accuracy.

[0145] In summary, by performing post-processing on the predicted segmentation mask of each target, such as denoising, smoothing and edge optimization, the segmentation accuracy can be improved, which can effectively improve the segmentation accuracy of the Mask RCNN algorithm for MRI images and reduce errors.

[0146] Step S8, based on the target frame, classification label, and post-processed mask image of each target, the target frame of each target is subjected to non-maximum suppression processing according to different categories, and the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data are determined.

[0147] Specifically, for all detection frames, they are grouped according to categories, and an NMS operation is performed for each category; for each category, they are sorted from large to small according to the confidence score; the detection frame with the highest confidence score is selected, and the IOU (Intersection Over Union) is calculated with the remaining detection frames, and the detection frames whose IOU values ​​are greater than a certain threshold are removed from the candidate set; the detection frame with the highest confidence score is repeatedly selected, and the IOU (Intersection Over Union) is calculated with the remaining detection frames, and the detection frames whose IOU values ​​are greater than a certain threshold are removed from the candidate set until all detection frames have been processed once; the remaining detection frames are used as the final output results, and the segmentation mask of each object is output, that is, the final target frame, classification label and post-processed mask image corresponding to each target in the MRI image data, in other words, the specific position, classification information and segmentation mask of each object are finally output, so as to provide accuracy and visualization effect for the final presentation of the MRI image segmentation results.

[0148] In summary, through NMS processing, we can avoid repeated detection of the same target by multiple detection frames, while retaining the detection frames with higher confidence scores. This can improve the accuracy of the MRI image segmentation results of the Mask RCNN algorithm and reduce redundant results caused by overlap.

[0149] The MRI image segmentation method based on the Mask RCNN algorithm proposed in the present invention is used in combination with the Mask RCNN algorithm to perform MRI image segmentation, which can effectively overcome the shortcomings of traditional MRI image segmentation technology that relies on manually designed features, is costly, slow, and has poor adaptability. Compared with traditional MRI image segmentation technology, it has higher accuracy and generalization ability, realizes the simultaneous completion of target detection and segmentation tasks, and helps doctors provide more accurate diagnosis and treatment.

[0150] In the embodiment of the present application, a MRI image segmentation device based on the Mask RCNN algorithm is provided. Figure 2 , Figure 2 A structural diagram of an MRI image segmentation device based on the Mask RCNN algorithm in one embodiment, wherein the MRI image segmentation device based on the Mask RCNN algorithm includes: a preprocessing unit 201, a feature extraction unit 202, a proposal boxes generation unit 203, a deep target feature determination unit 204, a target classification and bounding box regression unit 205, a target segmentation unit 206, a post-processing unit 207 and a result determination unit 208.

[0151] The preprocessing unit 201 is configured to preprocess the MRI image data to obtain preprocessed MRI image data;

[0152] A feature extraction unit 202 is configured to extract features from the preprocessed MRI image using a convolutional neural network ResNet to obtain a feature map;

[0153] Proposal boxes generating unit 203; configured to generate final Proposal boxes based on the feature map using a region generation network RPN;

[0154] The deep target feature determination unit 204 is configured to select a region of interest ROI in the final Proposal boxes, and perform pooling operations, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and a deep target feature;

[0155] The target classification and bounding box regression unit 205 is configured to use a pre-trained model to classify and regress bounding boxes of targets in the region of interest ROI according to the deep target features, and determine a target box and a classification label for each target;

[0156] The object segmentation unit 206 is configured to determine the predicted segmentation mask of each object through a segmentation network model according to the encoded feature map and the deep object features;

[0157] A post-processing unit 207 is configured to post-process the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0158] The result determination unit 208 is configured to perform non-maximum suppression processing on the target frame of each target according to different categories based on the target frame, classification label, and post-processed mask image of each target, and determine the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data.

[0159] In some embodiments, the preprocessing unit 201 is further configured to sequentially perform Gaussian filtering, histogram equalization, correction, and contrast enhancement processing on the MRI image data to obtain the preprocessed MRI image data.

[0160] In some embodiments, the Proposal boxes generating unit 203 is further configured to divide the feature map into a plurality of regions of the same shape, and preset a plurality of Anchor boxes of different sizes and shapes in each region, each of the Anchor boxes representing a position that may contain a target;

[0161] Use the classification network to score each anchor box and evaluate whether each anchor box may contain the target;

[0162] Sort and select each Anchor box according to the score to obtain candidate Proposal boxes;

[0163] The candidate Proposal boxes are screened and merged through the non-maximum suppression NMS algorithm to generate the final Proposal boxes.

[0164] In some implementations, the deep target feature determination unit 204 is further configured to select k Proposal boxes with the highest scores among the final Proposal boxes as the region of interest ROI that needs to be viewed;

[0165] ROIPooling maps each region of interest ROI to a feature map of a fixed size;

[0166] Perform feature fusion of each fixed-size feature map obtained by ROIPooling with the global image to determine the fused ROI region features;

[0167] The fused ROI regional features are integrated and encoded using a convolutional neural network to obtain the encoded feature map and deep target features.

[0168] In some embodiments, the object classification and bounding box regression unit 205 is further configured to input the deep object features into a classification network to determine the classification probability of each object region;

[0169] Performing threshold processing on the classification probability of each target area, and performing category prediction for each target area to determine the target label;

[0170] Multiple ROI regions located on the same target are aggregated to obtain more accurate target positioning, where the target positioning is the target frame.

[0171] In some implementations, the object segmentation unit 206 is further configured to upsample the encoded feature map to determine an upsampled feature map;

[0172] The encoded feature map and the upsampled feature map are input into the segmentation network model to determine the segmentation mask of each object.

[0173] In some embodiments, the MRI image segmentation apparatus based on the Mask RCNN algorithm further includes a model optimization module 209, wherein the model optimization module 209 is configured to obtain each MRI image data in the test set;

[0174] Obtaining a predicted segmentation mask for each of the MRI image data;

[0175] Obtaining a true segmentation mask for each of the MRI image data;

[0176] Comparing the predicted segmentation mask of each MRI image data with the true segmentation mask pixel by pixel to determine the similarity;

[0177] determining a target mask score based on the similarity;

[0178] The segmentation network model is optimized and improved according to the object mask score.

[0179] For other details about how each module in the MRI image segmentation device based on the Mask RCNN algorithm implements the above technical solution, please refer to the description of the MRI image segmentation method based on the Mask RCNN algorithm provided above, which will not be repeated here.

[0180] In an embodiment of the present application, a computer device is provided. Figure 3 , Figure 3 3 is a schematic diagram of the structure of a computer device in an embodiment. The device includes a memory 301 and a processor 302. The memory 301 stores a computer program. When the computer program is executed by the processor 302, the processor 302 performs the following steps:

[0181] Preprocessing the MRI image data to obtain preprocessed MRI image data;

[0182] Using a convolutional neural network ResNet to extract features from the preprocessed MRI image to obtain a feature map;

[0183] Based on the feature map, the final Proposal boxes are generated using the regional generation network RPN;

[0184] Select a region of interest (ROI) in the final proposal boxes, and perform pooling, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and deep target features;

[0185] Using the pre-trained model, classifying and regressing the objects in the region of interest (ROI) according to the deep target features, and determining the target frame and classification label of each target;

[0186] Determining a predicted segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target features;

[0187] Post-processing the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0188] Based on the target frame, classification label, and post-processed mask image of each target, the target frame of each target is subjected to non-maximum suppression processing according to different categories, and the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data are determined.

[0189] Among them, the processor 302 can also be called a CPU (Central Processing Unit), and the processor 302 may be an integrated circuit chip with signal processing capabilities; the processor 302 can also be a general-purpose processor, DSP (Digital Signal Process), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, among which the general-purpose processor can be a microprocessor or the processor 302 can also be any conventional processor, etc.

[0190] In an embodiment of the present application, a computer readable storage medium is provided. Figure 4 , Figure 4 The structure diagram of a computer-readable storage medium in an embodiment is a schematic diagram, on which a readable computer program 401 is stored; wherein the computer program 401 may be stored in the above storage medium in the form of a software product, and includes a number of instructions for causing a computer device (which may be a personal computer, a server machine, or a network device, etc.) or a processor to perform the following steps:

[0191] Preprocessing the MRI image data to obtain preprocessed MRI image data;

[0192] Using a convolutional neural network ResNet to extract features from the preprocessed MRI image to obtain a feature map;

[0193] Based on the feature map, the final Proposal boxes are generated using the regional generation network RPN;

[0194] Select a region of interest (ROI) in the final proposal boxes, and perform pooling, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and deep target features;

[0195] Using the pre-trained model, classifying and regressing the objects in the region of interest (ROI) according to the deep target features, and determining the target frame and classification label of each target;

[0196] Determining a predicted segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target features;

[0197] Post-processing the predicted segmentation mask of each target to obtain a post-processed mask image of each target;

[0198] Based on the target frame, classification label, and post-processed mask image of each target, the target frame of each target is subjected to non-maximum suppression processing according to different categories, and the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data are determined.

[0199] The aforementioned storage media include: U disk, mobile hard disk, magnetic disk or CD, ROM (Read-Only Memory), RAM (Random Access Memory) and other media that can store program code, or terminal devices such as computers, service machines, mobile phones, and tablets.

[0200] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0201] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0202] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A MRI image segmentation method based on Mask RCNN algorithm, characterized in that: The method includes: Preprocessing the MRI image data to obtain preprocessed MRI image data; Using a convolutional neural network ResNet to extract features from the preprocessed MRI image to obtain a feature map; Based on the feature map, the final Proposalboxes are generated using the region generation network RPN; Select a region of interest (ROI) in the final proposal boxes, and perform pooling, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and deep target features; Using the pre-trained model, classifying and regressing the targets in the region of interest (ROI) according to the deep target features, and determining the target box and classification label of each target; Determining a predicted segmentation mask for each target through a segmentation network model according to the encoded feature map and the deep target features; Post-processing the predicted segmentation mask of each target to obtain a post-processed mask image of each target; Based on the target frame, classification label, and post-processed mask image of each target, the target frame of each target is subjected to non-maximum suppression processing according to different categories, and the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data are determined.

2. The MRI image segmentation method based on the Mask RCNN algorithm according to claim 1, characterized in that: The preprocessing of the MRI image data to obtain the preprocessed MRI image data includes: The MRI image data is subjected to Gaussian filtering, histogram equalization, correction and contrast enhancement processing in sequence to obtain the pre-processed MRI image data.

3. The MRI image segmentation method based on the Mask RCNN algorithm according to claim 2, characterized in that: Based on the feature map, the final Proposal boxes are generated using the regional generation network RPN, including: The feature map is divided into a plurality of regions of the same shape, and a plurality of anchor boxes of different sizes and shapes are preset in each region, each of the anchor boxes representing a position that may contain an object; Use the classification network to score each anchor box and evaluate whether each anchor box may contain the target; Sort and select each Anchor box according to the score to obtain candidate Proposal boxes; The candidate Proposal boxes are screened and merged through the non-maximum suppression NMS algorithm to generate the final Proposal boxes.

4. The MRI image segmentation method based on the Mask RCNN algorithm according to claim 3, characterized in that: The process of selecting a region of interest (ROI) in the final proposal boxes, and sequentially performing pooling operations, feature fusion, and feature encoding on the ROI to obtain a coded feature map and deep target features includes: Among the final Proposal boxes, select the k Proposal boxes with the highest scores as the region of interest ROI that needs to be viewed; ROIPooling maps each region of interest ROI to a feature map of a fixed size; Perform feature fusion of each fixed-size feature map obtained by ROIPooling with the global image to determine the fused ROI region features; The fused ROI regional features are integrated and encoded using a convolutional neural network to obtain the encoded feature map and the deep target features.

5. The MRI image segmentation method based on the Mask RCNN algorithm according to claim 4, characterized in that: Using the pre-trained model, the targets in the region of interest ROI are classified and bounding box regressed according to the deep target features, and the target box and classification label of each target are determined, including: Inputting the deep target features into a classification network to determine the classification probability of each target region; Performing threshold processing on the classification probability of each target area, and performing category prediction for each target area to determine the target label; Multiple ROI regions located on the same target are aggregated to obtain more accurate target positioning, where the target positioning is the target frame.

6. The MRI image segmentation method based on the Mask RCNN algorithm according to claim 5, characterized in that: Determining the segmentation mask of each target through a segmentation network model according to the encoded feature map and the deep target feature includes: Upsampling the deep target features to determine an upsampled feature map; The encoded feature map and the upsampled feature map are input into the segmentation network model to determine the segmentation mask of each object.

7. The MRI image segmentation method based on the Mask RCNN algorithm according to claim 6, characterized in that: The MRI image segmentation method based on the Mask RCNN algorithm also includes: Get each MRI image data in the test set; Obtaining a predicted segmentation mask for each of the MRI image data; Obtaining a true segmentation mask for each of the MRI image data; Comparing the predicted segmentation mask of each MRI image data with the true segmentation mask pixel by pixel to determine the similarity; determining a target mask score based on the similarity; The segmentation network model is optimized and improved according to the object mask score.

8. An MRI image segmentation device based on the Mask RCNN algorithm, characterized in that: The device includes: A preprocessing unit, used for preprocessing the MRI image data to obtain preprocessed MRI image data; A feature extraction unit, used to extract features from the preprocessed MRI image using a convolutional neural network ResNet to obtain a feature map; Proposal boxes generation unit; used to generate final Proposal boxes based on the feature map using a region generation network RPN; A deep target feature determination unit is used to select a region of interest ROI in the final Proposal boxes, and perform pooling operations, feature fusion, and feature encoding on the ROI in sequence to obtain a coded feature map and a deep target feature; A target classification and bounding box regression unit, used to use a pre-trained model to classify and regress bounding boxes of targets in the region of interest ROI according to the deep target features, and determine a target box and a classification label for each target; An object segmentation unit, which determines a predicted segmentation mask of each object through a segmentation network model according to the encoded feature map and the deep object features; A post-processing unit, configured to post-process the predicted segmentation mask of each target to obtain a post-processed mask image of each target; A result determination unit is used to perform non-maximum suppression processing on the target frame of each target according to different categories based on the target frame, classification label, and post-processed mask image of each target, and determine the final target frame, classification label, and post-processed mask image corresponding to each target in the MRI image data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

Images