Placenta accreta MRI sign detection and classification method and device based on deep neural network

Abstract

This invention discloses a method and device for detecting and classifying MRI signs of placenta accreta based on a deep neural network. The method comprises an MRI image preprocessing module and an MRI placenta accreta sign region detection and classification module. The MRI placenta accreta sign region detection and classification module comprises a feature extraction module, a sign region generation module, a false-positive region reduction module, and a mask generation module. A deep neural network-based model for detecting and classifying MRI sign regions of placenta accreta is constructed. Based on medical MRI image data, the model uses a small amount of annotation information and data processing as input. The model output contains a three-dimensional bounding box containing the sign region and the category to which the sign belongs. This model can be used to detect and classify MRI sign regions of placenta accreta. The invention is scalable, supporting the modification of sign types and the processing and classification of a wide range of three-dimensional data types.

Description

Technical Field

[0001] The present invention belongs to the field of medical information technology, and relates to medical image processing, image disease area detection and classification technology, and in particular to a method and device for detecting and classifying placenta accreta MRI signs based on a deep neural network. Background Art

[0002] In the interdisciplinary field of medical information technology, medical images can usually be obtained by ultrasound angiography or magnetic resonance imaging, which can be used for auxiliary diagnosis. For the evaluation of MRI signs of placenta accreta, existing work includes the use of MRI-based scoring models to assess the risk of invasive placenta accreta and adverse outcomes (Chen Lian et al., The value of MRI-based scoring models in predicting invasive placenta accreta and adverse clinical outcomes [J]. Chinese Journal of Perinatal Medicine, 24(01):32-39. 2021). However, the current evaluation technology based on magnetic resonance imaging relies on the professional knowledge of senior imaging physicians and is difficult to efficiently process the growing large-scale imaging data.

[0003] Recently, some studies have used radiomics technology to predict the amount of bleeding caused by placenta implantation by manually outlining the placenta region in the image (QINGXIA WU et al., Radiomics analysis of placenta on T2WI facilitates prediction of postpartum hemorrhage: A multicentre study [J / OL]. EBioMedicine, 50: 355-365. 2019). However, this method uses the method of manually labeling the placenta organ region in the image, which is inefficient; and the low-order features of the extracted image are used to obtain prediction results through linear models, which has limited representation capabilities. There are also studies that use automated processes for image processing and regional prediction. First, the placenta region of some cases is manually outlined, and then the nnU-Net model is used (ISENSEE JAEGER et al., nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation [J / OL]. Nature Methods, 18 (2): 203-211.

[0004] https: / / doi.org / 10.1038 / s41592-020-01008-z, 2021) to segment the image organ area; then, based on this, an imaging omics prediction model is constructed to automate the prediction process, but the prediction effect of this work still needs to be improved.

[0005] In recent years, image processing methods based on deep convolutional neural networks have been used in computer-assisted medicine to leverage massive amounts of imaging data, resulting in numerous applications in areas such as feature extraction and lesion detection. However, no models or methods currently exist that can automatically extract and classify MRI signs of placenta accreta. Summary of the Invention

[0006] To overcome the shortcomings of the aforementioned prior art, the present invention provides a method and apparatus for detecting and classifying placenta accreta MRI signs based on a deep neural network. The method comprises sign prediction model construction, model training, and model prediction techniques. The deep neural network-based placenta accreta MRI sign region detection and classification model, constructed for medical MRI image data, uses a small amount of annotation information and simple data processing to output a three-dimensional bounding box containing the sign region and the category to which the sign belongs. This model can be used to detect and classify placenta accreta MRI sign regions. The present invention is scalable and supports modification of sign types and a wide range of three-dimensional data types.

[0007] The technical solutions of the present invention are as follows:

[0008] A method for detecting and classifying MRI sign regions of placenta accreta based on a deep neural network mainly includes three parts: data preprocessing, model prediction, and post-processing of the prediction results. The method is constructed based on a deep neural network. The model input is a preprocessed three-dimensional MRI image. A multi-layer convolutional neural network encoder and decoder are used to obtain features containing multi-scale information. Multiple prediction modules are then used to obtain the predicted bounding box of the sign region and the sign category information. The method mainly includes the following steps:

[0009] A. Preprocess the MRI image as the model input;

[0010] Preprocessing includes grayscale normalization, cropping, and other operations. Set the image size and ensure that the image size of the model input is (1, 24, 128, 128) (the four tuples represent: number of channels, depth, height, and width).

[0011] A deep neural network-based placenta accreta MRI sign area detection and classification model was constructed and trained. The model consists of four main modules: feature extraction module, sign area generation module, false positive area reduction module and mask generation module; among them, the sign area generation module, false positive area reduction module and mask generation module are prediction modules.

[0012] The feature extraction module consists of a basic convolutional block (containing two convolutional layers, denoted as C1), three maximum pooling layers, seven residual convolutional blocks with channel attention (denoted as C2 to C8), three concatenation operations, and three transposed convolutional layers. This module extracts and combines features from the input image to generate high-quality feature features that serve as input to other modules. The portion of the feature extraction module that downsamples the input image is called the encoder (including convolutional blocks C1 to C4 and the maximum pooling layer), while the portion that restores the downsampled features to their original size is called the decoder (including convolutional blocks C5 to C8 and the transposed convolutional layer).

[0013] The structure of the feature region generation module includes two 1×1 convolution operations, which generate feature region bounding box coordinates and length predictions, as well as confidence predictions for the input features.

[0014] The false positive region reduction module consists of a feature map cropping operation, a region of interest pooling layer, and two parallel fully connected layers. The region of interest pooling layer pools features of different spatial shapes into a uniform shape. After passing through the fully connected layer, it generates category information for the predicted features and fine-tunes the position of the region bounding box.

[0015] The structure of the mask generation module includes: three feature cropping operations, two upsampling layers and two feature splicing operations, and the final connected Sigmoid activation layer.

[0016] Model construction and training include steps B to E:

[0017] B. Constructing an MRI image feature extraction module to extract the MRI sign features or feature maps of placenta accreta;

[0018] In the present invention, the convolutional neural network is mainly used to extract image features and combine the features to generate high-level semantic features. Low-level features mainly refer to features such as image texture and corner points, and high-level features refer to features with certain visual meanings, such as various signs. During model training, the input data passes through the MRI image feature extraction module, where the basic convolution block C1 extracts low-level features in the image. Afterwards, seven residual convolution blocks with channel attention, denoted as C2 to C8, are used to combine the image features to generate high-level features as input to the subsequent prediction module. This module also uses splicing operations and transposed convolution layers to fuse features at different levels to achieve better characterization effects;

[0019] C builds and trains a sign region generation module, which uses the placenta accreta MRI sign features as input to obtain the 3D bounding box sign region and the corresponding confidence level;

[0020] The feature region generation module takes the placenta accreta MRI feature map extracted by the MRI image feature extraction module as input and generates six 3D bounding boxes and their corresponding confidence scores at each location of the input feature through two 1×1 convolution operations. Each 3D bounding box is represented by the center point coordinates and six numbers: length, width, and height.

[0021] D. Build and train a false positive region reduction module to classify and predict the category of the sign area, suppress the non-sign area, and fine-tune the position and size of the bounding box;

[0022] The false positive region reduction module further classifies the above bounding box areas to reduce false positive sign predictions. Specifically, the above bounding boxes are regarded as regions of interest, cropped on the feature map of the lower level (i.e., the output feature map of the residual convolution block C2), and the region of interest pooling (RoI Pooling) operation is used to make them of the same size as the input of the fully connected layer. In this part, the bounding box containing the sign area (also called the foreground bounding box) is classified to determine the sign category, and the bounding box coordinates and length, width and height information are fine-tuned to obtain the corrected bounding box;

[0023] E. Build and train the mask generation module, which segments the feature shape within the input bounding box through multiple upsampling and splicing operations and outputs the feature mask;

[0024] To further improve detection accuracy, the grayscale and shape information within the feature region is introduced and trained through image segmentation. The mask generation module crops the corresponding region of the corrected bounding box from the low-level features (output features of the C1 module) and the input MRI image. The original image pixels are used as the target to supervise the training of the mask generation module. The mask generation module uses the cropped feature map as input to generate a shape mask prediction of the feature, and then generates more accurate bounding box information based on the mask.

[0025] F uses the trained placenta accreta MRI sign region detection and classification model, inputs the MRI image to be detected for classification prediction, and obtains the placenta accreta MRI sign region category.

[0026] Further, step A is specifically as follows:

[0027] 1) Traverse every MRI image I in the entire dataset i , read the window width and window position information in the medical image DICOM format file, denoted as c i and w i , performing grayscale normalization on the MRI image; manually annotating the MRI image training data, including marking the feature area by a bounding box, or outlining the feature shape by contour;

[0028] 2) Traverse the training image set annotations, analyze the position and size of the annotation bounding box (i.e., the center point coordinates (z, y, x) and (d, h, w) depth, length, and width information), and calculate the median and variance statistics of the position and size of the annotation bounding box;

[0029] 3) Perform a random cropping operation on the original image using Gaussian weighted sampling, with the crop size ranging from 0.8 to 1.2 times the target size. The cropped data region is then resized to the target size. The cropped data region is flipped horizontally or vertically with a probability of 50%. The target size here is (1, 24, 128, 128), and insufficient areas are filled by padding. The same operation is performed on the bounding box annotations. The purpose of this step is to augment the dataset and improve the model's generalization ability.

[0030] Further, step B is specifically as follows:

[0031] 1) First, it passes through a basic convolutional block, which consists of two convolution operations followed by batch normalization and nonlinear activation layers. This stage extracts the basic features of the lower level of the image. The feature map size becomes (24, 24, 64, 64);

[0032] 2) The network then passes through three residual convolutional blocks with channel-wise attention, labeled C2, C3, and C4. Each module consists of three residual layers. The residual layers incorporate a channel-wise attention mechanism that weights input features in the channel dimension to improve the model's feature combination and representation capabilities.

[0033] 3) Upsampling is performed through two transposed convolution operations and concatenated with the intermediate features obtained by the encoder to obtain a feature map containing multi-scale information with a size of (128, 12, 32, 32).

[0034] Further, step C is specifically as follows:

[0035] 1) Based on this feature map, two 1×1 convolution operations are performed to generate the coordinates and length, width, and height information of the predicted bounding box at each point, as well as the confidence that the bounding box contains the feature area; this step only calculates whether the bounding box contains the feature, and the bounding box features are further classified in step D.

[0036] 2) The supervision information for model training is given by predefined anchor boxes, Figure 4 The anchor box generation mechanism is demonstrated. Specifically, six cubes of size (5, 10, 20, 30, 40, 50) are set based on the distribution of data annotations to cover MRI image symptoms of different sizes. Anchor boxes are assigned values ​​based on their intersection-over-union (IoU) with the ground-truth bounding box, transforming the detection problem into a classification and regression problem.

[0037] 3) Decode the predicted offset and suppress redundant boxes (non-maximum suppression) based on the confidence level to obtain a higher-quality bounding box.

[0038] Further, step D is specifically as follows:

[0039] 1) Using the predicted bounding box generated in step C as input, a cropping operation is performed on the feature map after C2. Since the sizes of the cropped feature maps are different, a region of interest pooling operation is used to obtain a fixed-size feature map as the input of the fully connected layer;

[0040] 2) Two fully connected layers further classify the cropped feature maps based on the feature input. The classification threshold in this step is set more strictly, and the specific category to which each bounding box foreground belongs needs to be classified;

[0041] 3) This step may face the problem of sample imbalance. Through online difficult sample mining technology, the loss functions of the samples are sorted and the difficult samples are studied in detail to improve the network's discrimination ability.

[0042] 4) Average the confidence of the bounding box obtained in this step and the bounding box in step B to obtain the corrected bounding box.

[0043] Further, step E is specifically as follows:

[0044] 1) Using the corrected bounding box obtained in step D as input, perform cropping operations on the deep features (after C8), shallow features (after C1), and input image to extract feature maps in the corresponding areas;

[0045] 2) Using the progressive upsampling operation mentioned in the U-Net model structure, the features of different scales are spliced ​​and fused, and the output of the original image size is gradually obtained;

[0046] 3) Use the mask of the corresponding area of ​​the original image as the target to guide model training. If the mask information is manually annotated, the mask information is used as the supervisory signal; if the annotated mask information is not provided, all pixels in the bounding box are considered as positive sample pixels to optimize model training. Therefore, the model can use part of the annotated information to guide model training;

[0047] 4) This step can output a mask of the implanted features, including shape information. A more compact prediction bounding box can be derived based on the circumscribed cuboid of this mask.

[0048] In specific implementation, the present invention implements a device for detecting and classifying signs of placenta accreta from magnetic resonance imaging (MRI), comprising an MRI image preprocessing module and an MRI placenta accreta sign region detection and classification model module. The MRI placenta accreta sign region detection and classification model module comprises a feature extraction module, a sign region generation module, a false positive region reduction module, and a mask generation module. The feature extraction module, primarily composed of convolutional layers, normalization layers, and activation layers, represents the bulk of the model parameters and extracts feature information from the input image data. The sign region generation module generates a large number of 3D bounding boxes based on the input features, along with confidence scores for the sign regions within the bounding boxes. The bounding boxes with high confidence scores and the image features of the corresponding regions are input to the false positive reduction module and the mask generation module. The false positive reduction module further predicts the sign category and suppresses non-signal regions. This module can also fine-tune the position and size of the bounding boxes. The mask generation module segments the sign shape within the input bounding boxes through multiple upsampling and concatenation operations, outputting a sign mask. The device provided by the present invention can predict the sign areas related to placenta accreta based on the input placenta MRI image, providing automated assistance for placenta accreta diagnosis.

[0049] Compared with the prior art, the present invention has the following beneficial technical effects:

[0050] The present invention provides a method and device for detecting and classifying placenta accreta MRI signs based on a deep neural network. By constructing a nuclear magnetic resonance imaging disease detection model based on a deep convolutional neural network and efficient channel attention, the method realizes the detection and classification of three-dimensional areas of MRI signs of placenta accreta based on multi-task learning and multi-stage integrated prediction.

[0051] Specifically implemented, a device for detecting and classifying signs of placenta accreta in MRI images is implemented, including an MRI image preprocessing module and an MRI placenta accreta sign region detection and classification model module. Based on medical MRI image data, the device uses a small amount of annotation information and simple data processing to construct a sign prediction model, train the model, and then use it to output a three-dimensional bounding box containing the sign region and the category to which the sign belongs. This can be used to detect and classify MRI sign regions of placenta accreta, further assisting in disease diagnosis. The invention is scalable, supporting the modification of sign types and the processing and classification detection of a wide range of three-dimensional application data types. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is the overall model framework and prediction process diagram of the present invention;

[0053] The model input is a preprocessed 3D MRI image. Through an encoder and decoder composed of a multi-layer convolutional neural network, features containing multi-scale information are obtained. Multiple prediction modules then generate predicted bounding boxes for the feature regions and feature categories. The numbered cubes in the figure represent the input image or feature map. The numbers represent their shape information during the model execution. The numbers on the left indicate depth, the numbers on the lower left indicate length and width, and the numbers on the upper right indicate the number of channels. If the depth is not marked on the left, it means the depth is equal to the length and width.

[0054] Figure 2 This is a schematic diagram of the composition and implementation of the efficient channel attention module used in the present invention.

[0055] Figure 3 This is the structure and operation mode of the residual convolution module in the present invention.

[0056] Figure 4 This is a schematic diagram of generating a three-dimensional anchor box of the model supervision information in the present invention;

[0057] Among them, according to the data annotation distribution, 6 bounding boxes of different scales are set at each point to cover all disease areas as much as possible to avoid missed detection. DETAILED DESCRIPTION

[0058] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings by way of examples:

[0059] The present invention provides a method and device for detecting and classifying MRI signs of placenta accreta based on a deep neural network. Deep learning methods are used to detect and classify key sign areas in MRI images, which can provide better interpretability and application value, and can improve efficiency and accuracy.

[0060] Figure 1 The overall model framework and classification prediction process of the present invention are shown. The nuclear magnetic image region detection and classification method based on deep neural network provided by the present invention can be used for placenta accreta MRI image sign region detection and classification. The specific implementation includes the following steps:

[0061] (1) Preprocessing of MRI images;

[0062] Traverse each MRI image I in the entire dataset i , read the window width and window position information in the DICOM format file of medical image data, recorded as c i and w i, perform grayscale normalization on the MRI image. Since the grayscale range of MRI images is wide, which exceeds the range that the human eye can view, the window width and window position indicate the grayscale range used by the doctor when viewing the current image. The grayscale that exceeds the grayscale range is cropped, and the remaining grayscale values ​​are normalized. Placental MRI scan images are usually 512×512 in size, with a frame number of about 30 to 80. It is not advisable to directly input the original size image into the model, and there are also many non-organ areas in the MRI scan images. In fact, the distribution of disease areas is relatively concentrated. Manually annotate the MRI image training data, and the annotation information includes marking the sign area with a bounding box, or outlining the sign shape with a contour.

[0063] The annotations of the MRI training set are traversed. Based on the center point coordinates (z, y, x) and the depth, length, and width (d, h, w) of all annotated bounding boxes, the median and variance are calculated as the mean and variance of Gaussian-weighted sampling. The original image is randomly cropped, with the crop size ranging from 0.8 to 1.2 times the target size, and then scaled to the target size. The cropped data region is flipped horizontally or vertically with a certain probability (e.g., 50%) to enhance the image. The target size of the cropped region from the input image is (1, 24, 128, 128) as the model input, which covers most of the diseased area and the surrounding organ background. Insufficient areas are filled in. Similarly, the bounding box annotations are subjected to the same random cropping, scaling, flipping, and other geometric transformations. This effectively expands the training dataset (by approximately 500 times) and improves the model's generalization ability.

[0064] (2) Constructing an image sign region detection and classification model, including: a feature extraction module, a sign region generation module, a false positive region reduction module, and a mask generation module;

[0065] The first is the overall process from the preprocessed MRI image to the bounding box of the sign area. The instance data is an image with a shape of (1, 24, 128, 128) after cropping and enhancement. The shapes of all images and intermediate features in the overall process are represented by the above-mentioned quaternion, representing the number of channels, depth, height and width. The input image first passes through a basic convolution block consisting of two layers of convolution. Each convolution layer is followed by a batch normalization and nonlinear activation layer. This stage extracts the basic features of the lower level in the image. The feature map size becomes (24, 24, 64, 64). Here, the model is downsampled in the length and width dimensions, but the depth dimension remains unchanged, and the number of channels is increased to 24 to ensure that it contains rich semantic information. It is then passed through a residual convolution block composed of residual layers, with a structure as shown below. Figure 2As shown. The residual convolution block does not change the size of the data, but adjusts the channel dimension. By introducing residual connections, the gradient vanishing problem during training can be alleviated and the representation ability can be enhanced. The channel attention module used in the present invention weights the channel dimension of the input features, dynamically screens out channel features with representation capabilities, and uses one-dimensional convolution to calculate the channel position weights. It avoids the dimension reduction operation in other attention mechanisms, so it has better representation capabilities, and the computational efficiency of one-dimensional convolution is very high. The size of its convolution kernel is determined by the number of input channels and is given by formula (1). The basic idea of ​​formula (1) is to ensure that the convolution kernel size k increases monotonically with the number of input channels C. Since the number of channels C is usually a power of 2, log2 is used here for processing.

[0066]

[0067] Among them, k is the one-dimensional convolution kernel size of the aggregated channel features; C is the number of input feature channels; log2 represents the logarithmic operation with a base of 2; γ and b are hyperparameters that control the incremental amplitude and intercept; the function |·|odd means taking the closest odd number.

[0068] After the residual convolution block, the input spatial scale is downsampled using a max pooling operation, downsampling the input to 1 / 2 of the input. Therefore, the feature map size after passing through the encoder is (64, 3, 8, 8). Next, the decoder is composed of transposed convolution and residual convolution blocks. Transposed convolution can be considered the inverse of convolution. By setting an appropriate step size, the feature map size can be increased, achieving upsampling. Here, some intermediate features from the encoder layer are introduced into the decoder through skip connections, combined through concatenation, and then fused through the residual convolution block, giving the feature map semantic information at different levels. After two upsampling operations, the resulting feature map size is (128, 24, 32, 32).

[0069] The input feature map is subjected to two 1×1 convolution operations, generating 6 bounding box vectors and corresponding confidence scalars at each position in the feature map. The bounding box vector consists of (z, y, x, d, h, w), which respectively represent the center point coordinates and depth, length, and width offset information of the bounding box (prediction box). Therefore, in this stage, the feature region generation module predicts and generates a total of 24×32×32×6=147,456 bounding box vectors and confidences. The supervision information for feature region generation comes from the predefined anchor box, which is generated as follows: through a sliding window, 6 cubic boxes of different sizes are generated for each position in the feature map, recorded as anchor boxes, and assigned values ​​according to their intersection-over-union ratio with the true box. The boxes with an intersection-over-union ratio greater than 0.5 are set as foreground bounding boxes, and the corresponding category information and regression targets are annotated to supervise model training. The regression targets for supervised model training here are the center point coordinates (z, y, x) and the offsets of depth, length, and width (d, h, w), which are defined as shown in formula (2):

[0070]

[0071] Among them t. * Indicates the target value that the network needs to predict, z a ,y a , x a , d a , h a , w a Respectively represent the center coordinates and depth, length, and width of the current anchor box, and z, y, x, d, h, and w represent the center coordinates and depth, length, and width of the real box that matches the anchor box. The target value is processed by difference and logarithm, which makes the network easier to learn and the training more stable. Finally, according to the confidence ranking of the predicted bounding boxes, redundant boxes with a large overlap are suppressed. This process is called non-maximum suppression, resulting in a higher-quality bounding box (prediction box). However, the number of bounding boxes obtained is still large, around 500 to 1000, and there are a large number of false positive areas, so the false positive area reduction module is also needed for fine-tuning and optimization.

[0072] The resulting predicted box is used as input and cropped on lower-level features (the output features of the C2 module). Region of Interest Pooling (RoI Pooling) is then used to normalize the cropped bounding box to a fixed size for input to the fully connected classification layer. For example, if the input bounding box is (6, 20, 23), RoI Pooling will produce a fixed-size input (e.g., (3, 7, 7)). This module further classifies the cropped region into the corresponding disease category and adjusts the corresponding confidence score and regression offset information. Because a higher threshold is set for the target anchor box, the quality is higher and a large number of false positive samples are eliminated. Finally, the confidence scores of the bounding box obtained in this step are averaged with those of the input bounding box to obtain the revised predicted box. Cropping and classification are performed on low-level features to ensure that the results obtained in this step are different from those of the original branch, thereby achieving better results in sign prediction.

[0073] Using the corrected prediction frame, cropping operations are performed on the deep features (output of the C8 module), shallow features (output of the C1 module) and input image respectively to extract the feature maps in the corresponding areas. Then, the features of different scales are gradually upsampled through cubic interpolation, and the above features are spliced ​​and fused to obtain an output of the same size as the original image. If the sample contains manually annotated mask information, the mask information is used as a supervisory signal to assist model training. If the annotated mask information is not provided, all pixels of the bounding box are regarded as positive sample pixels. Therefore, the model can use part of the annotation information to guide model training. This stage can output a mask of the implanted signs, including shape, volume and other information. Finally, a more compact prediction bounding box can be obtained based on the circumscribed cuboid of this mask;

[0074] (3) Model training

[0075] Given an input annotation, the model needs to calculate the loss function and optimize the model parameters by gradient back propagation. This process is called model training. The three prediction modules in the model of the present invention all share the image features generated by the feature extraction module and optimize the parameters therein. Therefore, the overall loss function is composed of the loss functions of each module, as shown in formula (3). rpn , L fpr and L mask They represent the loss functions of the feature region generation module, the false positive region reduction branch, and the mask generation branch, respectively.

[0076] L=L rpn +L fpr +L mask (3)

[0077] Among them, the loss L of the sign area generation rpnand false positive reduction loss L fpr Using the same calculation function, it consists of two parts, namely classification loss L cls and regression loss L reg The classification loss uses the cross entropy function, and the regression loss uses the Smooth-L1 loss function. See formulas (4)(5)(6) respectively:

[0078]

[0079]

[0080]

[0081] Among them, p i and Represent the true label and prediction probability of the i-th prediction box, t i and Represent the true value and predicted value of the i-th regression term (regression offset); N cls and L reg Respectively represent the total number of classification items and regression items involved in the calculation. In the present invention, N cls is the number of prediction boxes. Since each prediction box contains 6 regression items to represent coordinate information, L reg Yes N cls λ is used to control the weight of classification and regression loss in the overall loss.

[0082] For the mask generation module branch, the loss function used is Soft Dice loss, and the calculation method is shown in formulas (7) and (8).

[0083]

[0084]

[0085] Among them, D(m i , g i ) indicates that Soft Dice calculation is performed on each predicted mask and the real mask, indicating the similarity between the two masks. i and g i Represents the i-th predicted mask and the true mask. m ij and g ij Represents m i and g i The predicted value and true value of the jth position in the numerator are 1 if the corresponding positions are the same. The larger the overall value, the smaller the value. N represents the number of predicted bounding boxes. vRepresents the number of voxels in the bounding box. In the present invention, formula (8) improves the existing Soft Dice loss calculation process. The present invention uses two parameters α and β to regulate the false positive and true positive ratios when the predicted value matches the true value, making the model easier to train.

[0086] The above loss function measures the degree of consistency between the model prediction results and the true annotations. The higher the consistency, the smaller the loss function. The training goal of the model is to optimize the parameters of each module in the model to minimize the value of the loss function. In order to minimize the loss function, the training method of the model is to start from the loss function and calculate the gradient information of the loss to the parameters for each module of the input part layer by layer. This step is called gradient back propagation. And the parameters are updated using the gradient information, which is called parameter update. The training process is to repeat the above loss calculation, gradient back propagation and gradient update processes many times until the model reaches the target training round and the output prediction results are well consistent with the labeled data. In addition, since the model in the present invention adopts a multi-branch structure, the task difficulty of different branches is different, and there is a certain cascade relationship. Therefore, during the training process, the candidate area generation branch is trained first, the false positive area reduction branch is trained later, and the mask area generation branch is trained last, so as to ensure that the model can continuously produce high-quality prediction results.

[0087] (IV) Model prediction

[0088] After the model training is completed, the model parameters can be fixed and the inference prediction process can be entered. For input data of any size, if it is forcibly scaled to the target size required during model training, which is (1, 24, 128, 128) in this case, or the corresponding size area is randomly cropped as the model input, the best prediction effect cannot be achieved. In order to avoid missed detection and maximize the detection effect of the disease area, the present invention uses a sliding window method to crop the input data, cropping the input data of the target size, and overlapping (2, 16, 16) voxels between each sliding window. The cropped data is input into the model, and the multiple prediction results are summarized, and non-maximum suppression is used to eliminate overlapping boxes or redundant boxes to ensure that the prediction effect is optimal.

[0089] It should be noted that the purpose of disclosing the embodiments is to facilitate a further understanding of the present invention. However, those skilled in the art will appreciate that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be determined by the scope defined in the claims.

Claims

1. A method for detecting and classifying placenta accreta MRI signs based on deep neural networks, characterized in that: A deep neural network-based placenta accreta MRI sign region detection and classification model is constructed. For medical MRI image data, data preprocessing is performed, and the preprocessed MRI image is input. Feature extraction is performed using a multi-layer convolutional neural network of the detection and classification model to obtain MRI sign features of placenta accreta containing multi-scale information. Multiple prediction modules of the detection and classification model then output a three-dimensional bounding box containing the sign region and the category to which the sign belongs. The method comprises the following steps: 1) Preprocess the MRI image, represent the image size as a four-tuple (number of channels, depth, height, and width), and set the image size; 2) Construct and train a deep neural network-based model for placenta accreta MRI sign region detection and classification; the model includes a feature extraction module and an MRI sign region prediction module; the MRI sign region prediction module includes a sign region generation module, a false positive region reduction module, and a mask generation module; The feature extraction module is used to extract and combine input image features to obtain characteristic features. The module structure includes: basic convolution block, maximum pooling layer, residual convolution block with channel attention mechanism, and transposed convolution layer. The sign region generation module is used to predict the sign region bounding box information of the input sign features through convolution operation, including the coordinates, length, and confidence of the sign region bounding box; The false positive region reduction module is used to classify and predict the category of the sign area and suppress the non-sign area. It can also fine-tune the position and size of the bounding box. The module structure includes: a region of interest pooling layer and two parallel fully connected layers. The mask generation module segments the feature shape within the input bounding box through multiple upsampling and splicing operations, and outputs the feature mask; it includes: upsampling layer and activation layer; The total loss function used in model training consists of the sign region generation loss, the false positive region reduction loss, and the mask generation loss function. Both the sign region generation loss and the false positive reduction loss include classification loss and regression loss. The mask generation loss is calculated based on the Soft Dice loss function. After model training, a trained placenta accreta MRI sign region detection and classification model is obtained. 3) Using the trained placenta accreta MRI sign region detection and classification model, the MRI image to be detected is input and classified and predicted to obtain the placenta accreta MRI sign region category.

2. The method for detecting and classifying placenta accreta MRI signs based on a deep neural network as claimed in claim 1, wherein: The preprocessing of MRI images specifically includes: 11) Traverse each MRI image in the dataset, read the window width and window position information in the medical image DICOM format file, and perform grayscale normalization on the MRI image; manually annotate the MRI training image, including annotating the bounding box, corresponding feature area or feature shape; 12) Traverse the training set annotations, analyze the positions of the annotation bounding boxes, and calculate the median and variance; the positions of the bounding boxes include the center point coordinates and the depth, length, and width information; 13) Randomly crop the image using Gaussian weighted sampling, with the crop size ranging from 0.8 to 1.2 times the target size, and then scale it to the target size; flip the cropped data area horizontally or vertically with a set probability; fill the missing area by filling; perform the same operation on the bounding box annotation to augment the training dataset.

3. The method for detecting and classifying placenta accreta MRI signs based on a deep neural network as claimed in claim 1, wherein: Step 1) The MRI image is preprocessed and the image size is represented as a four-tuple (number of channels, depth, height and width); the target size of the model input image is set to (1, 24, 128, 128).

4. The method for detecting and classifying placenta accreta MRI signs based on a deep neural network as claimed in claim 1, wherein the feature The structure of the extraction module includes: a basic convolution block containing two convolution layers, denoted as C1; three maximum pooling layers, seven residual convolution blocks with channel attention mechanism, denoted as C2 to C8; three transposed convolution layers; the feature extraction module also includes three splicing operations.

5. The method for detecting and classifying placenta accreta MRI signs based on a deep neural network as claimed in claim 4, wherein: During model training, the input image passes through the MRI image feature extraction module, where the basic convolution block extracts low-level features in the image; The image features are then combined through seven residual convolution blocks with channel attention mechanisms to generate high-level features; the features are then fused through splicing operations and transposed convolution layers.

6. The method for detecting and classifying placenta accreta MRI signs based on a deep neural network as claimed in claim 4, wherein: The training sign region generation module uses the placenta accreta MRI sign features as input features and obtains the 3D bounding box sign area and the corresponding confidence level through two 1×1 convolution operations. Specifically, six 3D bounding boxes and their corresponding confidence levels are generated at each position of the input feature. The 3D bounding boxes are represented by the center point coordinates and length, width, and height data.

7. The method for detecting and classifying placenta accreta MRI signs based on a deep neural network as claimed in claim 6, wherein: The bounding box area is further classified through the false positive area reduction module. Specifically, the bounding box obtained by the sign area generation module is regarded as the region of interest, cropped on the output feature map of the residual convolution block C2, and made consistent in size through the region of interest pooling operation, which serves as the input of the fully connected layer. The bounding box containing the sign area is classified to obtain the sign category to which it belongs, and the bounding box coordinates and length, width and height information are fine-tuned to obtain the corrected bounding box.

8. The method for detecting and classifying placenta accreta MRI signs based on deep neural network as claimed in claim 6, wherein the mask The activation layer of the generation module uses the Sigmoid activation layer; The mask generation module cuts out the corresponding area of ​​the corrected bounding box on the output features of the C1 module and the input MRI image, generates a shape mask prediction of the feature, and optimizes the bounding box information based on the shape mask prediction of the feature.

9. The method for detecting and classifying placenta accreta MRI signs based on a deep neural network as claimed in claim 1, wherein the model During training, the classification loss in the feature region generation loss and false positive reduction loss uses cross entropy, and the regression loss uses the Smooth-L1 loss function; the mask generation loss is specifically calculated based on the improved Soft Dice loss function. mask , expressed as formula (7) and (8): Among them, D(m i , g i ) represents the similarity between the two masks calculated by Soft Dice for each predicted mask and the real mask; m i and g i Represents the i-th predicted mask and the true mask; m ij and g ij Represents m i and g i The predicted value and true value of the j-th position in; N represents the number of predicted bounding boxes, N v Represents the number of voxels in the bounding box; parameters α and β are parameters that regulate the proportion of false positives and true positives when the predicted value matches the true value.

10. A device for detecting and classifying signs of placenta accreta from magnetic resonance imaging based on a deep neural network, characterized in that: Used to implement the method according to any one of claims 1 to 9, comprising: an MRI image preprocessing module and an MRI placenta accreta sign region detection and classification module; wherein the MRI placenta accreta sign region detection and classification module comprises: a feature extraction module, a sign region generation module, a false positive region reduction module and a mask generation module; The feature extraction module includes a convolution layer, a normalization layer and an activation layer, which is used to extract MRI placenta accreta feature information from the input image data; The sign region generation module is used to generate a large number of 3D bounding boxes and the confidence of the sign region contained in the bounding boxes according to the input MRI placenta accreta sign features; and input the high-confidence bounding boxes and the image features of the corresponding regions into the false positive reduction module and the mask generation module; The false positive reduction module further predicts the feature category and suppresses the non-feature area, and can also fine-tune the position and size of the bounding box; The mask generation module is used to segment the sign shape within the input bounding box through multiple upsampling and splicing, and output a sign mask, that is, to obtain the sign area related to placenta implantation.

Images