Pelvic floor muscle edema focus area segmentation method based on MGBCU

Abstract

The invention discloses an MGBCU-based pelvic floor muscle edema focus area segmentation method, belongs to the technical field of image processing and application, and aims to solve the problem of inaccuracy in pelvic floor muscle edema focus area positioning and segmentation of an existing method. The method comprises the following steps: firstly, preprocessing an original diseased pelvic floor muscle image through a data acquisition module to ensure that the size of the image is consistent, and converting the image into a 224 * 224 standard input format; then, the processed image is segmented through a semantic segmentation network fusing a VSSM feature extraction network, a GCUP dynamic graph convolution feature aggregation network and an SU feature fusion up-sampling module, the VSSM feature extraction network extracts local and global features of the image through a multi-layer coding stage, and the GCUP method combines the features through a dynamic neighborhood graph to obtain a multi-layer coding image; and the SU module carries out feature fusion on multiple scales so as to obtain more accurate focus region segmentation. And finally, training the segmentation model through a training module, calculating loss by adopting a cross entropy loss function, and generating a model capable of efficiently segmenting the pelvic floor muscle edema focus area through optimization of 500 learning cycles. By means of the method, accurate and efficient segmentation of the pelvic floor muscle edema focus area can be achieved, and therefore reliable support is provided for related medical diagnosis and treatment.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing and application technology, and specifically relates to a method for segmenting pelvic floor muscle edema lesion areas based on deep learning. Background Art

[0002] It is reported that the incidence of pelvic floor muscle injury can be as high as 15% during first delivery. This pelvic floor muscle edema is usually caused by compression, stretching or injury of the pelvic floor muscles during vaginal delivery. Pelvic floor muscle edema not only affects the function of the muscles themselves, but may also be associated with later long-term complications such as vaginal prolapse and rectal prolapse. Especially in difficult delivery or when the fetus is in an improper position, the levator ani muscles (especially the part that inserts into the pubic bone) are prone to tearing, and this injury is usually not fully recovered after delivery. Before 2015, scholars mostly used traditional machine learning to achieve segmentation of target objects. Traditional segmentation methods include threshold method, region growing method and edge detection method, but the segmentation accuracy and efficiency of traditional machine learning in medical images are difficult to meet the requirements of practical applications; In recent years, many studies have achieved classification diagnosis of diseases through convolution technology in deep learning and achieved an accuracy of about 95%. However, classification results alone cannot provide accurate positioning of the lesion area for surgical treatment of pelvic floor muscle edema. At present, there are many application studies of convolution technology in the field of medical segmentation at home and abroad, and the segmentation accuracy has exceeded that of traditional machine learning. Although target information can be extracted through deep convolution, a lot of edge detail information is lost in the convolution process, resulting in unsatisfactory edge segmentation effect. Summary of the invention

[0003] Purpose of the invention: In view of the problems of small amount of data, low segmentation accuracy and difficult boundary segmentation of pelvic floor muscle edema, the present invention proposes a segmentation method of pelvic floor muscle edema lesion area based on VSSM. This method uses VSSM as the main body and convolutional neural network as the auxiliary, integrates dynamic graph convolution feature fusion, and uses the pelvic floor muscle edema lesion area data set annotated by experts to train the model, with the goal of extracting complete information of the pelvic floor muscle edema area. According to the characteristics of the model network structure and the requirements of the medical image segmentation task, a new segmentation method is proposed to achieve accurate segmentation of pelvic floor muscle edema. Technical Solution

[0004] 1. A segmentation method for pelvic floor muscle edema lesion area based on MGBCU, the segmentation method comprising an acquisition module, a semantic segmentation network module and a training module, and using the data acquisition module, the semantic segmentation network module and the training module to segment the diseased pelvic floor muscle image, characterized in that the method specifically comprises the following steps: (1) The diseased pelvic floor images constitute a set of pelvic floor muscle edema segmentation data sets, and the pelvic floor muscle edema segmentation data sets are used as original data samples. The data acquisition module performs a preprocessing operation on the images in the original data samples to ensure that the length and height of the images are the same, thereby forming a set of training set images; (2) The training set images are segmented using the MGBCU semantic segmentation network module, which includes a VSSM feature extraction network, a GCUP dynamic graph convolution feature aggregation network, and a SU feature fusion upsampling module; the VSSM feature extraction network processes the training set images to generate output features of the current layer, the GCUP dynamic graph convolution feature aggregation method obtains local features and global features by constructing a dynamic neighborhood graph, the SU fusion upsampling method performs dynamic feature aggregation on a multi-scale basis to obtain the image feature map, and a pelvic floor muscle edema segmentation map is obtained by a pyramid pooling method; (3) using the training module to train the segmentation model, inputting the pelvic floor muscle edema segmentation data set into the semantic segmentation network module for training, adjusting the model parameters by setting the learning rate, loss function method, and learning iteration cycle during training, and finally forming a pelvic floor muscle edema lesion area segmentation model; The pre-processing operation is: The input image size required by the method of the present invention is M×N×3, where M and N are positive integers, and the original image size is H×W, where H and W are positive integers. The image is first scaled to M×((N / H)×W), and then the shorter sides are evenly filled with gray edges to convert its size to 224×224. The VSSM feature extraction network is: Input the image Into the VSSM feature extraction network, it is first processed by the PatchEmbed2D module, divided into non-overlapping blocks, and mapped to a high-dimensional space to generate a size of The feature map is then processed in four encoding stages of the VSSM network, each of which consists of a multi-layer SS2D module and a PatchMerging module to extract local and global features while gradually downsampling the resolution; the first stage converts the image feature features from Convert to ; The second stage transforms the image features from Convert to ; The third stage is to transform the image features from Convert to ; The last stage converts the image features into ; The GCUP dynamic graph convolution feature fusion method is: Input the image features , a sparse dynamic neighborhood graph is generated by the dynamic neighborhood construction method, the distance between feature points is calculated using the KNN-based strategy, and the neighborhood relationship is optimized by combining the expansion and randomization strategies; in the dynamic graph convolution process, the input feature map is first processed by the convolution layer (fc1), and the size is kept as ; Then, the feature map is flattened and a neighborhood map is generated using a dynamic neighborhood construction method, and the features are reconstructed as The shape of the dynamic neighborhood graph is In the dynamic graph convolution stage, the features in the neighborhood are extracted through index operations, and the feature differences are aggregated and concatenated to generate a network with a size of The feature map is then updated to the shape of the updated feature map. Subsequently, the features are reduced in dimension through the convolutional layer (fc2), and the size of the final output feature map is returned to ; Output the image features; The SU fusion upsampling method is as follows: Input multi-level image features, deep features ( ), intermediate features ( ) and shallow features ( ) is input into the module, and the deep features are upsampled to make their resolution consistent with the intermediate features. At the same time, the shallow features are downsampled through the maximum pooling operation to make their resolution consistent with the intermediate features, which is ; Then, the upsampled deep features, the original intermediate features, and the downsampled shallow features are concatenated in the channel dimension to obtain a fused feature map with a size of ; Then, multi-branch convolution processing is performed on the fused feature map, and convolution branches with different kernel sizes are used to extract multi-scale information. The output dimension of each branch is the same as the input feature Figure 1 To, both ; The output results of multi-branch convolution are stacked to form a multi-scale feature tensor with a dimension of ; Perform global average pooling on the fusion features to obtain statistical information of the channel dimension, and reduce its dimension to The number of channels is then restored to the original number of channels using different fully connected layers , and generate a weight vector for each branch; the weight is normalized by softmax to form a weight matrix with the dimension Finally, the weight matrix is ​​multiplied element by element with the multi-branch features, and summed along the branch dimension to generate the final output image feature map, and a convolution operation is performed to obtain a semantic segmentation image; 2. According to the segmentation method, it is characterized in that: The loss function includes a cross entropy loss function and a Dice Loss function. The cross entropy loss function and the Dice Loss function are fused as the loss function of the semantic segmentation network model to minimize the following objective function: ;in, represents the cross entropy loss function; 3. The segmentation method according to claim 1, characterized in that: The learning iteration cycle is 500 cycles, and the learning rate is 1e-5. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 Schematic diagram of the MGBCU semantic segmentation network structure Figure 2 Schematic diagram of the VSSM structure Figure 3 Schematic diagram of the structure of GCUP dynamic graph convolution Figure 4 Schematic diagram of the structure of SU fusion upsampling Figure 5 Comparison chart before and after data preprocessing Figure 6 Schematic diagram of segmentation results DETAILED DESCRIPTION

[0006] Embodiment: The segmentation method of the pelvic floor muscle edema lesion area based on MGBCU provided by the present invention is used to segment the pelvic floor muscle edema lesion area. The specific operation is performed according to the following steps: 1. Pelvic floor muscle edema dataset The pelvic floor muscle edema segmentation dataset contains 458 pelvic floor images with pelvic floor muscle edema (including pelvic floor muscle edema symptoms of different degrees of disease), including 303 training and verification data and 155 test data. The lesion area of ​​each pelvic floor muscle edema image is personally annotated by a pelvic floor rehabilitation expert; 2. Data Preprocessing The input image size required by the method of the present invention is , the original image size is , converting its dimensions to ; 3. Training semantic segmentation network During network training, data is input to the network in batches, with an input size of The image is first processed by the PatchEmbed2D module, which divides it into non-overlapping patches and maps it to a high-dimensional space, generating a The feature map is then processed through four encoding stages of the VSSM network, each of which consists of a multi-layer SS2D module and a PatchMerging module to extract local and global features while gradually downsampling the resolution. Convert to The second stage starts from Convert to ; The third stage further transforms the features from Convert to ; The last stage keeps the feature size ; The four-stage feature maps are then input into the GPCU module with a size of As an example, a sparse dynamic neighborhood graph is generated by the dynamic neighborhood construction method, the distance between feature points is calculated using the KNN-based strategy, and the neighborhood relationship is optimized by combining the expansion and randomization strategies. In the dynamic graph convolution process, the input feature map is first processed by the convolution layer (fc1) and the size is kept as Then, the feature map is flattened and a neighborhood map is generated using a dynamic neighborhood construction method. The features are reconstructed as The shape of the dynamic neighborhood graph is In the dynamic graph convolution stage, the features in the neighborhood are extracted through index operations, and the feature differences are aggregated and concatenated to generate a network with a size of The feature map is then updated to the shape of the updated feature map. Subsequently, the features are reduced in dimension through the convolutional layer, and the size of the output feature map generated is returned to The final output feature size is , , , ; The four features are input into the SU feature fusion upsampling module for multi-scale feature fusion. ), intermediate features ( ) and shallow features ( ) as an example, the feature is input into the module, and the deep feature is upsampled to make its resolution consistent with the intermediate feature. At the same time, the shallow features are downsampled through the maximum pooling operation to make their resolution consistent with the intermediate features, which is Then, the upsampled deep features, the original intermediate features, and the downsampled shallow features are concatenated in the channel dimension to obtain a fused feature map with a size of The fused feature map is then subjected to multi-branch convolution processing, using convolution branches with different kernel sizes to extract multi-scale information. The output dimension of each branch is the same as the input feature Figure 1 To, both The output results of multi-branch convolution are stacked to form a multi-scale feature tensor with a dimension of The fused features are globally averaged and pooled to obtain the statistical information of the channel dimension, and then the dimension is reduced to The number of channels is then restored to the original number of channels using different fully connected layers , and generate a weight vector for each branch. The weights are normalized by softmax to form a weight matrix with the dimension Finally, the weight matrix is ​​multiplied element-wise with the multi-branch features and summed along the branch dimension to generate the final output feature map, whose size is , perform convolution operation to obtain the semantic segmentation image; The loss function is calculated for the pelvic floor muscle edema semantic segmentation map and the real pelvic floor muscle edema semantic segmentation map, and the network parameters are updated. The present invention combines the cross entropy loss function and the Dice Loss loss function as the loss function of the semantic segmentation network model to minimize the following objective function: in, represents the cross entropy loss function; remember , , defined as follows Objective function: ;The more accurate the pixel classification, the smaller the Loss; 4. Analysis of processing results This method uses the following two performance metrics to quantify the processing results: single-class intersection over union (IoU), mean intersection over union (MIoU) and single-class pixel accuracy (PA), mean pixel accuracy (MPA), calculated as follows: ; represents the segmented area, Represents the true area. The intersection over union (IOU) is obtained by dividing the intersection of the true value and the predicted value by their union; the mean intersection over union (MIOU) is obtained by calculating the intersection over union of each category (including the background category) and averaging all categories; ; represents the number of pixels predicted correctly, Represents the number of pixels predicted as j for category i. Pixel accuracy (PA) represents the proportion of correctly labeled pixels to the total pixels; average pixel accuracy is the ratio of correctly classified pixels in each category, and then the average of all categories.

[0007] The results of the present invention on the pelvic floor muscle edema test set are MIOU: 84.14%, MPA: 96.8%, IOU: 71.8%, PA: 90.91%. Extensive use shows that the segmentation method of the pelvic floor muscle edema lesion area based on VSSM proposed in the present invention has high segmentation performance. This is of great significance in the medical field; As described above, although the present invention has been shown and described with reference to certain preferred embodiments, it is not to be construed as limiting the invention itself. Various changes in form and details may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A segmentation method for pelvic floor muscle edema lesion area based on MGBCU, the segmentation method comprising an acquisition module, a semantic segmentation network module and a training module, and using the data acquisition module, the semantic segmentation network module and the training module to segment the diseased pelvic floor muscle image, characterized in that: The specific steps are as follows: (1) The diseased pelvic floor images constitute a set of pelvic floor muscle edema segmentation data sets, and the pelvic floor muscle edema segmentation data sets are used as original data samples. The data acquisition module performs a preprocessing operation on the images in the original data samples to ensure that the length and height of the images are the same, thereby forming a set of training set images; (2) The training set images are segmented using the semantic segmentation network module, which includes a VSSM feature extraction network, a GCUP dynamic graph convolution feature aggregation network, and a SU feature fusion upsampling module. The VSSM feature extraction network processes the training set images to generate output features of the current layer, the GCUP dynamic graph convolution feature aggregation method obtains local features and global features by constructing a dynamic neighborhood graph, the SU fusion upsampling method dynamically aggregates features on a multi-scale basis to obtain the image feature map, and a pterygium segmentation map is obtained by a pyramid pooling method; (3) using the training module to train the segmentation model, inputting the pelvic floor muscle edema segmentation data set into the semantic segmentation network module for training, adjusting the model parameters by setting the learning rate, loss function method, and learning iteration cycle during training, and finally forming a pelvic floor muscle edema lesion area segmentation model; The pre-processing operation is: The input image size required by the method of the present invention is M×N×3, where M and N are positive integers, and the original image size is H×W, where H and W are positive integers. The image is first scaled to M×((N / H)×W), and then the shorter sides are evenly filled with gray edges to convert its size to 224×224. The VSSM feature extraction network is: Input the image Into the VSSM feature extraction network, it is first processed by the PatchEmbed2D module, divided into non-overlapping blocks, and mapped to a high-dimensional space to generate a size of feature map; then, the feature map is processed in turn through four encoding stages of the VSSM network, each stage consisting of a multi-layer SS2D module and a PatchMerging module for extracting local and global features while gradually downsampling the resolution; the first stage converts the image feature features from Convert to ; The second stage transforms the image features from Convert to ; The third stage transforms the image features from Convert to ; The final stage converts the image features into ; The GCUP dynamic graph convolution feature fusion method is: Input the image features , a sparse dynamic neighborhood graph is generated by the dynamic neighborhood construction method, the distance between feature points is calculated using the KNN-based strategy, and the neighborhood relationship is optimized by combining the expansion and randomization strategies; in the dynamic graph convolution process, the input feature map is first processed by the convolution layer (fc1), and the size is kept as ; Then, the feature map is flattened and a neighborhood map is generated using a dynamic neighborhood construction method, and the features are reconstructed as The shape of the dynamic neighborhood graph is In the dynamic graph convolution stage, the features in the neighborhood are extracted through index operations, and the feature differences are aggregated and concatenated to generate a network with a size of The feature map is then updated to the shape of the updated feature map. Subsequently, the features are reduced in dimension through the convolutional layer (fc2), and the size of the final output feature map is returned to ; Output the image features; The SU fusion upsampling method is as follows: Input multi-level image features, deep features ( ), intermediate features ( ) and shallow features ( ) is input into the module, and the deep features are upsampled to make their resolution consistent with the intermediate features. At the same time, the shallow features are downsampled through the maximum pooling operation to make their resolution consistent with the intermediate features, which is ; Then, the upsampled deep features, the original intermediate features, and the downsampled shallow features are concatenated in the channel dimension to obtain a fused feature map with a size of ; Then, the fused feature map is subjected to multi-branch convolution processing, and convolution branches with different kernel sizes are used to extract multi-scale information. The output dimension of each branch is consistent with the input feature map, and both are ; The output results of multi-branch convolution are stacked to form a multi-scale feature tensor with a dimension of ; Perform global average pooling on the fusion features to obtain statistical information of the channel dimension, and reduce its dimension to The number of channels is then restored to the original number of channels using different fully connected layers , and generate a weight vector for each branch; the weight is normalized by softmax to form a weight matrix with the dimension ; Finally, the weight matrix is ​​multiplied element by element with the multi-branch features, and the sum is taken along the branch dimension to generate the final output image feature map, and a convolution operation is performed to obtain a semantic segmentation image.

2. The segmentation method according to claim 1, characterized in that: The loss function includes a cross entropy loss function and a Dice Loss function. The cross entropy loss function and the Dice Loss function are fused as the loss function of the semantic segmentation network model to minimize the following objective function: in, represents the cross entropy loss function.

3. The segmentation method according to claim 1, characterized in that: The learning iteration cycle is 500 cycles, and the learning rate is 1e-5.

Images