Heart mitral valve medical image segmentation model, training method, segmentation method and equipment

Abstract

The invention discloses a heart mitral valve medical image segmentation model, a training method, a segmentation method and equipment. The segmentation model is improved on the basis of a Double U-Net architecture, and two independent U-Net networks which are connected in series respectively comprise an encoder and a decoder; an OfficientNetB7 network is used as a feature extraction encoder of the front end U-Net network; after features are extracted from the two U-Net networks, a multi-kernel residual error convolution (MKRC) module, a cavity space convolution pyramid pooling (ASPP) module and a mixed triple attention (TAG) module are sequentially introduced into the two U-Net networks; a triple attention gate (TAM) module is introduced into the input end of the decoder. Segmentation model training adopts a mixed loss function, binary cross entropy loss and Dice loss are combined, the class imbalance problem is balanced, and segmentation precision is improved. The heart and related blood vessel structures can be automatically and accurately segmented, so that the technical problem that heart mitral valve image segmentation is difficult to accurately and efficiently perform in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically, to a cardiac mitral valve medical image segmentation model, a training method, a segmentation method and a device. Background Art

[0002] Cardiac intervention is an important means of treating cardiovascular diseases. In cardiac mitral valve intervention, accurate medical image segmentation is essential for surgical navigation and robotic operation. Traditional segmentation methods rely on manual annotation and image processing techniques, which are time-consuming and have limited accuracy. With the development of deep learning technology, medical image segmentation methods based on deep neural networks have shown superior performance and potential.

[0003] The segmentation of mitral valve images faces two major challenges: poor image quality and complex vascular morphology. In terms of image quality, X-ray angiography images contain a lot of noise and artifacts, low contrast between blood vessels and surrounding tissues, and uneven illumination, all of which affect the accuracy of segmentation and the robustness of the model. In terms of vascular morphology, the vascular network has a complex branching structure with different blood vessel diameters, shapes, and directions. Blood vessel bending, deformation, and overlap are common, which brings great difficulties to segmentation.

[0004] Traditional segmentation methods based on threshold, edge detection or region growing are difficult to adapt to the complexity and diversity of cardiac interventional surgery imaging data. The segmentation accuracy is not high, making it difficult to accurately identify subtle structures. In addition, the large consumption of computing resources leads to slow processing speed and low resource utilization. The strong dependence on training data leads to poor generalization ability. The lack of robustness to complex backgrounds and noise makes it easy to be disturbed. It is also difficult to process multimodal data, resulting in limited applicability. These problems limit the effectiveness and reliability of existing traditional methods in actual clinical applications.

[0005] Although U-Net and DoubleU-Net have achieved success in the field of medical image segmentation, they still have limitations when processing cardiac interventional surgery image data, such as difficulty in dealing with the bending deformation and overlap of blood vessels, and the segmentation accuracy needs to be improved. Therefore, it is of great clinical significance to develop a method that can accurately segment blood vessels in cardiac mitral valve interventional surgery image data. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention proposes a cardiac mitral valve medical image segmentation model, training method, segmentation method and device based on an improved DoubleU-Net model (called DoubleU-NetPlus), which can automatically and accurately segment the heart and related vascular structures, thereby solving the technical problem in the prior art that it is difficult to accurately and efficiently perform cardiac mitral valve image segmentation.

[0007] To achieve the above-mentioned object, in a first aspect, the present invention provides a cardiac mitral valve medical image segmentation model, which is improved based on the DoubleU-Net architecture, wherein two independent and serially connected U-Net networks both include an encoder and a decoder;

[0008] Use the EfficientNetB7 network as the feature extraction encoder of the front-end U-Net network;

[0009] After the two U-Net networks extract features, the multi-kernel residual convolution (MKRC) module, the atrous spatial convolution pyramid pooling (ASPP) module and the hybrid triple attention (TAG) module are introduced in sequence; the triple attention gate (TAM) module is introduced at the input end of the decoder.

[0010] In some embodiments of the first aspect of the present application, the EfficientNetB7 network uses an inverted bottleneck structure and embeds a squeeze and excitation network (SE-Net) block in the bottleneck layer.

[0011] In some embodiments of the first aspect of the present application, the MKRC block uses parallel convolution layers to splice feature maps with different convolution kernel sizes; after each convolution layer, a batch normalization layer and a ReLU activation function are used to further optimize the feature representation; at the same time, the multi-core residual convolution module adds a 1×1 convolution layer, followed by batch normalization and ReLU activation to achieve further abstraction and compression of features; in addition, the multi-core residual convolution module also introduces a residual shortcut connection, that is, an identity mapping, which is spliced ​​with the previously generated feature map through 1×1 convolution and batch normalization processing.

[0012] In some embodiments of the first aspect of the present application, the ASPP module integrates the squeezing and excitation network as a SE-ASPP module.

[0013] In some embodiments of the first aspect of the present application, the TAG module inherits the core concepts of the existing CBAM and Focus U-Net, and also achieves deep fusion and refinement of features by parallel processing of squeeze and excitation networks, channel attention and spatial attention mechanisms.

[0014] In some embodiments of the first aspect of the present application, the TAM module receives two inputs: one is the gating signal from the next layer; the other is the jump connection at the same level; by processing channel attention, spatial attention and squeeze-excitation-based attention in parallel, the model is guided to focus on learning those features that are efficient and closely related to the task.

[0015] In some embodiments of the first aspect of the present application, the feature extraction encoder of the back-end U-Net network adopts a residual convolutional network; the residual convolutional network extracts features by performing two 3×3 convolution operations, each of which is followed by batch normalization and ReLU activation function, and a 1×1 convolution shortcut residual connection is added, and the shortcut residual connection performs identity mapping on the input feature map and processes it through batch normalization and ReLU activation function; subsequently, the feature map generated by the 3×3 convolution operation and the 1×1 shortcut connection is spliced, and nonlinear processing is performed through another ReLU activation function.

[0016] In a second aspect, the present invention provides a training method for a cardiac mitral valve medical image segmentation model, characterized in that the cardiac mitral valve medical image segmentation model as described above is trained; the training method comprises the following steps:

[0017] Step S11, image acquisition: collecting mitral valve image data including but not limited to cardiac CT scan, MRI imaging and echocardiography;

[0018] Step S12, data annotation: a team of medical imaging experts and / or annotation personnel segments and annotates the collected imaging data;

[0019] Step S13, data preprocessing: including but not limited to adjusting image size, enhancing contrast and brightness, and uniformly adjusting all images to the size required by the model;

[0020] Step S14, data set division: the labeled data set is divided into a training set, a validation set and a test set;

[0021] Step S15, weight initialization: initialize the model using the pre-trained EfficientNetB7 weights;

[0022] Step S16, loss function design: adopt a hybrid loss function, combining binary cross entropy loss and Dice loss; the binary cross entropy loss (Loss BCE )The formula is as follows:

[0023]

[0024] Where y is the true label, which takes a value of 0 or 1; p is the probability of the model predicting a positive class, which takes a value range of [0,1];

[0025] The formula for the Dice loss function is as follows:

[0026]

[0027] Where N is the number of samples, i is the sequence number of the pixel sample,p i The model predicts i The probability or label of a pixel belonging to the target category, g i For the i The label of whether the pixel belongs to the target category in the real annotation, ϵ is a small constant; the final mixed loss is:

[0028] ;

[0029] Step S17: Set the learning rate and prepare the training environment for model training;

[0030] Step S18, select evaluation indicators and implement model evaluation: select precision, recall, Dice coefficient and intersection over union (IoU) as the main evaluation indicators.

[0031] In a third aspect, the present invention provides a method for segmenting a cardiac mitral valve medical image, characterized in that it comprises the following steps:

[0032] Step S21, acquisition and preprocessing of the image to be segmented: the image to be segmented includes a real-time image and / or a pre-recorded image; the image is preprocessed to meet the input requirements of the EfficientNetB7 encoder;

[0033] Step S22, using the heart mitral valve medical image segmentation model pre-trained by the training method described above to perform heart mitral valve medical image segmentation; the heart mitral valve medical image segmentation model is the heart mitral valve medical image segmentation model described above;

[0034] Step S23, outputting the image after model segmentation: including but not limited to segmentation map generation, accuracy assessment and structure verification.

[0035] In a fourth aspect, the present invention provides an electronic device, characterized by a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the training method as described above and / or the cardiac mitral valve medical image segmentation method as described above.

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] (1) The DoubleU-NetPlus model of the present invention adopts a two-layer U-Net structure. Compared with the traditional U-Net network, the two independent U-Net networks of DoubleU-NetPlus both contain an encoder and a decoder. The encoder is used to extract features, and the decoder is used to reconstruct images. The combination of the two enables the model to more accurately capture the subtle features in the image. Especially when processing images of cardiac mitral valve interventional surgery, this structure can effectively deal with the complex morphology and low contrast problems of blood vessels and provide more accurate segmentation results.

[0038] (2) The DoubleU-NetPlus model of the present invention uses EfficientNetB7 as the encoder for feature extraction. Compared with the traditional VGG19 encoder used by DoubleU-Net, this encoder extracts image features through depthwise separable convolution and moving inverted bottleneck convolution, and can automatically adjust the depth, width and resolution of the network, thereby improving the efficiency and accuracy of image feature extraction.

[0039] (3) The DoubleU-NetPlus model of the present invention introduces the MKRC module, which allows the capture of features of different scales through parallel multi-size convolution kernels. Multi-kernel convolution can simultaneously consider local and global features. This multi-scale feature capture is particularly important for processing complex vascular structures in cardiac interventional surgery images. It can effectively deal with the bending, deformation and overlapping of blood vessels and provide more accurate segmentation results.

[0040] (4) The TAM module in the DoubleU-NetPlus model of the present invention integrates spatial attention, channel attention and squeeze-excited attention mechanisms, and enhances the relevance of feature maps through parallel processing. The spatial attention mechanism enables the model to focus on important areas in the image and improve the accuracy of segmentation. The channel attention mechanism enhances the inter-channel correlation of feature maps and improves the expressiveness of features. The squeeze-excited attention mechanism further optimizes the distribution of features by re-weighting feature maps, thereby enhancing the segmentation ability of the model.

[0041] (5) The TAG module in the DoubleU-NetPlus model of the present invention can selectively focus on the target area in the skip connection, suppress the features of irrelevant areas, and reduce the interference of irrelevant information. In the images of cardiac mitral valve interventional surgery, the contrast between blood vessels and surrounding tissues is low. The TAG module can effectively focus on the blood vessel area and provide more accurate segmentation results.

[0042] (6) The DoubleU-NetPlus model of the present invention makes an innovative improvement on ASPP and proposes the SE-ASPP module, which further enhances the field of view of the convolution filter by integrating the squeeze and excitation network (SE-Net). The present invention adopts a deeper level of dilated convolution technology in the SE-ASPP module, which enables the network to perceive the local and global features in the mitral valve images of the heart more finely.

[0043] (7) The DoubleU-NetPlus model training of the present invention adopts a hybrid loss function, combining binary cross entropy loss and Dice loss, to balance the class imbalance problem and improve segmentation accuracy. Binary cross entropy loss ensures pixel-level classification accuracy, while Dice loss focuses on the overlap between the predicted area and the real area. The combination of the two enables the model to more accurately identify and segment the vascular area, improving the overall performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 4 is an architecture diagram of the DoubleU-NetPlus segmentation network model in one embodiment of the present invention.

[0045] Figure 2 Schematic diagram of an AG-Residual convolution module in one embodiment of the present invention.

[0046] Figure 3 FIG. 1 is a schematic diagram of a MKRC module in an embodiment of the present invention.

[0047] Figure 4 FIG. 4 is a schematic diagram of a SE-ASPP module in an embodiment of the present invention.

[0048] Figure 5 FIG. 4 is a schematic diagram of a TAM module in an embodiment of the present invention.

[0049] Figure 6 FIG. 1 is a schematic diagram of a TAG module in an embodiment of the present invention.

[0050] Figure 7 The flowchart is a flowchart of implementing the segmentation of a cardiac mitral valve medical image in one embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0052] In the following detailed description, many specific details are set forth to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that well-known algorithms are not shown in detail to avoid obscuring the subject matter of the present invention.

[0053] In addition, the order of execution of actions, steps, etc. in the devices and methods shown in the claims, specifications and drawings can be implemented in any order as long as there is no special explicit limitation on the order and the output of the previous processing is not used in the subsequent processing.

[0054] Example 1

[0055] See also Figure 1 This embodiment provides a cardiovascular regional image (especially mitral valve medical image) segmentation model based on the DoubleU-NetPlus model. DoubleU-NetPlus is a dual U-Net architecture based on deep learning, which is improved based on DoubleU-Net and is designed to improve the segmentation accuracy of medical images. The model combines multiple network structures and technologies, and innovatively introduces the EfficientNetB7 encoder, MKRC module, SE-ASPP module, TAG module and TAM module.

[0056] As a preferred specific example, see Figure 1 The DoubleU-NetPlus model uses a two-layer U-Net structure, which connects two U-net networks in series, divided into Network1 and Network2. This two-layer structure enhances the model's deep feature extraction capabilities. Each network contains an encoder and a decoder, which achieve image segmentation through feature extraction and reconstruction. After the decoder of each network extracts features, the MKRC, SE-ASPP, and TAM modules are introduced.

[0057] In the U-Net architecture, the encoder part is responsible for gradually reducing the spatial size of the image while increasing the number of feature channels to extract high-level semantic features of the image. This process is achieved through multiple convolutional layers and pooling layers, and finally outputs highly compressed feature maps for the decoder to reconstruct and segmentation prediction. In the DoubleU-NetPlus architecture, we made the following improvements to the encoder part: We abandoned the traditional VGG19 and used the pre-trained EfficientNet B7 network as the encoder backbone of Network 1, and extracted features through transfer learning methods. The encoder of Network 2 adopts a residual convolution structure, and realizes feature extraction through two 3×3 convolutional layers and jump connections. Residual convolution can effectively alleviate the gradient vanishing problem and improve the training efficiency and segmentation accuracy of the model.

[0058] The specific structure of the EfficientNet B7 network in Network1 in this embodiment is as follows: Stage 1: Conv3x3 input 600x600, output 32; MBConv1 input 300×300, output 16, repetition 1, expansion factor 1, SE ratio 0.25. Stage 2: MBConv6 input 300×300, output 24, repetition 2, expansion factor 6, SE ratio 0.25. Stage 3: MBConv6 input 150×150, output 40, repetition 2, expansion factor 6, SE ratio 0.25. Stage 4: MBConv6 input 75×75, output 80, repetition 3, expansion factor 6, SE ratio 0.25. Stage 5: MBConv6 input 38×38, output 112, repetition 3, expansion factor 6, SE ratio 0.25. EfficientNetB7 has the following advantages: EfficientNetB7 uses an inverted bottleneck structure to achieve efficient calculations through depth-wise separable convolutions and point-by-point convolutions, and reduces the number of model parameters. Secondly, EfficientNetB7 embeds SE-Net blocks in the bottleneck layer, enhances feature representation through the channel attention mechanism, and improves the segmentation performance of the model. Most importantly, EfficientNetB7 uses a compound scaling method to systematically adjust the depth, width, and resolution of the network to achieve optimal performance with fixed computing resources.

[0059] As a further preferred example, each encoder block of the encoder in Network 2 performs an AG-residual convolution operation, and the specific structure is as follows Figure 2 As shown. The AG-residual convolution module extracts features by performing two 3×3 convolution operations, each followed by batch normalization and ReLU activation functions. The batch normalization function is:

[0060]

[0061] x is the input data, is the mean of the current batch of data, is the standard deviation of the current batch of data, and are learnable parameters, which are used to scale and translate the normalized data, respectively, so that the network can learn the original representation of the data. Batch normalization can reduce internal covariate shift and standardize the model. The ReLU activation function introduces nonlinearity, and the formula is:

[0062]

[0063] The ReLU activation function can enhance the nonlinear expression ability of the model. In order to reduce the gradient vanishing problem, a 1×1 convolution shortcut residual connection is added to the module, which performs identity mapping on the input feature map and processes it through batch normalization and ReLU activation function. Subsequently, the feature map generated by the 3×3 convolution operation and the 1×1 shortcut connection is concatenated and processed nonlinearly through another ReLU activation function. Finally, the generated feature map is input into the TAM module for further attention mechanism processing to highlight relevant features. The last operation of the module is a max-pooling operation with a 2×2 window and a 2×2 step size to reduce the spatial dimension of the feature map in preparation for the subsequent decoding process.

[0064] In the mitral valve image segmentation task, a major challenge is that the size and shape of the objects to be segmented in the image may vary greatly. In order to effectively address this challenge, the key is to extract high-level multi-scale contextual features through different receptive fields. To this end, a multi-kernel residual convolution (MKRC) module inspired by the Inception architecture is introduced in the design of the model of this embodiment. This module works after the decoder operation of Network 1 and Network 2. Figure 3 As shown in the figure, the MKRC module expands the field of view of the feature by using parallel convolution layers and different convolution kernel sizes such as 1×1, 3×3, 5×5 and 7×7, so as to perform differentiated feature learning for each feature block and enhance the model's learning and recognition capabilities for heterogeneous features. After each convolution layer, this model also uses a batch normalization layer and a ReLU activation function to further optimize the feature representation. Subsequently, the four feature maps are concatenated to integrate information from different receptive fields. In order to further improve the expressiveness of the features, the module adds a 1×1 convolution layer, followed by batch normalization and ReLU activation to achieve further abstraction and compression of the features. In addition, the module also introduces a residual shortcut connection, i.e., an identity mapping, which is concatenated with the previously generated feature map through 1×1 convolution and batch normalization. This design not only ensures the smooth transmission of information in the network, but also effectively reduces the risk of overfitting. The feature map finally output by the MKRC module can be expressed as:

[0065]

[0066] Among them, MKRC_FeatureFusion is the MKRC multi-scale fusion feature, which can be expressed as follows:

[0067]

[0068] In the formula, Concat means concatenating the outputs of convolutional layers with different convolution kernel sizes, and then processing them through batch normalization and residual connection. Finally, the generated feature map is sent to an improved SE-ASPP module.

[0069] Atrous Spatial Convolutional Pooling Pyramid (ASPP) introduces parallel atrous convolutional layers with different dilation rates, which significantly expands the receptive field of the filter and enables it to capture multi-scale semantic feature contexts. This innovative method effectively solves the problem of reduced spatial resolution caused by multiple downsampling in traditional encoders. In order to more effectively capture multi-scale contextual features in medical images and improve segmentation accuracy and model robustness, we have made innovative improvements to ASPP and proposed the SE-ASPP module, which further enhances the field of view of the convolution filter by integrating the squeeze and excitation network (SE-Net). Figure 4 As shown in Figure 1, we use a deeper dilated convolution technique in the SE-ASPP module. Specifically, the seven parallel convolution layers use dilation rates of 1, 1, 2, 6, 10, 13, and 16, respectively. This design enables the network to perceive local and global features in the image more finely. Through the integration of SE-Net, we effectively recalibrate and refine these features to ensure the quality and accuracy of the feature maps. In the SE-ASPP module, the feature maps generated by the SE-Net module of each branch are cleverly connected together and then further integrated through a 1×1 convolution operation. This step not only enhances the expressiveness of the features, but also provides a more compact feature representation for subsequent processing. Next, the application of batch normalization and ReLU activation function further optimizes the distribution of feature maps and enhances the network's responsiveness to features. The dilated convolution can be expressed as:

[0070]

[0071] Among them, x is the input feature map, k is the convolution kernel, d is the dilation rate, and the dilate function dilates the input feature map according to the dilation rate d. Then the atrous spatial convolution pooling pyramid (ASPP) can be expressed as:

[0072]

[0073] Where n is the number of convolutional layers with different dilation rates, k i and d i is the size and dilation rate of the convolution kernel of the i-th convolution layer. The SE module can be expressed as:

[0074]

[0075] Where σ is the activation function (here is ReLU), and is a learnable parameter, is the global average pooling operation, represents element-by-element multiplication. Thus, the SE-ASPP module can be expressed as

[0076]

[0077] in It is the result of processing the feature map output by the ASPP module with the SE module. The process of SE-ASPP feature refinement can be expressed as:

[0078]

[0079] Where m is the number of SE-ASPP modules. The final feature map is expressed as:

[0080]

[0081] These carefully processed feature maps are fed into the hybrid triple attention module (TAM) for deeper feature extraction and optimization.

[0082] The hybrid triple attention module (TAM) is an innovative attention mechanism that not only inherits the core concepts of the existing CBAM and Focus U-Net, but also achieves deep fusion and refinement of features through parallel processing of squeeze and excitation networks, optimized channel attention and spatial attention mechanisms. Figure 5 As shown in the figure, TAM can deeply explore the complex relationship between features and improve the expressiveness of features through this parallel processing strategy, thereby enhancing the recognition accuracy of the model for the target area. TAM uses the channel attention mechanism of SE-Net to adjust channel weights, optimize the interdependence between channels, and effectively suppress background noise, thereby highlighting the area to be segmented in cardiovascular images. Among them, spatial attention: , channel attention: , squeezing stimulates attention: The TAM module integrates the three and can be expressed as:

[0083] On this basis, this embodiment innovatively introduces a triple attention gate module (TAG). This innovative structure significantly improves the representation ability of the model. At the same time, experiments have proved that it can effectively control the growth of computational costs and model parameters. Like the existing attention gate and focus gate modules, the TAG module integrates multiple attention mechanisms in one. By processing channel attention, spatial attention, and squeeze-based attention in parallel, it guides the model to focus on learning those features that are efficient and closely related to the task, such as vascular features in cardiovascular images. Figure 6 As shown in Figure 1, the TAG module cleverly receives two inputs: one is the gating signal from the next layer, which has learned to capture key features such as edges, textures, and points during training; the other is the skip connection at the same level, which carries rich spatial information. When fusing these two inputs, we first resize them to the same size and then generate attention coefficients through element-wise addition and ReLU nonlinear activation function. These coefficients are then fed into carefully designed channel, spatial, and squeeze-excitation attention modules, and finally merged into a refined feature representation to highlight the key areas in the image. To further enhance the spatial resolution of the features, we use a 2×2 upsampling technique to match the output size of the 1×1 convolutional layer and perform a sigmoid operation on the output of the nonlinear activation function to adjust the distribution of weights. In this way, the weights aligned with the target region are strengthened, while other weights are weakened accordingly. By combining the original skip connection with the newly generated attention coefficient, we are able to accurately capture the spatial context information of the region of interest, allowing the model to intelligently adjust the weights according to the relevance of the features. The introduction of the TAG module not only enriches the model's ability to understand and analyze medical images, but also provides strong support for achieving accurate image segmentation, especially when processing mitral valve interventional surgery images with complex backgrounds and subtle structures.

[0084] Example 2

[0085] This embodiment provides a training method for the DoubleU-NetPlus model described in Embodiment 1.

[0086] Before model training, a large amount of labeled medical imaging data needs to be prepared, which will be used to train and verify the model. Data preprocessing includes image cropping, scaling, rotation, and enhancement to improve the robustness and generalization ability of the model. Then the loss function is designed, and the appropriate learning rate and training strategy are selected. Finally, the validation set is used to evaluate the accuracy and robustness of the model.

[0087] In the mitral valve image segmentation task, in order to make the model more suitable for the task, collecting and annotating the dataset by oneself is a key step to ensure the effectiveness of model training.

[0088] S11. Image acquisition: Cooperate with medical institutions to collect mitral valve imaging data such as cardiac CT scans, MRI imaging, and echocardiography. Ensure that all relevant privacy protection and ethical regulations are followed during the collection process.

[0089] S12. Data annotation: A team of professional medical imaging experts and well-trained annotation personnel accurately segment and annotate the collected imaging data. Professional image annotation tools are used to ensure the accuracy and consistency of annotation.

[0090] S13, data preprocessing: necessary preprocessing is performed on the collected original images, including adjusting the image size, enhancing contrast and brightness, and uniformly adjusting all images to the size required by the model. The present invention uniformly adjusts them to 600×600 pixels.

[0091] Preferably, data enhancement is performed on the image: In order to improve the generalization ability and robustness of the model, a data enhancement strategy is implemented on the training data. In the experiment of the present invention, we used a variety of enhancement techniques to ensure that the small number of samples in the data set will not be overfitted. We used a variety of data enhancement techniques, including random rotation, grid distortion, horizontal and vertical flipping, transposition, vertical flipping and random rotation, random brightness, random contrast, random brightness contrast, random gamma, hue saturation contrast and RGB shift, etc. to simulate image characteristics under different imaging conditions.

[0092] Model training is the core link in achieving cardiovascular segmentation and needs to be carefully designed to optimize model performance.

[0093] S14. Dataset division: The labeled dataset consists of 240 samples, 80% of which are divided into a training set for the main training process of the model. The remaining 20% ​​of the data is divided into a validation set and a test set for model verification and final evaluation, respectively.

[0094] S15. Weight initialization: Use the pre-trained EfficientNetB7 weights to initialize the model, which helps the model have better feature extraction capabilities at the beginning of training, accelerates the convergence process, and improves the final segmentation accuracy.

[0095] S16. Loss function design: A hybrid loss function is used, combining binary cross entropy loss and Dice loss, to balance the class imbalance problem and improve segmentation accuracy. Binary cross entropy loss ensures pixel-level classification accuracy, while Dice loss focuses on the overlap between the predicted area and the true area. The combination of the two enables the model to more accurately identify and segment the vascular area, improving the overall performance of the model.

[0096] Among them, binary cross entropy loss (LossBCE): Binary cross entropy loss is a loss function for binary classification problems, which measures the difference between the predicted probability distribution and the true label distribution. The LossBCE formula is as follows:

[0097]

[0098] Where y is the true label, which takes the value of 0 or 1. p is the probability of the model predicting the positive class, which takes the value range of [0,1].

[0099] The Dice loss function is a loss function used in image segmentation tasks to measure the difference between the model prediction results and the true labels. It is particularly suitable for processing unbalanced data sets. The specific formula is as follows:

[0100]

[0101] Where N is the number of samples, i is the serial number of the pixel sample, and p i is the probability or label predicted by the model that the i-th pixel belongs to the target category, g i is the label of whether the i-th pixel belongs to the target category in the real annotation, is a small constant. In order to avoid division by zero errors, a very small value such as 0.00001 is usually added to ensure numerical stability.

[0102] The final mixed loss is as follows:

[0103]

[0104] S17, learning rate setting: Set the initial learning rate to 0.0001, which helps the model converge quickly in the early stage of training. As the training progresses, adopt the learning rate decay strategy to gradually reduce the learning rate to ensure that the model can adjust the weights in detail in the later stage of training to avoid overfitting.

[0105] Training environment: GPU is NVIDIA GeForce RTX 4090 with 24GB video memory. CPU is Intel Xeon E5-2690v4 @ 2.60GHz with 32GB RAM. Deep learning framework is TensorFlow 2.0.

[0106] S18. Evaluation is an important part of measuring the segmentation effect of the model. The accuracy and reliability of the model are evaluated through a series of quantitative indicators:

[0107] Selection of evaluation indicators: Precision, recall, Dice coefficient and intersection over union (IoU) are selected as the main evaluation indicators. Precision refers to the proportion of samples that are truly positive among all samples predicted as positive by the model. Recall refers to the proportion of samples that are correctly predicted as positive by the model among all samples that are actually positive. The Dice coefficient measures the overlap between the predicted segmented area and the true annotated area, while IoU focuses more on the alignment of the boundaries between the predicted area and the true area.

[0108] The specific formula is as follows:

[0109]

[0110]

[0111]

[0112]

[0113] Among them, TP (True Positive): True positive refers to the number of samples correctly predicted as positive by the model. FP (False Positive): False positive refers to the number of samples incorrectly predicted as positive by the model. TN (True Negative): True negative refers to the number of samples correctly predicted as negative by the model. FN (False Negative): False negative refers to the number of samples incorrectly predicted as negative by the model.

[0114] Evaluation process implementation: Evaluate the model on the validation set, calculate the precision, recall, Dice coefficient and IoU value of the sample, and perform an overall performance evaluation. This process helps identify possible problems with the model in the segmentation task and provides guidance for further optimization.

[0115] In order to verify the beneficial effects of the present invention, the inventors conducted comparative simulation experiments using the mitral valve region image segmentation method based on the DoubleU-NetPlus model of Example 1 of the present invention and the DoubleU-Net method and U-Net method. The experimental comparison results are shown in Table 1.

[0116] Table 1 Comparison of evaluation results of image segmentation simulation experiments based on DoubleU-NetPlus model, DoubleU-Net model, and U-Net model

[0117]

[0118] Through comparison, it can be seen from four evaluation indicators that the DoubleU-NetPlus of the present invention has better segmentation effect than the DoubleU-Net and U-Net models, which proves the advantage of the improved model of the present invention in the mitral valve image segmentation task.

[0119] Example 3

[0120] See also Figure 7 This embodiment provides a method for performing cardiac mitral valve medical image segmentation using the DoubleU-NetPlus model described in Example 1 that is trained using the training method of Example 2.

[0121] S21. Acquisition and preprocessing of images to be segmented: Real-time or pre-recorded medical imaging data are acquired from the cardiac mitral valve interventional surgery robot system. These data may be images generated by CTA, MRI or other imaging technologies. The acquired images are preprocessed as necessary to meet the input requirements of the model.

[0122] More specifically, the sources of the images to be segmented are mainly the following two ways: Real-time images: During the operation, the imaging device carried by the interventional surgery robot captures cardiac images in real time. These images are essential for surgical navigation and real-time decision-making. Pre-recorded images: Before the operation, the patient may have undergone a series of diagnostic imaging examinations. These pre-acquired images can provide important information for surgical planning and model training.

[0123] S22, image preprocessing: The acquired image is adjusted to a uniform size and resolution. In this embodiment, it is set to 600×600 pixels to meet the input requirements of the EfficientNetB7 encoder.

[0124] In addition, the image can be appropriately contrast enhanced: the image contrast is adjusted to highlight the mitral valve and its surrounding structures, further enhancing the model's ability to recognize details. Noise removal: filtering algorithms are applied to remove noise and artifacts in the image, improving image quality and reducing interference with model performance.

[0125] S23. Output the image segmented by the model: Use the trained DoubleU-NetPlus model to segment the preprocessed image and generate the segmentation result. The output of the model is a segmentation map of the same size as the input image, in which different regions are marked as different categories.

[0126] The segmented image is the key output of the mitral valve interventional surgery robot system, which directly affects the navigation and treatment effect of the surgery. The specific process is as follows:

[0127] Model application: The preprocessed image is segmented using the trained DoubleU-NetPlus model. The model learns the features and structure of the image to identify and segment the mitral valve area of ​​the heart.

[0128] Segmentation map generation: The segmentation map output by the model distinguishes the mitral valve region of the heart from other regions, and each region is labeled as a different category, usually represented by a different color or label.

[0129] Accuracy evaluation: By calculating indicators such as precision, recall, Dice coefficient, and IoU (intersection over union), the accuracy and reliability of the segmentation results are evaluated to ensure that the quality of the segmentation map meets clinical needs.

[0130] Preferably, image optimization is performed: post-processing of the segmentation results is performed, such as edge smoothing, hole filling, etc., to improve the visual effect and practicality of the segmentation map.

[0131] Result verification: Medical experts verify the segmentation results to ensure that the segmentation map accurately reflects the actual position and shape of the mitral valve of the heart.

[0132] Example 4

[0133] This embodiment provides an electronic device, comprising a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the model described in Example 1, the training method described in Example 2, and / or the cardiovascular region image (especially mitral valve medical image) segmentation method described in Example 3.

[0134] The above-mentioned segmentation model and its training method can be embodied in the form of a computer program product or a software functional unit. If the above-mentioned segmentation model and its training method are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Therefore, the essence of the technical solution or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for an electronic system (which can be a personal computer, a server, or a network system, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0135] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0136] In summary, the present invention provides a segmentation model, training method, segmentation method and device for cardiac mitral valve medical images. The segmentation model is improved based on the DoubleU-Net architecture, and its two independent and serially connected U-Net networks both include an encoder and a decoder; the EfficientNetB7 network is used as the feature extraction encoder of the front-end U-Net network; after the two U-Net networks extract features, the multi-kernel residual convolution (MKRC) module, the atrous spatial convolution pyramid pooling (ASPP) module and the hybrid triple attention (TAG) module are introduced in sequence; the input end of the decoder introduces the triple attention gate (TAM) module. The segmentation model training adopts a hybrid loss function, combined with binary cross entropy loss and Dice loss, to balance the category imbalance problem and improve the segmentation accuracy. The present invention can automatically and accurately segment the heart and related vascular structures, thereby solving the technical problem that it is difficult to accurately and efficiently segment cardiac mitral valve images in the prior art.

[0137] Those skilled in the art should understand that those skilled in the art can implement variations by combining the prior art and the above embodiments, which will not be described in detail here. Such variations do not affect the essential content of the present invention, and will not be described in detail here.

[0138] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the systems and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention are still within the scope of protection of the technical solutions of the present invention.

Claims

1. A cardiac mitral valve medical image segmentation model, characterized in that: Improved based on the DoubleU-Net architecture, its two independent and serially connected U-Net networks both contain encoders and decoders; Use the EfficientNetB7 network as the feature extraction encoder of the front-end U-Net network; After the two U-Net networks extract features, a multi-core residual convolution module, a dilated spatial convolution pyramid pooling module and a hybrid triple attention module are introduced in sequence; a triple attention gate module is introduced at the input end of the decoder.

2. The cardiac mitral valve medical image segmentation model according to claim 1, characterized in that: The EfficientNetB7 network uses an inverted bottleneck structure and embeds squeeze and excitation network blocks in the bottleneck layer.

3. The cardiac mitral valve medical image segmentation model according to claim 1, characterized in that: The multi-core residual convolution module uses parallel convolution layers to splice feature maps with different convolution kernel sizes; after each convolution layer, a batch normalization layer and a ReLU activation function are used to further optimize the feature representation; at the same time, the multi-core residual convolution module adds a 1×1 convolution layer, followed by batch normalization and ReLU activation to achieve further abstraction and compression of features; in addition, the multi-core residual convolution module also introduces a residual shortcut connection, that is, an identity mapping, which is spliced ​​with the previously generated feature map through 1×1 convolution and batch normalization processing.

4. The cardiac mitral valve medical image segmentation model according to claim 1, characterized in that: The atrous spatial convolutional pyramid pooling module integrates squeeze and excitation networks.

5. The cardiac mitral valve medical image segmentation model according to claim 1, characterized in that: The hybrid triple attention module inherits the core concepts of the existing CBAM and Focus U-Net, and also achieves deep fusion and refinement of features by parallel processing of squeeze and excitation networks, channel attention and spatial attention mechanisms.

6. The cardiac mitral valve medical image segmentation model according to claim 1, characterized in that: The triple attention gate module receives two inputs: one is the gating signal from the next layer; the other is the jump connection at the same level; by processing channel attention, spatial attention and squeeze-excitation-based attention in parallel, the model is guided to focus on learning features that are efficient and closely related to the task.

7. The cardiac mitral valve medical image segmentation model according to any one of claims 1 to 6, characterized in that: The feature extraction encoder of the backend U-Net network adopts a residual convolutional network; the residual convolutional network extracts features by performing two 3×3 convolution operations, each of which is followed by batch normalization and ReLU activation function, and a 1×1 convolution shortcut residual connection is added. The shortcut residual connection performs identity mapping on the input feature map and processes it through batch normalization and ReLU activation function; subsequently, the feature maps generated by the 3×3 convolution operation and the 1×1 shortcut connection are spliced ​​and processed nonlinearly through another ReLU activation function.

8. A training method for a cardiac mitral valve medical image segmentation model, characterized in that: Training the cardiac mitral valve medical image segmentation model according to any one of claims 1 to 7 comprises the following steps: Step S11, image acquisition: collecting mitral valve image data including but not limited to cardiac CT scan, MRI imaging and echocardiography; Step S12, data annotation: a team of medical imaging experts and / or annotation personnel segments and annotates the collected imaging data; Step S13, data preprocessing: including but not limited to adjusting image size, enhancing contrast and brightness, and uniformly adjusting all images to the size required by the model; Step S14, data set division: the labeled data set is divided into a training set, a validation set and a test set; Step S15, weight initialization: initialize the model using the pre-trained EfficientNetB7 weights; Step S16, loss function design: adopt a hybrid loss function, combining binary cross entropy loss and Dice loss; the binary cross entropy loss (Loss BCE )The formula is as follows: Where y is the true label, which takes a value of 0 or 1; p is the probability of the model predicting a positive class, which ranges from [0, 1]; The formula for the Dice loss function is as follows: Where 𝑁 is the number of samples, i is the serial number of the pixel sample, and p i is the probability or label predicted by the model that the 𝑖th pixel belongs to the target category, g i is the label of whether the 𝑖th pixel belongs to the target category in the real annotation, ϵ is a small constant; the final hybrid loss is: Step S17: Set the learning rate and prepare the training environment for model training; Step S18, select evaluation indicators and implement model evaluation: select precision, recall, Dice coefficient and intersection over union (IoU) as the main evaluation indicators.

9. A method for segmenting a cardiac mitral valve medical image, characterized in that: The following steps are involved: Step S21, acquisition and preprocessing of the image to be segmented: the image to be segmented includes a real-time image and / or a pre-recorded image; the image is preprocessed to meet the input requirements of the EfficientNetB7 encoder; Step S22, performing cardiac mitral valve medical image segmentation using the cardiac mitral valve medical image segmentation model pre-trained by the training method as claimed in claim 8; the cardiac mitral valve medical image segmentation model is the cardiac mitral valve medical image segmentation model as claimed in any one of claims 1 to 7; Step S23, outputting the image after model segmentation: including but not limited to segmentation map generation, accuracy assessment and structure verification.

10. An electronic device, characterized in that: A memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the training method of claim 8 and / or the cardiac mitral valve medical image segmentation method of claim 9.

Images