Cardiac mitral valve medical image segmentation model, training method, segmentation method and device

Abstract

The application discloses a kind of heart mitral valve medical image segmentation model, training method, segmentation method and equipment.The segmentation model is improved based on DoubleU-Net architecture, and the two independent and serial U-Net networks contain encoder and decoder;Use EfficientNetB7 network as the feature extraction encoder of front-end U-Net network;Two U-Net networks are sequentially introduced into multi-core residual convolution (MKRC) module, hollow space convolution pyramid pooling (ASPP) module and mixed triple attention (TAG) module after extracting features;The input end of the decoder introduces triple attention gate (TAM) module.The segmentation model training uses hybrid loss function, combines binary cross-entropy loss and Dice loss, balances the class imbalance problem and improves the segmentation precision.The application can automatically and accurately segment heart and related vascular structures, thereby solving the technical problems that accurate and efficient heart mitral valve image segmentation is difficult in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing technology, and more specifically, to a cardiac mitral valve medical image segmentation model, training method, segmentation method, and device. Background Art

[0002] Cardiac intervention is an important treatment for cardiovascular disease. Accurate medical image segmentation is crucial for surgical navigation and robotic manipulation in mitral valve intervention. Traditional segmentation methods rely on manual annotation and image processing techniques, which are time-consuming and have limited accuracy. With the development of deep learning technology, medical image segmentation methods based on deep neural networks have demonstrated superior performance and potential.

[0003] Mitral valve image segmentation faces two major challenges: poor image quality and complex vascular morphology. X-ray angiography images suffer from significant noise and artifacts, low contrast between blood vessels and surrounding tissue, and uneven illumination, all of which affect segmentation accuracy and model robustness. Regarding vascular morphology, the vascular network has a complex branching structure with varying diameters, shapes, and orientations. Curvature, deformation, and overlap are common, making segmentation challenging.

[0004] Traditional segmentation methods based on thresholding, edge detection, or region growing struggle to adapt to the complexity and diversity of cardiac interventional imaging data. Their low segmentation accuracy makes it difficult to accurately identify subtle structures. Furthermore, their high computational resource consumption results in slow processing and low resource utilization. Their strong dependence on training data leads to poor generalization, and they are vulnerable to interference due to their lack of robustness to complex backgrounds and noise. Furthermore, their difficulty processing multimodal data limits their applicability. These issues limit the effectiveness and reliability of existing traditional methods in actual clinical applications.

[0005] Despite the success of U-Net and DoubleU-Net in the field of medical image segmentation, they still have limitations when processing cardiac interventional surgery image data. For example, they have difficulty handling curved and overlapping blood vessels, and segmentation accuracy needs to be improved. Therefore, developing a method that can accurately segment blood vessels in cardiac mitral valve interventional surgery image data is of great clinical significance. Summary of the Invention

[0006] Due to the problems existing in the existing technology, the present invention proposes a cardiac mitral valve medical image segmentation model, training method, segmentation method and equipment based on an improved DoubleU-Net model (called DoubleU-NetPlus), which can automatically and accurately segment the heart and related vascular structures, thereby solving the technical problem in the existing technology that it is difficult to accurately and efficiently perform cardiac mitral valve image segmentation.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a cardiac mitral valve medical image segmentation model, which is improved based on the DoubleU-Net architecture, wherein two independent and serially connected U-Net networks each include an encoder and a decoder;

[0008] Use the EfficientNetB7 network as the feature extraction encoder of the front-end U-Net network;

[0009] After the two U-Net networks extract features, the multi-kernel residual convolution (MKRC) module, the atrous spatial convolution pyramid pooling (ASPP) module and the hybrid triple attention (TAG) module are introduced in sequence; the triple attention gate (TAM) module is introduced at the input end of the decoder.

[0010] In some embodiments of the first aspect of the present application, the EfficientNetB7 network uses an inverted bottleneck structure and embeds a squeeze and excitation network (SE-Net) block in the bottleneck layer.

[0011] In some embodiments of the first aspect of the present application, the MKRC block uses parallel convolution layers to splice feature maps with different convolution kernel sizes; after each convolution layer, a batch normalization layer and a ReLU activation function are used to further optimize the feature representation; at the same time, the multi-core residual convolution module adds a 1×1 convolution layer, followed by batch normalization and ReLU activation to achieve further abstraction and compression of features; in addition, the multi-core residual convolution module also introduces a residual shortcut connection, that is, an identity mapping, which is spliced ​​with the previously generated feature map through 1×1 convolution and batch normalization processing.

[0012] In some embodiments of the first aspect of the present application, the ASPP module integrates the squeeze and excitation network as a SE-ASPP module.

[0013] In some embodiments of the first aspect of the present application, the TAG module inherits the core concepts of the existing CBAM and Focus U-Net, and also achieves deep fusion and refinement of features by parallel processing of squeeze and excitation networks, channel attention and spatial attention mechanisms.

[0014] In some embodiments of the first aspect of the present application, the TAM module receives two inputs: one is the gating signal from the next layer; the other is the jump connection at the same level; by processing channel attention, spatial attention and squeeze-excitation-based attention in parallel, the model is guided to focus on learning those features that are efficient and closely related to the task.

[0015] In some embodiments of the first aspect of the present application, the feature extraction encoder of the back-end U-Net network adopts a residual convolutional network; the residual convolutional network extracts features by performing two 3×3 convolution operations, each operation is followed by batch normalization and ReLU activation function, and a 1×1 convolution shortcut residual connection is added. The shortcut residual connection performs identity mapping on the input feature map and processes it through batch normalization and ReLU activation function; subsequently, the feature map generated by the 3×3 convolution operation and the 1×1 shortcut connection is spliced ​​and nonlinearly processed through another ReLU activation function.

[0016] In a second aspect, the present invention provides a training method for a cardiac mitral valve medical image segmentation model, characterized in that the training method comprises the following steps:

[0017] Step S11, image acquisition: collecting mitral valve image data including but not limited to cardiac CT scan, MRI imaging and echocardiography;

[0018] Step S12, data annotation: A team composed of medical imaging experts and / or annotation personnel segments and annotates the collected image data;

[0019] Step S13, data preprocessing: including but not limited to adjusting image size, enhancing contrast and brightness, and uniformly adjusting all images to the size required by the model;

[0020] Step S14, data set division: the labeled data set is divided into a training set, a validation set, and a test set;

[0021] Step S15, weight initialization: initialize the model using the pre-trained EfficientNetB7 weights;

[0022] Step S16, loss function design: adopt hybrid loss function, combining binary cross entropy loss and Dice loss; binary cross entropy loss (Loss BCE )The formula is as follows:

[0023]

[0024] Where y is the true label, which takes a value of 0 or 1; p is the probability of the model predicting the positive class, which ranges from [0,1];

[0025] The formula for the Dice loss function is as follows:

[0026]

[0027] Where N is the number of samples, i is the sequence number of the pixel sample,p i The model predicts i The probability or label of a pixel belonging to the target class, g i For the i The label of whether the pixel belongs to the target category in the real annotation, ϵ is a small constant; the final mixed loss is:

[0028] ;

[0029] Step S17: Set the learning rate and prepare the training environment for model training;

[0030] Step S18: Select evaluation indicators and implement model evaluation: Precision, recall, Dice coefficient, and intersection over union (IoU) are selected as the main evaluation indicators.

[0031] In a third aspect, the present invention provides a method for segmenting a cardiac mitral valve medical image, characterized in that it comprises the following steps:

[0032] Step S21, acquiring and preprocessing the image to be segmented: the image to be segmented includes a real-time image and / or a pre-recorded image; the image is preprocessed to meet the input requirements of the EfficientNetB7 encoder;

[0033] Step S22: performing cardiac mitral valve medical image segmentation using the cardiac mitral valve medical image segmentation model pre-trained by the training method described above; the cardiac mitral valve medical image segmentation model is the cardiac mitral valve medical image segmentation model described above;

[0034] Step S23: Output the image after model segmentation: including but not limited to segmentation map generation, accuracy assessment and structure verification.

[0035] In a fourth aspect, the present invention provides an electronic device, characterized in that it comprises a memory storing executable program code; a processor coupled to the memory; and the processor calls the executable program code stored in the memory to execute the training method as described above and / or the cardiac mitral valve medical image segmentation method as described above.

[0036] Compared with the prior art, the present invention has the following technical effects:

[0037] (1) The DoubleU-NetPlus model of the present invention adopts a two-layer U-Net structure. Compared with the traditional U-Net network, the two independent U-Net networks of DoubleU-NetPlus both contain an encoder and a decoder. The encoder is used to extract features, and the decoder is used to reconstruct images. The combination of the two enables the model to more accurately capture subtle features in the image. Especially when processing images of cardiac mitral valve interventional surgery, this structure can effectively deal with the complex morphology and low contrast problems of blood vessels, providing more accurate segmentation results.

[0038] (2) The DoubleU-NetPlus model of the present invention uses EfficientNetB7 as the encoder for feature extraction. Compared with the traditional VGG19 encoder used by DoubleU-Net, this encoder extracts image features through depthwise separable convolution and moving inverted bottleneck convolution, and can automatically adjust the depth, width and resolution of the network, thereby improving the efficiency and accuracy of image feature extraction.

[0039] (3) The DoubleU-NetPlus model of the present invention introduces the MKRC module, which enables the capture of features at different scales through parallel multi-size convolution kernels. Multi-kernel convolution can simultaneously consider local and global features. This multi-scale feature capture is particularly important for processing complex vascular structures in cardiac interventional surgery images. It can effectively deal with vascular curvature, deformation, and overlap, providing more accurate segmentation results.

[0040] (4) The TAM module in the DoubleU-NetPlus model of the present invention integrates spatial attention, channel attention, and squeeze-excited attention mechanisms, and enhances the relevance of feature maps through parallel processing. The spatial attention mechanism enables the model to focus on important areas in the image, improving the accuracy of segmentation. The channel attention mechanism enhances the inter-channel correlation of feature maps and improves the expressiveness of features. The squeeze-excited attention mechanism further optimizes the distribution of features by reweighting the feature maps, thereby enhancing the segmentation ability of the model.

[0041] (5) The TAG module in the DoubleU-NetPlus model of the present invention can selectively focus on the target region in the skip connection, suppress the features of irrelevant regions, and reduce the interference of irrelevant information. In the images of mitral valve interventional surgery, the contrast between blood vessels and surrounding tissues is low. The TAG module can effectively focus on the blood vessel region and provide more accurate segmentation results.

[0042] (6) The DoubleU-NetPlus model of the present invention makes innovative improvements to ASPP and proposes the SE-ASPP module, which further enhances the field of view of the convolutional filter by integrating the squeeze and excitation network (SE-Net). The present invention adopts a deeper dilated convolution technique in the SE-ASPP module, which enables the network to perceive local and global features in the mitral valve images in a more refined manner.

[0043] (7) The DoubleU-NetPlus model training of the present invention adopts a hybrid loss function, combining binary cross entropy loss and Dice loss, to balance the class imbalance problem and improve segmentation accuracy. Binary cross entropy loss ensures pixel-level classification accuracy, while Dice loss focuses on the overlap between the predicted area and the true area. The combination of the two enables the model to more accurately identify and segment vascular areas, improving the overall performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 FIG. 4 is an architectural diagram of the DoubleU-NetPlus segmented network model in one embodiment of the present invention.

[0045] Figure 2 Schematic diagram of the AG-Residual convolution module in one embodiment of the present invention.

[0046] Figure 3 Schematic diagram of a MKRC module in one embodiment of the present invention.

[0047] Figure 4 FIG. 1 is a schematic diagram of a SE-ASPP module in an embodiment of the present invention.

[0048] Figure 5 Schematic diagram of a TAM module in one embodiment of the present invention.

[0049] Figure 6 FIG. 1 is a schematic diagram of a TAG module in an embodiment of the present invention.

[0050] Figure 7 FIG. 4 is a flowchart of implementing medical image segmentation of the mitral valve in one embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0052] In the following detailed description, many specific details are set forth to provide a more thorough understanding of the present invention. However, it is obvious to those skilled in the art that well-known algorithms are not shown in detail to avoid obscuring the subject matter of the present invention.

[0053] In addition, the order of execution of actions, steps, etc. in the devices and methods shown in the claims, specifications and drawings can be implemented in any order as long as there is no special explicit limitation on the order and the output of the previous processing is not used in the subsequent processing.

[0054] Example 1

[0055] See also Figure 1 This embodiment provides a segmentation model for cardiovascular images (particularly mitral valve images) based on the DoubleU-NetPlus model. DoubleU-NetPlus is a deep learning-based dual U-Net architecture, an improvement on DoubleU-Net, designed to improve the segmentation accuracy of medical images. This model combines multiple network structures and technologies, innovatively incorporating the EfficientNetB7 encoder, MKRC module, SE-ASPP module, TAG module, and TAM module.

[0056] As a preferred specific example, see Figure 1 The DoubleU-NetPlus model uses a two-layer U-Net structure, connecting two U-Net networks in series, divided into Network 1 and Network 2. This dual-layer structure enhances the model's deep feature extraction capabilities. Each network consists of an encoder and a decoder, achieving image segmentation through feature extraction and reconstruction. After feature extraction, the decoder of each network introduces the MKRC, SE-ASPP, and TAM modules.

[0057] In the U-Net architecture, the encoder is responsible for gradually reducing the spatial size of the image while increasing the number of feature channels to extract high-level semantic features. This process is achieved through multiple convolutional and pooling layers, ultimately outputting highly compressed feature maps for the decoder to reconstruct and predict segmentation. In the DoubleU-NetPlus architecture, we have made the following improvements to the encoder: We abandoned the traditional VGG19 and used the pre-trained EfficientNet B7 network as the encoder backbone of Network 1, performing feature extraction through transfer learning. The encoder of Network 2 adopts a residual convolution structure, achieving feature extraction through two 3×3 convolutional layers and skip connections. Residual convolution can effectively alleviate the vanishing gradient problem and improve model training efficiency and segmentation accuracy.

[0058] The specific structure of the EfficientNet B7 network in Network1 in this embodiment is as follows: Stage 1: Conv3x3 input 600x600, output 32; MBConv1 input 300×300, output 16, repetition 1, expansion factor 1, SE ratio 0.25. Stage 2: MBConv6 input 300×300, output 24, repetition 2, expansion factor 6, SE ratio 0.25. Stage 3: MBConv6 input 150×150, output 40, repetition 2, expansion factor 6, SE ratio 0.25. Stage 4: MBConv6 input 75×75, output 80, repetition 3, expansion factor 6, SE ratio 0.25. Stage 5: MBConv6 input 38×38, output 112, repetition 3, expansion factor 6, SE ratio 0.25. EfficientNetB7 offers the following advantages: It utilizes an inverted bottleneck architecture, achieving efficient computation through depthwise separable convolutions and pointwise convolutions, while reducing the number of model parameters. Furthermore, EfficientNetB7 embeds SE-Net blocks in the bottleneck layer, enhancing feature representation through a channel-wise attention mechanism and improving segmentation performance. Most importantly, EfficientNetB7 employs a compound scaling approach, systematically adjusting the network's depth, width, and resolution to achieve optimal performance with fixed computing resources.

[0059] As a further preferred example, each encoder block in Network 2 performs AG-residual convolution operations, with the specific structure as follows: Figure 2 As shown in Figure 2, the AG-residual convolution module extracts features by performing two 3×3 convolution operations, each followed by batch normalization and ReLU activation functions. The batch normalization function is:

[0060]

[0061] x is the input data, is the mean of the current batch of data, is the standard deviation of the current batch of data, and are learnable parameters that are used to scale and translate the normalized data, allowing the network to learn the original representation of the data. Batch normalization can reduce internal covariate shift and standardize the model. The ReLU activation function introduces nonlinearity, and the formula is:

[0062]

[0063] The ReLU activation function can enhance the nonlinear expression ability of the model. To reduce the gradient vanishing problem, a 1×1 convolution shortcut residual connection is added to the module, which performs identity mapping on the input feature map and processes it through batch normalization and ReLU activation function. Subsequently, the feature map generated by the 3×3 convolution operation and the 1×1 shortcut connection is spliced ​​and nonlinearly processed through another ReLU activation function. Finally, the generated feature map is input into the TAM module for further attention mechanism processing to highlight relevant features. The last operation of the module is a max-pooling operation using a 2×2 window and a 2×2 stride to reduce the spatial dimension of the feature map in preparation for the subsequent decoding process.

[0064] In the mitral valve image segmentation task, a major challenge is that the size and shape of the objects to be segmented in the image may vary greatly. To effectively address this challenge, the key is to extract high-level multi-scale contextual features through different receptive fields. To this end, the design of the model in this embodiment introduces a multi-kernel residual convolution (MKRC) module inspired by the Inception architecture. This module plays a role after the decoder operations of Network 1 and Network 2. Figure 3 As shown, the MKRC module expands the feature field of view through parallel convolutional layers using different convolution kernel sizes such as 1×1, 3×3, 5×5, and 7×7, thereby performing differentiated feature learning for each feature block and enhancing the model's ability to learn and recognize heterogeneous features. After each convolutional layer, this model also uses a batch normalization layer and ReLU activation function to further optimize feature representation. Subsequently, the four feature maps are concatenated to integrate information from different receptive fields. To further enhance the expressiveness of features, the module adds a 1×1 convolution layer followed by batch normalization and ReLU activation to achieve further feature abstraction and compression. In addition, the module introduces a residual shortcut connection, or identity mapping, which is concatenated with the previously generated feature map through 1×1 convolution and batch normalization. This design not only ensures smooth information transmission in the network but also effectively reduces the risk of overfitting. The feature map finally output by the MKRC module can be expressed as:

[0065]

[0066] Among them, MKRC_FeatureFusion is the MKRC multi-scale fusion feature, which can be expressed as follows:

[0067]

[0068] Where Concat represents concatenating the outputs of convolutional layers with different kernel sizes, which are then processed through batch normalization and residual connections. Finally, the generated feature maps are fed into an improved SE-ASPP module.

[0069] Atrous Spatial Convolution Pooling Pyramid (ASPP) refers to the introduction of parallel atrous convolution layers with different dilation rates, which significantly expands the receptive field of the filter and enables it to capture multi-scale semantic feature context. This innovative method effectively solves the problem of reduced spatial resolution caused by multiple downsampling in traditional encoders. In order to more effectively capture multi-scale contextual features in medical images and improve segmentation accuracy and model robustness, we have made innovative improvements to ASPP and proposed the SE-ASPP module, which further enhances the field of view of the convolution filter by integrating the squeeze and excitation network (SE-Net). Figure 4 As shown in the figure, we use a deeper dilated convolution technique in the SE-ASPP module. Specifically, the seven parallel convolution layers use dilation rates of 1, 1, 2, 6, 10, 13, and 16 respectively. This design enables the network to perceive the local and global features in the image more finely. Through the integration of SE-Net, we effectively recalibrate and refine these features to ensure the quality and accuracy of the feature maps. In the SE-ASPP module, the feature maps generated by the SE-Net module of each branch are cleverly connected together and then further integrated through a 1×1 convolution operation. This step not only enhances the expressive power of the features, but also provides a more compact feature representation for subsequent processing. Subsequently, the application of batch normalization and ReLU activation function further optimizes the distribution of the feature maps and enhances the network's responsiveness to features. The dilated convolution can be expressed as:

[0070]

[0071] Where x is the input feature map, k is the convolution kernel, and d is the dilation rate. The dilate function dilates the input feature map by the dilation rate d. The Atrous Spatial Convolution Pooling Pyramid (ASPP) can be expressed as:

[0072]

[0073] Where n is the number of convolutional layers with different expansion rates, k i and d i is the size and dilation rate of the convolution kernel of the i-th convolution layer. The SE module can be expressed as:

[0074]

[0075] Where σ is the activation function (here is ReLU), and are learnable parameters, is the global average pooling operation, represents element-by-element multiplication. Thus the SE-ASPP module can be expressed as

[0076]

[0077] in It is the result of processing the feature map output by the ASPP module with the SE module. The process of SE-ASPP feature refinement can be expressed as:

[0078]

[0079] Where m is the number of SE-ASPP modules. The final feature map is expressed as:

[0080]

[0081] These carefully processed feature maps are fed into the hybrid triple attention module (TAM) for deeper feature extraction and optimization.

[0082] The hybrid triple attention module (TAM) is an innovative attention mechanism that not only inherits the core concepts of the existing CBAM and Focus U-Net, but also achieves deep fusion and refinement of features through parallel processing of squeeze and excitation networks, optimized channel attention and spatial attention mechanisms. Figure 5 As shown in the figure, TAM uses this parallel processing strategy to deeply explore the complex relationships between features, improve the expressiveness of features, and thus enhance the model's recognition accuracy of target areas. TAM uses the channel attention mechanism of SE-Net to adjust channel weights, optimize the interdependence between channels, and effectively suppress background noise, thereby highlighting the area to be segmented in cardiovascular images. Among them, spatial attention: , channel attention: , squeezing stimulates attention: The TAM module integrates the three and can be expressed as:

[0083] On this basis, this embodiment innovatively introduces the triple attention gate module (TAG). This innovative structure significantly improves the representation ability of the model. At the same time, experiments have shown that it can effectively control the growth of computational costs and model parameters. Like the existing attention gate and focus gate modules, the TAG module integrates multiple attention mechanisms in one. By processing channel attention, spatial attention, and squeeze-based attention in parallel, it guides the model to focus on learning those features that are efficient and closely related to the task, such as vascular features in cardiovascular images. Figure 6 As shown, the TAG module cleverly receives two inputs: a gating signal from the next layer, which has been trained to capture key features such as edges, textures, and points; and skip connections within the same layer, which carry rich spatial information. To fuse these two inputs, we first resize them to the same size and then generate attention coefficients through element-wise addition and a ReLU nonlinear activation function. These coefficients are then fed into carefully designed channel-wise, spatial, and squeeze-excitation attention modules, ultimately merging into a refined feature representation that highlights key regions in the image. To further enhance the spatial resolution of the features, we employ a 2×2 upsampling technique to match the output size of the 1×1 convolutional layer and perform a sigmoid operation on the output of the nonlinear activation function to adjust the weight distribution. This strengthens weights aligned with the target region, while weakening other weights. By combining the original skip connections with the newly generated attention coefficients, we can accurately capture the spatial context of the region of interest, enabling the model to intelligently adjust weights based on feature relevance. The introduction of the TAG module not only enriches the model's ability to understand and analyze medical images, but also provides strong support for achieving accurate image segmentation, especially when processing mitral valve interventional surgery images with complex backgrounds and subtle structures.

[0084] Example 2

[0085] This embodiment provides a training method for the DoubleU-NetPlus model described in Example 1.

[0086] Before model training, a large amount of labeled medical imaging data is required. This data will be used for model training and validation. Data preprocessing, including image cropping, scaling, rotation, and augmentation, improves the model's robustness and generalization capabilities. Next, a loss function is designed, and an appropriate learning rate and training strategy are selected. Finally, a validation set is used to evaluate the model's accuracy and robustness.

[0087] In the mitral valve image segmentation task, in order to make the model more suitable for the task, collecting and labeling the dataset by yourself is a key step to ensure the effectiveness of model training.

[0088] S11. Image Acquisition: Collaborate with medical institutions to collect mitral valve imaging data, including cardiac CT scans, MRI imaging, and echocardiography. Ensure compliance with all relevant privacy protection and ethical regulations during the acquisition process.

[0089] S12. Data Annotation: A team of professional medical imaging experts and well-trained annotators accurately segment and annotate the collected imaging data. Using professional image annotation tools, we ensure the accuracy and consistency of the annotations.

[0090] S13. Data preprocessing: The collected original images are preprocessed as necessary, including adjusting the image size, enhancing the contrast and brightness, and uniformly adjusting all images to the size required by the model. In the present invention, the size is uniformly adjusted to 600×600 pixels.

[0091] Ideally, data augmentation should be performed on the images: To improve the generalization and robustness of the model, data augmentation strategies are implemented on the training data. In our experiments, we used a variety of augmentation techniques to ensure that the small number of samples in the dataset does not overfit. We employed a variety of data augmentation techniques, including random rotation, grid distortion, horizontal and vertical flips, transposition, vertical flip and random rotation, random brightness, random contrast, random brightness contrast, random gamma, hue saturation contrast, and RGB shifting, to simulate image characteristics under different imaging conditions.

[0092] Model training is the core step in achieving cardiovascular segmentation and requires careful design to optimize model performance.

[0093] S14. Dataset Partitioning: The labeled dataset consists of 240 samples, 80% of which are divided into a training set for the main training process of the model. The remaining 20% ​​of the data is divided into a validation set and a test set for model verification and final evaluation, respectively.

[0094] S15. Weight initialization: Use the pre-trained EfficientNetB7 weights to initialize the model. This helps the model have better feature extraction capabilities at the beginning of training, accelerates the convergence process, and improves the final segmentation accuracy.

[0095] S16. Loss Function Design: A hybrid loss function, combining binary cross-entropy loss and Dice loss, balances the class imbalance and improves segmentation accuracy. Binary cross-entropy loss ensures pixel-level classification accuracy, while Dice loss focuses on the overlap between the predicted and true regions. The combination of these two enables the model to more accurately identify and segment vascular regions, improving overall model performance.

[0096] Among them, binary cross entropy loss (LossBCE): Binary cross entropy loss is a loss function for binary classification problems, which measures the difference between the predicted probability distribution and the true label distribution. The LossBCE formula is as follows:

[0097]

[0098] Where y is the true label, which takes the value of 0 or 1. p is the probability of the model predicting the positive class, which ranges from [0,1].

[0099] The Dice loss function is a loss function used in image segmentation tasks to measure the difference between the model prediction results and the true labels. It is particularly suitable for processing unbalanced datasets. The specific formula is as follows:

[0100]

[0101] Where N is the number of samples, i is the sequence number of the pixel sample, p i is the probability or label predicted by the model that the i-th pixel belongs to the target category, g i is the label of whether the i-th pixel belongs to the target category in the real annotation, It is a small constant. In order to avoid the error of dividing by zero, a very small value such as 0.00001 is usually added to ensure numerical stability.

[0102] The final hybrid loss is shown below:

[0103]

[0104] S17. Learning rate setting: Set the initial learning rate to 0.0001, which helps the model converge quickly in the early stages of training. As training progresses, use a learning rate decay strategy to gradually reduce the learning rate to ensure that the model can fine-tune weights in the later stages of training to avoid overfitting.

[0105] Training environment: The GPU is an NVIDIA GeForce RTX 4090 with 24GB of video memory. The CPU is an Intel Xeon E5-2690v4 @ 2.60GHz with 32GB of RAM. The deep learning framework is TensorFlow 2.0.

[0106] S18. Evaluation is an important step in measuring the segmentation effect of the model. The accuracy and reliability of the model are evaluated through a series of quantitative indicators:

[0107] Evaluation Metrics: Precision, Recall, Dice coefficient, and Intersection over Union (IoU) are used as the primary evaluation metrics. Precision refers to the proportion of samples predicted as positive by the model that are actually positive. Recall refers to the proportion of samples correctly predicted as positive by the model among all samples that are actually positive. The Dice coefficient measures the overlap between the predicted segmented region and the ground-truth annotated region, while IoU focuses on the alignment of the boundaries between the predicted and ground-truth regions.

[0108] The specific formula is as follows:

[0109]

[0110]

[0111]

[0112]

[0113] TP (True Positive): True Positive, refers to the number of samples correctly predicted as positive by the model. FP (False Positive): False Positive, refers to the number of samples incorrectly predicted as positive by the model. TN (True Negative): True Negative, refers to the number of samples correctly predicted as negative by the model. FN (False Negative): False Negative, refers to the number of samples incorrectly predicted as negative by the model.

[0114] Evaluation process implementation: The model is evaluated on the validation set, and the precision, recall, Dice coefficient, and IoU value of the samples are calculated to evaluate the overall performance. This process helps identify potential problems in the model in the segmentation task and provides guidance for further optimization.

[0115] In order to verify the beneficial effects of the present invention, the inventors conducted comparative simulation experiments using the mitral valve region image segmentation method based on the DoubleU-NetPlus model of Example 1 of the present invention and the DoubleU-Net method and the U-Net method. The experimental comparison results are shown in Table 1.

[0116] Table 1 Comparison of evaluation results of image segmentation simulation experiments based on DoubleU-NetPlus model, DoubleU-Net model, and U-Net model

[0117]

[0118] Through comparison, it can be seen from the four evaluation indicators that the DoubleU-NetPlus of the present invention has better segmentation effect than the DoubleU-Net and U-Net models, which proves the advantage of the improved model of the present invention in the mitral valve image segmentation task.

[0119] Example 3

[0120] See also Figure 7 This embodiment provides a method for performing cardiac mitral valve medical image segmentation using the DoubleU-NetPlus model described in Example 1 that is trained using the training method of Example 2.

[0121] S21. Image Acquisition and Preprocessing for Segmentation: Acquire real-time or pre-recorded medical imaging data from the mitral valve interventional surgery robotic system. This data can be generated by CTA, MRI, or other imaging technologies. Perform necessary preprocessing on the acquired images to meet the model's input requirements.

[0122] More specifically, the images to be segmented come from two main sources: Real-time images: During surgery, the imaging device onboard the interventional surgical robot captures cardiac images in real time. These images are crucial for surgical navigation and real-time decision-making. Pre-recorded images: Before surgery, the patient may have undergone a series of diagnostic imaging examinations. These pre-recorded images can provide important information for surgical planning and model training.

[0123] S22. Image preprocessing: The acquired image is adjusted to a uniform size and resolution. In this embodiment, it is set to 600×600 pixels to meet the input requirements of the EfficientNetB7 encoder.

[0124] In addition, appropriate image contrast enhancement can be performed: adjusting the image contrast to highlight the mitral valve and surrounding structures, further enhancing the model's ability to recognize details. Noise removal: applying filtering algorithms to remove noise and artifacts from the image, improving image quality and reducing interference with model performance.

[0125] S23. Output the segmented image: Use the trained DoubleU-NetPlus model to segment the preprocessed image and generate a segmentation result. The model output is a segmentation map of the same size as the input image, where different regions are labeled as different categories.

[0126] The segmented image is the key output of the mitral valve interventional surgery robotic system, which directly affects the surgical navigation and treatment effect. The specific process is as follows:

[0127] Model Application: The pre-processed image is segmented using the trained DoubleU-NetPlus model. The model learns the image's features and structure to identify and segment the mitral valve region.

[0128] Segmentation map generation: The segmentation map output by the model distinguishes the mitral valve area of ​​the heart from other areas, and each area is labeled as a different category, usually represented by a different color or label.

[0129] Accuracy assessment: By calculating indicators such as precision, recall, Dice coefficient, and IoU (Intersection over Union), the accuracy and reliability of the segmentation results are evaluated to ensure that the quality of the segmentation map meets clinical needs.

[0130] Better yet, perform image optimization: perform post-processing on the segmentation results, such as edge smoothing and hole filling, to improve the visual effect and practicality of the segmentation map.

[0131] Result Verification: Medical experts verify the segmentation results to ensure that the segmentation map accurately reflects the actual position and morphology of the mitral valve.

[0132] Example 4

[0133] This embodiment provides an electronic device comprising a memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the model described in Example 1, the training method described in Example 2, and / or the cardiovascular region image (especially mitral valve medical image) segmentation method described in Example 3.

[0134] The above-mentioned segmentation model and its training method can be embodied in the form of a computer program product or a software functional unit. If the above-mentioned segmentation model and its training method are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Therefore, the essence of this technical solution or the part that contributes to the existing technology or the part of this technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic system (which can be a personal computer, server, or network system, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program code.

[0135] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] In summary, the present invention provides a segmentation model, training method, segmentation method, and device for cardiac mitral valve medical images. The segmentation model is improved based on the DoubleU-Net architecture. Its two independent and serially connected U-Net networks both include an encoder and a decoder. The EfficientNetB7 network is used as the feature extraction encoder of the front-end U-Net network. After the two U-Net networks extract features, the multi-kernel residual convolution (MKRC) module, the atrous spatial convolution pyramid pooling (ASPP) module, and the hybrid triple attention (TAG) module are sequentially introduced. The input of the decoder is introduced with a triple attention gate (TAM) module. The segmentation model training uses a hybrid loss function, combining binary cross entropy loss and Dice loss, to balance the class imbalance problem and improve segmentation accuracy. The present invention can automatically and accurately segment the heart and related vascular structures, thereby solving the technical problem of the difficulty in accurately and efficiently performing cardiac mitral valve image segmentation in the prior art.

[0137] Those skilled in the art should understand that they can implement variations by combining the prior art with the above embodiments, which will not be described in detail here. Such variations do not affect the essence of the present invention and will not be described in detail here.

[0138] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the systems and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-mentioned disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention that do not depart from the content of the technical solutions of the present invention are still within the scope of protection of the technical solutions of the present invention.

Claims

1. A cardiac mitral valve medical image segmentation model, characterized by: Improved based on the DoubleU-Net architecture, its two independent and serially connected U-Net networks both contain encoders and decoders; Use the EfficientNetB7 network as the feature extraction encoder of the front-end U-Net network; the EfficientNetB7 network uses an inverted bottleneck structure and embeds squeeze and excitation network blocks in the bottleneck layer; After the two U-Net networks extract features, the multi-core residual convolution module, the void spatial convolution pyramid pooling module and the hybrid triple attention module are introduced in sequence; the triple attention gate module is introduced at the input end of the decoder; the triple attention gate module receives two inputs: one is the gating signal from the next layer; the other is the jump connection at the same level; by processing channel attention, spatial attention and attention based on squeeze excitation in parallel, the model is guided to focus on learning task-related features including vascular features; the multi-core residual convolution module uses parallel convolution layers to splice feature maps with different convolution kernel sizes; after each convolution layer, Batch normalization layer and ReLU activation function are used to further optimize feature representation; at the same time, the multi-core residual convolution module adds a 1×1 convolution layer, followed by batch normalization and ReLU activation to achieve further abstraction and compression of features; in addition, the multi-core residual convolution module also introduces residual shortcut connection, which is spliced ​​with the previously generated feature map through 1×1 convolution and batch normalization processing; the void spatial convolution pyramid pooling module integrates the squeeze and excitation network; the hybrid triple attention module achieves deep fusion and refinement of features by parallel processing of the squeeze and excitation network, channel attention and spatial attention mechanism.

2. The cardiac mitral valve medical image segmentation model according to claim 1, characterized in that: The feature extraction encoder of the back-end U-Net network adopts a residual convolutional network; the residual convolutional network extracts features by performing two 3×3 convolution operations, each followed by batch normalization and ReLU activation function, and adding a 1×1 convolution shortcut residual connection. The shortcut residual connection performs identity mapping on the input feature map and processes it through batch normalization and ReLU activation function; then, the feature map generated by the 3×3 convolution operation and the 1×1 shortcut connection is spliced ​​and processed nonlinearly through another ReLU activation function.

3. A training method for a cardiac mitral valve medical image segmentation model, characterized in that: Training the cardiac mitral valve medical image segmentation model according to claim 1 or 2 comprises the following steps: Step S11, image acquisition: collecting mitral valve image data including but not limited to cardiac CT scan, MRI imaging and echocardiography; Step S12, data annotation: A team composed of medical imaging experts and / or annotation personnel segments and annotates the collected image data; Step S13, data preprocessing: including but not limited to adjusting image size, enhancing contrast and brightness, and uniformly adjusting all images to the size required by the model; Step S14, data set division: the labeled data set is divided into a training set, a validation set, and a test set; Step S15, weight initialization: initialize the model using the pre-trained EfficientNetB7 weights; Step S16, loss function design: adopt hybrid loss function, combining binary cross entropy loss and Dice loss; binary cross entropy loss (Loss BCE )The formula is as follows: in y is the true label, which takes the value of 0 or 1; p is the probability that the model predicts the positive class, and its value range is (0, 1); The formula for the Dice loss function is as follows: Where 𝑁 is the number of samples, i is the sequence number of the pixel sample, p i is the probability or label predicted by the model that the 𝑖th pixel belongs to the target category, g i is the label of whether the 𝑖th pixel belongs to the target category in the real annotation, ϵ is a small constant; the final mixed loss is: ； Step S17: Set the learning rate and prepare the training environment for model training; Step S18: Select evaluation indicators and implement model evaluation: Precision, Recall, Dice coefficient, and Intersection over Union (IoU) are selected as evaluation indicators.

4. A cardiac mitral valve medical image segmentation method, characterized in that: The following steps are involved: Step S21, acquiring and preprocessing the image to be segmented: the image to be segmented includes a real-time image and / or a pre-recorded image; the image is preprocessed to meet the input requirements of the EfficientNetB7 encoder; Step S22: Utilize the cardiac mitral valve medical image segmentation model pre-trained by the training method according to claim 3 to perform cardiac mitral valve medical image segmentation; the cardiac mitral valve medical image segmentation model is the cardiac mitral valve medical image segmentation model according to claim 1 or 2; Step S23: Output the image after model segmentation: including but not limited to segmentation map generation, accuracy assessment and structure verification.

5. An electronic device, characterized in that: A memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the training method according to claim 3 and / or the cardiac mitral valve medical image segmentation method according to claim 4.

Images