A training method, a segmentation method and a system of an image segmentation model

Abstract

The application discloses a kind of training method, segmentation method and system of image segmentation model, wherein the training method obtains the training image of coronary artery;Double-branch feature extraction is carried out on the training image by first branch network and second branch network, to obtain the first target feature output by the first branch network and the second target feature output by the second branch network, the network type of the first branch network is different from the network type of the second branch network;Branch feature fusion processing is carried out on the first target feature and the second target feature, to obtain target fusion feature;According to the target fusion feature, the initialized image segmentation model is updated, to obtain the trained image segmentation model.The training method can effectively improve the global information capture capability of the model on coronary artery image, and is beneficial to improve the segmentation effect of coronary artery image segmentation.The application relates to the field of image processing technology.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a training method, segmentation method and system for an image segmentation model. Background Technology

[0002] With the development of medical technology, the automatic segmentation of coronary artery images acquired through coronary computed tomography angiography (CCTA) has gradually become one of the key focuses for practitioners.

[0003] Currently, existing techniques for automatic segmentation of coronary artery images are typically based on convolutional neural networks (CNNs). However, due to the relatively fixed kernel size of CNNs, their receptive field is limited. This means the network model cannot effectively capture global information in the coronary artery image, often resulting in discontinuous segments and unsatisfactory segmentation outcomes.

[0004] Therefore, the problems existing in the current technology still need to be solved and optimized. Summary of the Invention

[0005] The purpose of this invention is to at least partially solve one of the technical problems existing in the related art.

[0006] Therefore, one objective of this invention is to provide a training method, a segmentation method, and a system for an image segmentation model. The training method can effectively improve the model's ability to capture global information of coronary artery images, thereby improving the segmentation effect of coronary artery image segmentation.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include:

[0008] In a first aspect, embodiments of this application provide a method for training an image segmentation model, comprising:

[0009] Obtain training images of the coronary arteries;

[0010] The training image is subjected to bi-branch feature extraction through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network. The network types of the first branch network and the second branch network are different. The first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image.

[0011] The first target feature and the second target feature are subjected to branch feature fusion processing to obtain the target fusion feature;

[0012] Based on the target fusion features, the parameters of the initialized image segmentation model are updated to obtain a trained image segmentation model.

[0013] In addition, the method according to the above embodiments of this application may also have the following additional technical features:

[0014] Furthermore, in one embodiment of this application, the step of performing bi-branch feature extraction on the training image through a first branch network and a second branch network to obtain the first target feature and the second target feature includes:

[0015] The first branch network is used to extract local features from the training image to obtain several first branch features with different feature levels.

[0016] The training image is subjected to global feature extraction through the second branch network to obtain several second branch features with different feature levels.

[0017] For each third branch feature and its corresponding fourth branch feature, inter-layer feature fusion is performed to obtain an inter-layer fused feature corresponding to each feature level. The third branch feature is any first branch feature other than the first branch feature of the last feature level, and the fourth branch feature is a second branch feature with the same feature level as the third branch feature.

[0018] Based on the first branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the first target feature.

[0019] Based on the second branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the second target feature.

[0020] Furthermore, in one embodiment of this application, the step of extracting local features from the training image to obtain several first branch features with different feature levels includes:

[0021] Obtain input features, wherein the input features are image features of the training image or the first branch features of the previous feature level;

[0022] The input features are subjected to first-branch feature extraction to obtain intermediate features;

[0023] The intermediate features are downsampled to obtain the first branch features of the current feature level.

[0024] Furthermore, in one embodiment of this application, the step of performing global feature extraction on the training image to obtain several second-branch features with different feature levels includes:

[0025] The training image is subjected to feature convolution block processing to obtain the second branch features of the first feature level;

[0026] Multi-layer global attention extraction is performed on the second branch features of the first feature level to obtain several fifth branch features, which are second branch features other than the second branch features of the first feature level.

[0027] Furthermore, in one embodiment of this application, the step of performing inter-layer feature fusion on each third branch feature and the corresponding fourth branch feature to obtain an inter-layer fused feature corresponding to each feature level includes:

[0028] The third branch features and the fourth branch features are concatenated to obtain inter-layer concatenated features;

[0029] Channel attention is extracted from the interlayer stitching features to obtain a channel attention map;

[0030] Based on the channel attention map, channel attention fusion is performed on the inter-layer splicing features to obtain inter-layer channel features;

[0031] The interlayer fusion feature is obtained based on the interlayer channel feature, the third branch feature, and the fourth branch feature.

[0032] Furthermore, in one embodiment of this application, the step of performing branch feature fusion processing on the first target feature and the second target feature to obtain target fusion features includes:

[0033] The first target feature and the second target feature are concatenated to obtain the branch concatenated feature;

[0034] The branch splicing features are subjected to bi-branch parallel clustering processing to obtain a first clustering feature map and a second clustering feature map;

[0035] Spatial attention is extracted from the first cluster feature map and the second cluster feature map to obtain a spatial attention map;

[0036] Based on the spatial attention map, spatial attention fusion is performed on the first target feature and the second target feature to obtain the target fusion feature.

[0037] Secondly, embodiments of this application provide a segmentation method for an image segmentation model, comprising:

[0038] Obtain the image of the coronary artery to be segmented;

[0039] The coronary artery image is input into the trained image segmentation model described above for image segmentation to obtain the image segmentation result of the coronary artery image.

[0040] Thirdly, embodiments of this application provide a training system for an image segmentation model, comprising:

[0041] The first processing unit is used to acquire training images of the coronary arteries;

[0042] The second processing unit is used to perform bi-branch feature extraction on the training image to obtain a first target feature and a second target feature. The feature size of the first target feature and the feature size of the second target feature are the same. The first target feature records local detail information of the training image, and the second target feature records global context information of the training image.

[0043] The third processing unit is used to perform branch feature fusion processing on the first target feature and the second target feature to obtain target fusion features;

[0044] The fourth processing unit is used to update the parameters of the initialized image segmentation model according to the target fusion features to obtain the trained image segmentation model.

[0045] Fourthly, embodiments of this application also provide an electronic device, including:

[0046] At least one processor;

[0047] At least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0049] Fifthly, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by the processor, is used to implement the above-described method.

[0050] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application:

[0051] This application discloses a training method, segmentation method, and system for an image segmentation model. The training method acquires training images of coronary arteries; it performs bi-branch feature extraction on the training images using a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network. The network types of the first and second branch networks are different. The first branch network is used to extract local feature information from the training images, while the second branch network is used to extract global feature information. The first and second target features are then fused using branch features to obtain a target fusion feature. Based on the target fusion feature, the parameters of the initialized image segmentation model are updated to obtain a trained image segmentation model. This training method, by performing bi-branch feature extraction on the training images, extracts a first target feature containing local detail information and a second target feature containing global context information through a bi-branch network structure. This effectively improves the model's ability to capture global information from coronary artery images, thus improving the segmentation effect of coronary artery image segmentation. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions in this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0053] Figure 1 A flowchart illustrating a training method for an image segmentation model provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the overall network structure of an image segmentation model provided in an embodiment of this application;

[0055] Figure 3 A schematic diagram of the network structure of an FFM module provided in an embodiment of this application;

[0056] Figure 4 A schematic diagram of the network structure of a BFM module provided in an embodiment of this application;

[0057] Figure 5 A schematic diagram of the structural framework of a training system for an image segmentation model provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0059] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit the scope of this application.

[0061] Currently, existing techniques for automatic segmentation of coronary artery images are typically based on convolutional neural networks (CNNs). However, due to the relatively fixed kernel size of CNNs, their receptive field is limited. This means the network model cannot effectively capture global information in the coronary artery image, often resulting in discontinuous segments and unsatisfactory segmentation outcomes.

[0062] In addition, some existing technologies capture global image information of the input image directly using the Transformer network. However, since the Transformer network cannot capture local details well, it is easy to have inaccurate segmentation of the extracted coronary arteries in small vessel regions and introduce artifact noise, resulting in unsatisfactory segmentation results.

[0063] Furthermore, a very small number of existing technologies use the TransUNet dual-branch network coupled with a CNN-ViT encoder to automatically segment coronary artery images. However, since the TransUNet dual-branch network simply integrates and superimposes the CNN features extracted by the CNN branch network and the Transformer features extracted by the Transformer network, it cannot fully utilize the fusion effect of local and global features, resulting in unsatisfactory segmentation results for coronary artery images.

[0064] In view of this, embodiments of the present invention provide a training method, a segmentation method, and a system for an image segmentation model. The training method extracts features from the training image in a two-branch manner. Specifically, it extracts a first target feature containing local detail information through local feature extraction and a second target feature containing global context information through global feature extraction. This can effectively improve the model's ability to capture global information of coronary artery images, which is beneficial to improving the segmentation effect of coronary artery image segmentation.

[0065] Furthermore, this method also fuses the first and second branch features at the same feature level, which can break the field-of-view limitations between different branch networks. By deeply interacting and fusing at different feature levels, it enriches the semantic information content of the first and second branch features, effectively integrating the feature information of different branch networks at different feature levels. In addition, this method also fuses the branch features between the first and second target features, which can effectively solve the semantic differences between different branch networks and effectively fit the semantic gap between different branch networks. This makes the obtained target fused features beneficial to improving the segmentation effect of coronary artery image segmentation.

[0066] Reference Figure 1 and Figure 2 In this embodiment of the application, a method for training an image segmentation model includes:

[0067] Step 110: Obtain training images of the coronary arteries;

[0068] In this application embodiment, the training images may be angiographic images of the coronary artery portion of the subject acquired through CCTA technology after obtaining the subject's permission; or, they may be obtained through a set of coronary artery images publicly available on the Internet, such as ImageCAS, a dataset for coronary artery imaging research publicly available on the Internet.

[0069] Step 120: Perform bi-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network. The network types of the first branch network and the second branch network are different. The first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image.

[0070] In this embodiment, training images can be input into a dual-branch network to extract first target features and second target features from the training images. The dual-branch network includes a first branch network and a second branch network. Specifically, the first branch network can be a U-Net branch based on a convolutional neural network (CNN), which is used to extract local detail information from the training images. Specifically, the second branch network can be a U-Net branch based on a visual neural network (Vision Transformer, ViT), which is used to extract global context information from the training images.

[0071] In some embodiments, the step of performing bi-branch feature extraction on the training image through a first branch network and a second branch network to obtain first target features and second target features includes:

[0072] The first branch network is used to extract local features from the training image to obtain several first branch features with different feature levels.

[0073] Furthermore, the local feature extraction of the training image yields several first-branch features at different feature levels, including:

[0074] Obtain input features, wherein the input features are image features of the training image or the first branch features of the previous feature level;

[0075] The input features are subjected to feature convolution to extract intermediate features;

[0076] The intermediate features are downsampled to obtain the first branch features of the current feature level.

[0077] In this embodiment, the image features of the training image can be input into the first branch network. The image features of the training image are extracted by several local feature extraction modules cascaded in the first branch network, thereby obtaining the first branch features output by each local feature extraction module. Each local feature extraction module corresponds to a feature level.

[0078] Continue to refer to Figure 2 In this embodiment, taking a cascaded local feature extraction module of the first branch network as an example with three modules, each module includes a convolutional layer and a downsampling layer. The convolutional layer consists of two 3×3×3 convolutional operations, while the downsampling layer is a 2×2×2 max pooling operation. This downsampling layer halves the image resolution of the input intermediate features while doubling the number of feature channels. Specifically, in this embodiment, the first branch feature output by the first local feature extraction module is the branch feature of the first feature level, and its feature size can be... The first branch feature output by the second local feature extraction module is the branch feature of the second feature level, and its feature size can be... The first branch feature output by the third local feature extraction module is the branch feature of the third feature level, and its feature size can be... Where H, W, and D are the height, width, and depth of the input features, respectively, and C is the number of feature channels of the image features of the training image in the first branch network.

[0079] Understandably, if the input feature is the image feature of the training image, it can be input into the first local feature extraction module. The convolutional layer of this module performs a convolution operation on the input feature, resulting in the convolutional input feature, which is denoted as the intermediate feature. Then, the downsampling layer of the first local feature extraction module downsamples the intermediate feature to obtain the first branch feature of the first feature level. Alternatively, if the input feature is the first branch feature of a previous feature level, for example, the first branch feature of a second feature level, it can be input into the third local feature extraction module to obtain the first branch feature of the third feature level.

[0080] The training image is subjected to global feature extraction through the second branch network to obtain several second branch features with different feature levels.

[0081] Furthermore, the global feature extraction of the training image yields several second-branch features at different feature levels, including:

[0082] The training image is subjected to feature convolution block processing to obtain the second branch features of the first feature level;

[0083] Multi-layer global attention extraction is performed on the second branch features of the first feature level to obtain several fifth branch features, which are second branch features other than the second branch features of the first feature level.

[0084] In this embodiment, the image features of the training image can be input into the second branch network. The image features of the training image are extracted globally by several global feature extraction modules cascaded in the second branch network, thereby obtaining the second branch features output by each global feature extraction module. The number of global feature extraction modules in the second branch network is the same as the number of local feature extraction modules in the first branch network, and each global feature extraction module corresponds to a feature level.

[0085] Continue to refer to Figure 2In this embodiment of the application, the number of local feature extraction modules in the first branch network cascade is 3, and the corresponding number of global feature extraction modules in the cascade is also 3. At this time, the first global feature extraction module includes a residual convolutional layer and a patch partitioning layer, while the second global feature extraction module consists of a Swin-Transformer layer and a patch merging layer. The third global feature extraction module is similar to the aforementioned second global feature extraction module, and can be easily deduced by analogy.

[0086] Specifically, the image features of the training image can be input into the first global feature extraction module for convolution block processing. This involves performing residual convolution operations on the input image features using a residual convolution layer, and then dividing the residual convolutional image features into blocks using a patch partitioning layer. This yields the second branch features of the first feature level, with a spatial dimension half that of the input image features. The feature size of this second branch feature of the first feature level can be...

[0087] After obtaining the second branch features of the first feature level, these features can be input into the second global feature extraction module for the first layer of global attention extraction, thus obtaining the second branch features of the second feature level. The feature size of this second branch feature can be... Furthermore, after obtaining the second branch features of the second feature level, these features can be input into the third global feature extraction module for second-layer global attention extraction, thereby obtaining the second branch features of the third feature level, whose feature size can be... Among them, C ′ The number of feature channels of the training image features in the second branch network, C in this embodiment. ′ It can be the same as C.

[0088] Understandably, through Figure 2 The examples provided are for illustrative purposes only and are not intended to limit this application. For example, the number of feature layers for feature extraction in the first branch network and the second branch network can be any of 4, 5, 7, etc. Specifically, if the number of feature layers is 5, the first branch network can include 5 cascaded local feature extraction modules, while the second branch network can include 5 cascaded global feature extraction modules. This application will not elaborate further here.

[0089] For each third branch feature and its corresponding fourth branch feature, inter-layer feature fusion is performed to obtain an inter-layer fused feature corresponding to each feature level. The third branch feature is any first branch feature other than the first branch feature of the last feature level, and the fourth branch feature is a second branch feature with the same feature level as the third branch feature.

[0090] Further, the step of performing inter-layer feature fusion on each third branch feature and the corresponding fourth branch feature to obtain inter-layer fused features corresponding to each feature level includes:

[0091] The third branch features and the fourth branch features are concatenated to obtain inter-layer concatenated features;

[0092] Channel attention is extracted from the interlayer stitching features to obtain a channel attention map;

[0093] Based on the channel attention map, channel attention fusion is performed on the inter-layer splicing features to obtain inter-layer channel features;

[0094] The interlayer fusion feature is obtained based on the interlayer channel feature, the third branch feature, and the fourth branch feature.

[0095] In this embodiment, the third branch feature is used as the first branch feature of the first feature level, and the corresponding fourth branch feature can be the second branch feature of the first feature level. Hierarchical feature fusion can be implemented using a feature fusion module (FFM) based on a channel attention mechanism. Specifically, the FFM module concatenates the first branch feature and the second branch feature of the first feature level through a concat operation to obtain the inter-layer concatenated features of the first and second branch networks at the first feature level. Then, a compact information representation of the inter-layer concatenated features in the channel dimension is learned through a global average pooling layer and a sigmoid activation function, and a channel attention map is generated. Next, element-wise multiplication is performed on the inter-layer concatenated features and the channel attention map, and inter-layer channel features are obtained by assigning different weights to different channels.

[0096] Understandably, in the first feasible implementation, the number of channels in the inter-layer channel features can be halved by using a convolutional layer with a kernel size of 3×3×3, and the inter-layer channel features with the halved number of channels can be concatenated with the third and fourth branch features of the first feature level to obtain the inter-layer fusion features of the first feature level shared by the first branch network and the second branch network.

[0097] Or, refer to Figure 3In a second feasible implementation, the inter-layer channel features with half the number of channels can be concatenated with the third branch features to obtain the inter-layer fusion features of the first branch network at the first feature level; and the inter-layer channel features with half the number of channels can be concatenated with the fourth branch features to obtain the inter-layer fusion features of the second branch network at the second feature level. The inter-layer fusion features for other feature levels can be derived similarly, for example, the inter-layer fusion features of the second feature level can be derived based on the first and second branch features of the second feature level, which will not be elaborated further in this application.

[0098] For example, in the second feasible implementation, the inter-layer fusion characteristics of the first branch network and the inter-layer fusion characteristics of the second branch network can be expressed as follows:

[0099]

[0100] in, The inter-layer fusion feature of the first branch network at the i-th feature level; represents the first branch feature of the first branch network at the i-th feature level; W(·) is the convolution operation function symbol; This is the channel attention map for the i-th feature level; For the inter-layer splicing features of the i-th feature level; The inter-layer fusion feature of the second branch network at the i-th feature level; The second branch feature of the second branch network at the i-th feature level.

[0101] Based on the first branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the first target feature.

[0102] Based on the second branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the second target feature.

[0103] In this embodiment, multi-level cross-layer fusion of all inter-layer fusion features can be performed based on feature levels, using the first branch features of the last feature level and the training image. Specifically, this embodiment takes an example with 3 cascaded local and global feature extraction modules. In this case, there are 2 inter-layer fusion features, which are the inter-layer fusion features of the first feature level and the inter-layer fusion features of the second feature level, respectively. At this time, a deconvolution operation can be performed on the first branch features of the last feature level to double the resolution of the first branch features of the last feature level. Then, the inter-layer fusion features of the second feature level after residual convolution processing are concatenated with the first branch features of the last feature level after deconvolution to obtain the cross-layer fusion features of the second feature level.

[0104] Understandably, after obtaining the cross-layer fusion features of the second feature level, the inter-layer fusion features of the first feature level after residual convolution processing can be concatenated with the cross-layer fusion features of the second feature level after deconvolution to obtain the cross-layer fusion features of the first feature level. After obtaining the cross-layer fusion features of the first feature level, the image features of the training image after residual convolution processing can be concatenated with the cross-layer fusion features of the first feature level after deconvolution to obtain the first target feature.

[0105] It should be noted that the content of the second target feature is similar to that of the first target feature mentioned above, and can be easily deduced by analogy. In addition, in practical applications, the number of cascaded local and global feature extraction modules can be any of 5, 7, 8, etc., which can be easily deduced based on the above content. The difference lies in the number of repetitions of cross-layer feature concatenation, which will not be elaborated here.

[0106] Step 130: Perform branch feature fusion processing on the first target feature and the second target feature to obtain the target fusion feature;

[0107] In this embodiment of the application, the first target feature and the second target feature can be input into a branch fusion module (BFM) based on spatial attention mechanism for branch feature fusion processing. The BFM module is used to make up for the semantic gap between the first target feature of the first branch network and the second target feature of the second branch network, so as to obtain the target fusion feature.

[0108] In some embodiments, the branch feature fusion processing of the first target feature and the second target feature to obtain the target fused feature includes:

[0109] The first target feature and the second target feature are concatenated to obtain the branch concatenated feature;

[0110] The branch splicing features are subjected to bi-branch parallel clustering processing to obtain a first clustering feature map and a second clustering feature map;

[0111] Spatial attention is extracted from the first cluster feature map and the second cluster feature map to obtain a spatial attention map;

[0112] Based on the spatial attention map, spatial attention fusion is performed on the first target feature and the second target feature to obtain the target fusion feature.

[0113] In the embodiments of this application, reference is made to Figure 4 The first and second target features can be concatenated using the BFM module to obtain branch concatenated features. Then, Global Max Pooling (GMP) and Global Average Pooling (GAP) operations are performed on the branch concatenated features to obtain a first cluster feature map representing the salient feature distribution of the branch concatenated features across channels, and a second cluster feature map representing the average feature distribution of the branch concatenated features across channels. Next, spatial attention representations of the first and second cluster feature maps are learned through a 3×3×3 convolutional layer and a sigmoid activation function layer to generate a spatial attention map. Finally, element-wise multiplication is performed on the first and second target features based on the spatial attention map to suppress irrelevant information in key spatial regions. The element-wise multiplied first and second target features are then concatenated to obtain the target fusion feature.

[0114] Step 140: Update the parameters of the initialized image segmentation model according to the target fusion features to obtain the trained image segmentation model.

[0115] In this embodiment of the application, after obtaining the target fusion features, the image segmentation result of the training image can be obtained by thresholding the target fusion features. Then, the segmentation accuracy of the image segmentation model can be evaluated based on the coronary artery label of the training image and the image segmentation result, thereby updating the parameters of the model.

[0116] Specifically, the accuracy of image segmentation results can be measured by a loss function. Many types of loss functions are commonly used, such as 0-1 loss, squared loss, absolute loss, logarithmic loss, and cross-entropy loss, which will not be elaborated upon here. For a given training round, the target loss value for the training data can be determined using the label of a single training image, the model's segmentation prediction result for that image, and the loss function. Based on the target loss value, the backpropagation algorithm is used to update the model's parameters. After several iterations, a well-trained image segmentation model can be obtained. The specific number of iterations can be preset, or training can be considered complete when the required accuracy is achieved on the test set.

[0117] It is worth mentioning that, according to experimental data, when the image segmentation model is a TransUNet dual-branch network coupled with a CNN-ViT encoder, the trained image segmentation model has a DSC value of 0.81321, a JC value of 0.59381, and an HD value of 10.55173. Here, DSC is the Dice similarity coefficient, which measures the degree of overlap between the segmented region and the ground truth label; JC is the Jaccard coefficient, which measures the degree of region overlap; and HD is the Hausdorff Distance, which measures the maximum mismatch between segmentation boundaries. A larger HD value indicates a larger maximum error at the boundary, resulting in poorer model performance and segmentation effect. In contrast, the image segmentation model trained in this application has a DSC value of 0.83092, a JC value of 0.71194, and an HD value of 7.46226, demonstrating superior segmentation performance for coronary artery images.

[0118] In this embodiment of the application, a segmentation method for an image segmentation model includes:

[0119] Step 210: Obtain the image of the coronary artery to be segmented;

[0120] Step 220: Input the coronary artery image into the trained image segmentation model described above for image segmentation to obtain the image segmentation result of the coronary artery image.

[0121] In this embodiment of the application, in practical applications, a coronary artery image to be segmented provided by the user can be obtained, and the coronary artery image can be input into a trained image segmentation model. The trained image segmentation model can then be used to segment the coronary arteries in the coronary artery image, thereby obtaining the image segmentation result of the coronary artery image.

[0122] The following describes in detail, with reference to the accompanying drawings, a training system for an image segmentation model according to an embodiment of this application.

[0123] Reference Figure 5 The image segmentation model training system proposed in this application includes:

[0124] The first processing unit 101 is used to acquire training images of the coronary arteries;

[0125] The second processing unit 102 is used to perform bi-branch feature extraction on the training image to obtain a first target feature and a second target feature. The feature size of the first target feature and the feature size of the second target feature are the same. The first target feature records local detail information of the training image, and the second target feature records global context information of the training image.

[0126] The third processing unit 103 is used to perform branch feature fusion processing on the first target feature and the second target feature to obtain target fusion features;

[0127] The fourth processing unit 104 is used to update the parameters of the initialized image segmentation model according to the target fusion features to obtain a trained image segmentation model.

[0128] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0129] Reference Figure 6 This application also provides an electronic device, including:

[0130] At least one processor 201;

[0131] At least one memory 202 is used to store at least one program;

[0132] When the at least one program is executed by the at least one processor 201, the at least one processor 201 implements the method embodiment described above.

[0133] Similarly, it can be understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] This application also provides a computer-readable storage medium storing a program executable by a processor 201, which, when executed by the processor 201, is used to implement the above-described method embodiments.

[0135] Similarly, the content of the above method embodiments is applicable to the present computer-readable storage medium embodiments. The specific functions implemented by the present computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0137] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0138] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0140] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0141] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0142] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0143] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0144] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A training method for an image segmentation model, characterized in that, include: Obtain training images of the coronary arteries; The training image is subjected to bi-branch feature extraction through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network. The network types of the first branch network and the second branch network are different. The first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image. The first target feature and the second target feature are subjected to branch feature fusion processing to obtain the target fusion feature; Based on the target fusion features, the parameters of the initialized image segmentation model are updated to obtain the trained image segmentation model; The step of performing dual-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network includes: The first branch network is used to extract local features from the training image to obtain several first branch features with different feature levels. The training image is subjected to global feature extraction through the second branch network to obtain several second branch features with different feature levels. For each third branch feature and its corresponding fourth branch feature, inter-layer feature fusion is performed to obtain an inter-layer fused feature corresponding to each feature level. The third branch feature is any first branch feature other than the first branch feature of the last feature level, and the fourth branch feature is a second branch feature with the same feature level as the third branch feature. Based on the first branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the first target feature. Based on the second branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the second target feature.

2. The method according to claim 1, characterized in that, The step of extracting local features from the training image yields several first-branch features at different feature levels, including: Obtain input features, wherein the input features are image features of the training image or the first branch features of the previous feature level; The input features are subjected to feature convolution to extract intermediate features; The intermediate features are downsampled to obtain the first branch features of the current feature level.

3. The method according to claim 1, wherein the step of performing global feature extraction on the training image to obtain several second-branch features with different feature levels includes: The training image is subjected to feature convolution block processing to obtain the second branch features of the first feature level; Multi-layer global attention extraction is performed on the second branch features of the first feature level to obtain several fifth branch features, which are second branch features other than the second branch features of the first feature level.

4. The method according to claim 1, characterized in that, The step of fusing inter-layer features for each third branch feature and its corresponding fourth branch feature to obtain inter-layer fused features corresponding to each feature level includes: The third branch features and the fourth branch features are concatenated to obtain inter-layer concatenated features; Channel attention is extracted from the interlayer stitching features to obtain a channel attention map; Based on the channel attention map, channel attention fusion is performed on the inter-layer splicing features to obtain inter-layer channel features; The interlayer fusion feature is obtained based on the interlayer channel feature, the third branch feature, and the fourth branch feature.

5. The method according to any one of claims 1-4, characterized in that, The step of performing branch feature fusion processing on the first target feature and the second target feature to obtain target fusion features includes: The first target feature and the second target feature are concatenated to obtain the branch concatenated feature; The branch splicing features are subjected to bi-branch parallel clustering processing to obtain a first clustering feature map and a second clustering feature map; Spatial attention is extracted from the first cluster feature map and the second cluster feature map to obtain a spatial attention map; Based on the spatial attention map, spatial attention fusion is performed on the first target feature and the second target feature to obtain the target fusion feature.

6. A segmentation method for an image segmentation model, characterized in that, include: Obtain the image of the coronary artery to be segmented; The coronary artery image is input into the trained image segmentation model as described in any one of claims 1-5 to perform image segmentation, thereby obtaining the image segmentation result of the coronary artery image.

7. A training system for an image segmentation model, characterized in that, include: The first processing unit is used to acquire training images of the coronary arteries; The second processing unit is used to perform bi-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network. The network types of the first branch network and the second branch network are different. The first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image. The third processing unit is used to perform branch feature fusion processing on the first target feature and the second target feature to obtain target fusion features; The fourth processing unit is used to update the parameters of the initialized image segmentation model according to the target fusion features to obtain the trained image segmentation model; The step of performing dual-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network includes: The first branch network is used to extract local features from the training image to obtain several first branch features with different feature levels. The training image is subjected to global feature extraction through the second branch network to obtain several second branch features with different feature levels. For each third branch feature and its corresponding fourth branch feature, inter-layer feature fusion is performed to obtain an inter-layer fused feature corresponding to each feature level. The third branch feature is any first branch feature other than the first branch feature of the last feature level, and the fourth branch feature is a second branch feature with the same feature level as the third branch feature. Based on the first branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the first target feature. Based on the second branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the second target feature.

8. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to implement the method as described in any one of claims 1-6.

Images