Training method, segmentation method and system of image segmentation model

Abstract

The invention discloses a training method, a segmentation method and a system for an image segmentation model, and the method comprises the steps: obtaining a training image of a coronary artery; performing double-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network, the network type of the first branch network being different from the network type of the second branch network; performing branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature; and according to the target fusion feature, performing parameter updating on the initialized image segmentation model to obtain a trained image segmentation model. The training method can effectively improve the ability of the model to capture global information of the coronary artery image, and is beneficial for improving the segmentation effect of coronary artery image segmentation. The invention relates to the technical field of image processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image segmentation model training method, a segmentation method and a system. Background Art

[0002] With the development of medical technology, automatic segmentation of coronary artery images acquired by computed tomography angiography (CCTA) technology has gradually become one of the key contents of relevant practitioners.

[0003] Currently, existing technologies for automatic segmentation of coronary artery images are typically based on convolutional neural networks (CNNs). However, due to the relatively fixed size of the convolution kernel in CNNs, the receptive field of CNNs is limited. Consequently, these models cannot effectively capture the global information in coronary artery images, resulting in discontinuities in the extracted coronary arteries and unsatisfactory segmentation results.

[0004] Therefore, the problems existing in the existing technology still need to be solved and optimized. Summary of the Invention

[0005] The purpose of the present invention is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, an object of an embodiment of the present invention is to provide a training method, a segmentation method, and a system for an image segmentation model, wherein the training method can effectively improve the model's ability to capture global information of coronary artery images, which is conducive to improving the segmentation effect of coronary artery image segmentation.

[0007] In order to achieve the above technical objectives, the technical solutions adopted in the embodiments of the present application include:

[0008] In a first aspect, an embodiment of the present application provides a method for training an image segmentation model, comprising:

[0009] Obtain training images of coronary arteries;

[0010] Performing dual-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network, wherein the network type of the first branch network is different from the network type of the second branch network, the first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image;

[0011] Performing branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature;

[0012] According to the target fusion features, the parameters of the initialized image segmentation model are updated to obtain a trained image segmentation model.

[0013] In addition, the method according to the above embodiment of the present application may also have the following additional technical features:

[0014] Furthermore, in one embodiment of the present application, performing dual-branch feature extraction on the training image by using the first branch network and the second branch network to obtain the first target feature and the second target feature includes:

[0015] Performing local feature extraction on the training image through the first branch network to obtain a plurality of first branch features at different feature levels;

[0016] Performing global feature extraction on the training image through the second branch network to obtain a plurality of second branch features at different feature levels;

[0017] Perform inter-layer feature fusion on each third branch feature and the corresponding fourth branch feature to obtain an inter-layer fusion feature corresponding to each feature level, where the third branch feature is any first branch feature except the first branch feature of the last feature level, and the fourth branch feature is a second branch feature having the same feature level as the third branch feature;

[0018] Performing multi-level cross-layer fusion processing on all the inter-layer fusion features according to the first branch feature of the last feature level and the training image to obtain the first target feature;

[0019] According to the second branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the second target feature.

[0020] Furthermore, in one embodiment of the present application, the local feature extraction is performed on the training image to obtain a plurality of first branch features with different feature levels, including:

[0021] Obtaining an input feature, where the input feature is an image feature of the training image or a first branch feature of a previous feature level;

[0022] Performing first branch feature extraction on the input feature to obtain intermediate features;

[0023] The intermediate features are downsampled to obtain the first branch features of the current feature level.

[0024] Furthermore, in one embodiment of the present application, the global feature extraction is performed on the training image to obtain a plurality of second branch features at different feature levels, including:

[0025] Performing feature convolution block processing on the training image to obtain a second branch feature of the first feature level;

[0026] Multi-layer global attention extraction is performed on the second branch features of the first feature level to obtain a plurality of fifth branch features, where the fifth branch features are second branch features other than the second branch features of the first feature level.

[0027] Furthermore, in one embodiment of the present application, performing inter-layer feature fusion on each third branch feature and the corresponding fourth branch feature to obtain an inter-layer fusion feature corresponding to each feature level includes:

[0028] Performing feature splicing on the third branch feature and the fourth branch feature to obtain an inter-layer splicing feature;

[0029] Performing channel attention extraction on the inter-layer splicing features to obtain a channel attention map;

[0030] According to the channel attention map, channel attention fusion is performed on the inter-layer splicing features to obtain inter-layer channel features;

[0031] The inter-layer fusion feature is obtained according to the inter-layer channel feature, the third branch feature and the fourth branch feature.

[0032] Furthermore, in one embodiment of the present application, performing branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature includes:

[0033] Performing feature splicing on the first target feature and the second target feature to obtain a branch splicing feature;

[0034] Performing a dual-branch parallel clustering process on the branch splicing features to obtain a first cluster feature map and a second cluster feature map;

[0035] Performing spatial attention extraction on the first cluster feature map and the second cluster feature map to obtain a spatial attention map;

[0036] According to the spatial attention map, spatial attention fusion is performed on the first target feature and the second target feature to obtain the target fusion feature.

[0037] In a second aspect, an embodiment of the present application provides a segmentation method of an image segmentation model, comprising:

[0038] Acquire a coronary artery image to be segmented;

[0039] The coronary artery image is input into the trained image segmentation model to perform image segmentation, thereby obtaining an image segmentation result of the coronary artery image.

[0040] In a third aspect, an embodiment of the present application provides a training system for an image segmentation model, comprising:

[0041] a first processing unit, configured to acquire a training image of the coronary artery;

[0042] a second processing unit, configured to perform dual-branch feature extraction on the training image to obtain a first target feature and a second target feature, wherein the feature size of the first target feature is the same as the feature size of the second target feature, the first target feature records local detail information of the training image, and the second target feature records global context information of the training image;

[0043] a third processing unit, configured to perform branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature;

[0044] The fourth processing unit is used to update the parameters of the initialized image segmentation model according to the target fusion feature to obtain a trained image segmentation model.

[0045] In a fourth aspect, an embodiment of the present application further provides an electronic device, including:

[0046] at least one processor;

[0047] at least one memory for storing at least one program;

[0048] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0049] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor, and the program executable by the processor is used to implement the above method when executed by the processor.

[0050] The advantages and benefits of this application will be partially given in the following description, and partially become apparent from the following description, or learned through practice of this application:

[0051] The embodiments of the present application disclose a training method, a segmentation method, and a system for an image segmentation model, wherein the training method obtains a training image of a coronary artery; performs dual-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network, wherein the network type of the first branch network is different from the network type of the second branch network, the first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image; performs branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature; and updates the parameters of the initialized image segmentation model based on the target fusion feature to obtain a trained image segmentation model. The training method performs dual-branch feature extraction on the training image, and extracts a first target feature that records local detail information and a second target feature that records global context information through a dual-branch network structure. This can effectively improve the model's ability to capture global information of the coronary artery image, which is beneficial to improving the segmentation effect of the coronary artery image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly expressing some embodiments of the technical solutions of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 A flowchart of a method for training an image segmentation model provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of the overall network structure of an image segmentation model provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of the network structure of an FFM module provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of the network structure of a BFM module provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of the structural framework of an image segmentation model training system provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0061] Currently, existing technologies for automatic segmentation of coronary artery images are typically based on convolutional neural networks (CNNs). However, due to the relatively fixed size of the convolution kernel in CNNs, the receptive field of CNNs is limited. Consequently, these models cannot effectively capture the global information in coronary artery images, resulting in discontinuities in the extracted coronary arteries and unsatisfactory segmentation results.

[0062] In addition, some existing technologies directly use the Transformer network to capture the global image information of the input image. However, since the Transformer network cannot capture local details well, the extracted coronary arteries are prone to inaccurate segmentation in small blood vessel areas and the introduction of artifact noise, resulting in unsatisfactory segmentation results.

[0063] In addition, there are very few existing technologies that automatically segment coronary artery images through the TransUNet dual-branch network coupled with the CNN-ViT encoder. However, since the TransUNet dual-branch network simply integrates and superimposes the CNN features extracted by the CNN branch network and the Transformer features extracted by the Transformer network, it cannot fully utilize the fusion effect of local features and global features, resulting in unsatisfactory segmentation results for coronary artery images.

[0064] In view of this, an embodiment of the present invention provides a training method, a segmentation method and a system for an image segmentation model, wherein the training method performs dual-branch feature extraction on the training image, specifically extracting a first target feature that records local detail information through local feature extraction, and extracting a second target feature that records global context information through global feature extraction. This can effectively improve the model's ability to capture the global information of coronary artery images, which is conducive to improving the segmentation effect of coronary artery image segmentation.

[0065] In addition, this method also breaks the field of view limitation between different branch networks by fusing the first branch features and the second branch features with the same feature level, enriches the semantic information content of the first branch features and the second branch features through deep interactive fusion at different feature levels, and effectively integrates the feature information of different branch networks at different feature levels; moreover, this method also effectively solves the semantic differences between different branch networks by fusing the branch features between the first target feature and the second target feature, effectively fits the semantic gap between different branch networks, so that the obtained target fusion features can be beneficial to improving the segmentation effect of coronary artery image segmentation.

[0066] Reference Figure 1 and Figure 2 In an embodiment of the present application, a method for training an image segmentation model includes:

[0067] Step 110: Acquire a training image of the coronary artery;

[0068] In an embodiment of the present application, the training image can be obtained by obtaining the subject's permission through CCTA technology to obtain angiographic images of the subject's coronary artery portion; or, it can be obtained from a set of coronary artery images publicly available on the Internet, such as the ImageCAS dataset for coronary artery imaging research publicly available on the Internet.

[0069] Step 120: Perform dual-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network. The network type of the first branch network is different from the network type of the second branch network. The first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image.

[0070] In an embodiment of the present application, a training image can be input into a dual-branch network, and the first target feature and the second target feature of the training image can be extracted through the dual-branch network; wherein the dual-branch network includes a first branch network and a second branch network, and the first branch network can specifically be a U-Net branch based on a convolutional neural network (CNN), and the first branch network is used to extract local detail information of the training image; the second branch network can specifically be a U-Net branch based on a visual neural network (Vision Transformer, ViT), and the second branch network is used to extract global context information in the training image.

[0071] In some embodiments, performing dual-branch feature extraction on the training image by using the first branch network and the second branch network to obtain the first target feature and the second target feature includes:

[0072] Performing local feature extraction on the training image through the first branch network to obtain a plurality of first branch features at different feature levels;

[0073] Furthermore, the local feature extraction is performed on the training image to obtain a plurality of first branch features with different feature levels, including:

[0074] Obtaining an input feature, where the input feature is an image feature of the training image or a first branch feature of a previous feature level;

[0075] Performing feature convolution extraction on the input features to obtain intermediate features;

[0076] The intermediate features are downsampled to obtain the first branch features of the current feature level.

[0077] In an embodiment of the present application, the image features of the training image can be input into the first branch network, and local feature extraction is performed on the image features of the training image through several local feature extraction modules cascaded through the first branch network, so as to obtain the first branch features output by each local feature extraction module, and each local feature extraction module corresponds to a feature level.

[0078] Continue to refer to Figure 2 In the embodiment of the present application, the number of local feature extraction modules of the first branch network cascade is 3 as an example. At this time, each local feature extraction module includes a convolution layer and a downsampling layer. The convolution layer is composed of two 3×3×3 convolution operations, and the downsampling layer is a maximum pooling operation of size 2×2×2. The downsampling layer is used to halve the image resolution of the input intermediate feature and double the number of feature channels. Specifically, in the embodiment of the present application, the first branch feature output by the first local feature extraction module is the branch feature of the first feature level, and its feature size can be The first branch feature output by the second local feature extraction module is the branch feature of the second feature level, and its feature size can be The first branch feature output by the third local feature extraction module is the branch feature of the third feature level, and its feature size can be Among them, H, W, and D are the height, width, and depth of the input features, respectively, and C is the number of feature channels of the image features of the training image in the first branch network.

[0079] It is understandable that if the input feature is an image feature of a training image, the input feature can be input into the first local feature extraction module, and the input feature is convolved through the convolution layer of the first local feature extraction module to obtain the convolved input feature, which is recorded as the intermediate feature; then, the intermediate feature is downsampled through the downsampling layer of the first local feature extraction module to obtain the first branch feature of the first feature level. Alternatively, if the input feature is the first branch feature of the previous feature level, for example, the input feature is the first branch feature of the second feature level, then the input feature can be input into the third local feature extraction module to obtain the first branch feature of the third feature level.

[0080] Performing global feature extraction on the training image through the second branch network to obtain a plurality of second branch features at different feature levels;

[0081] Furthermore, the global feature extraction is performed on the training image to obtain a plurality of second branch features with different feature levels, including:

[0082] Performing feature convolution block processing on the training image to obtain a second branch feature of the first feature level;

[0083] Multi-layer global attention extraction is performed on the second branch features of the first feature level to obtain a plurality of fifth branch features, where the fifth branch features are second branch features other than the second branch features of the first feature level.

[0084] In an embodiment of the present application, the image features of the training image can be input into the second branch network, and global feature extraction is performed on the image features of the training image through several global feature extraction modules cascaded in the second branch network to obtain the second branch features output by each global feature extraction module. The number of global feature extraction modules of the second branch network is the same as the number of local feature extraction modules of the first branch network, and each global feature extraction module corresponds to a feature level.

[0085] Continue to refer to Figure 2In the embodiment of the present application, the number of local feature extraction modules in the cascade of the first branch network is 3, and the corresponding number of global feature extraction modules in the cascade is also 3 as an example. At this time, the first global feature extraction module includes a residual convolution layer and a patch partitioning layer, and the second global feature extraction module is composed of a Swin-Transformer layer and a patch merging layer. The third global feature extraction module is similar to the aforementioned second global feature extraction module and can be simply inferred.

[0086] Specifically, the image features of the training image can be input into the first global feature extraction module for convolution block processing. Specifically, the residual convolution operation is performed on the input image features through the residual convolution layer, and the image features after residual convolution are divided into blocks through the patch partitioning layer, so as to obtain a second branch feature with a spatial dimension of half of the input image features and the first feature level. The feature size of the second branch feature of the first feature level can be

[0087] After obtaining the second branch feature of the first feature level, the second branch feature of the first feature level can be input into the second global feature extraction module for first-layer global attention extraction to obtain the second branch feature of the second feature level, whose feature size can be In addition, after obtaining the second branch feature of the second feature level, the second branch feature of the second feature level can be input into the third global feature extraction module for the second layer global attention extraction, so as to obtain the second branch feature of the third feature level, and its feature size can be Among them, C ′ The image features of the training image have the following feature channels in the second branch network: C in the embodiment of the present application ′ Can be the same as C.

[0088] It is understandable that through Figure 2 The examples provided are for illustration only and do not limit the present application. For example, the number of feature layers for feature extraction by the first branch network and the second branch network can also be any one of 4, 5, 7, etc. Specifically, if the number of feature layers is 5, the first branch network can include 5 cascaded local feature extraction modules, and the second branch network can include 5 cascaded global feature extraction modules. This application will not go into details here.

[0089] Perform inter-layer feature fusion on each third branch feature and the corresponding fourth branch feature to obtain an inter-layer fusion feature corresponding to each feature level, where the third branch feature is any first branch feature except the first branch feature of the last feature level, and the fourth branch feature is a second branch feature having the same feature level as the third branch feature;

[0090] Furthermore, performing inter-layer feature fusion on each third branch feature and the corresponding fourth branch feature to obtain an inter-layer fusion feature corresponding to each feature level includes:

[0091] Performing feature splicing on the third branch feature and the fourth branch feature to obtain an inter-layer splicing feature;

[0092] Performing channel attention extraction on the inter-layer splicing features to obtain a channel attention map;

[0093] According to the channel attention map, channel attention fusion is performed on the inter-layer splicing features to obtain inter-layer channel features;

[0094] The inter-layer fusion feature is obtained according to the inter-layer channel feature, the third branch feature and the fourth branch feature.

[0095] In an embodiment of the present application, the embodiment of the present application takes the third branch feature as the first branch feature of the first feature level, and the corresponding fourth branch feature can be the second branch feature of the first feature level as an example. Hierarchical feature fusion can be implemented by a hierarchical feature fusion module (FFM) based on a channel attention mechanism. The FFM module specifically concatenates the first branch feature and the second branch feature of the first feature level through a concatenation operation to obtain inter-layer concatenation features of the first branch network and the second branch network at the first feature level; then, a global average pooling layer and a Sigmoid activation function are used to learn the compact information representation of the inter-layer concatenation features in the channel dimension and generate a channel attention map; then, the inter-layer concatenation features and the channel attention map are element-wise multiplied to obtain inter-layer channel features by assigning different weights to different channels.

[0096] It can be understood that in the first feasible implementation, the number of channels of the inter-layer channel features can be halved through a convolution layer with a convolution kernel size of 3×3×3, and the inter-layer channel features after the number of channels is halved are spliced with the third branch features and the fourth branch features of the first feature level, so as to obtain the inter-layer fusion features of the first feature level shared by the first branch network and the second branch network.

[0097] Or, refer to Figure 3In a second feasible implementation, the inter-layer channel features after the number of channels is halved can be spliced with the third branch features to obtain the inter-layer fusion features of the first branch network at the first feature level; and the inter-layer channel features after the number of channels is halved can be spliced with the fourth branch features to obtain the inter-layer fusion features of the second branch network at the second feature level. The inter-layer fusion features of the other feature levels can be derived similarly, for example, the inter-layer fusion features of the second feature level can be derived based on the first branch features and the second branch features of the second feature level, which will not be elaborated in this application.

[0098] For example, for the second feasible implementation, the inter-layer fusion features of the first branch network and the inter-layer fusion features of the second branch network can be expressed as:

[0099]

[0100] in, is the inter-layer fusion feature of the first branch network at the i-th feature level; is the first branch feature of the first branch network at the i-th feature level; W(·) is the symbol of the convolution operation function; is the channel attention map of the i-th feature level; is the inter-layer splicing feature of the i-th feature level; is the inter-layer fusion feature of the second branch network at the i-th feature level; is the second branch feature of the second branch network at the i-th feature level.

[0101] Performing multi-level cross-layer fusion processing on all the inter-layer fusion features according to the first branch feature of the last feature level and the training image to obtain the first target feature;

[0102] According to the second branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the second target feature.

[0103] In an embodiment of the present application, all the inter-layer fusion features can be multi-level cross-layer fused based on the feature level through the first branch features of the last feature level and the training image. Specifically, the embodiment of the present application takes the number of cascaded local and global feature extraction modules as 3 as an example. At this time, the number of inter-layer fusion features is 2, which are the inter-layer fusion features of the first feature level and the inter-layer fusion features of the second feature level. At this time, the first branch features of the last feature level can be deconvolved to increase the resolution of the first branch features of the last feature level by two times. Then, the inter-layer fusion features of the second feature level after residual convolution processing and the first branch features of the last feature level after deconvolution are feature spliced to obtain cross-layer fusion features at the second feature level.

[0104] It can be understood that after obtaining the cross-layer fusion features of the second feature level, the inter-layer fusion features of the first feature level after residual convolution processing and the cross-layer fusion features of the second feature level after deconvolution can be feature spliced to obtain the cross-layer fusion features at the first feature level; after obtaining the cross-layer fusion features of the first feature level, the image features of the training image after residual convolution processing and the cross-layer fusion features of the first feature level after deconvolution can be feature spliced to obtain the first target features.

[0105] It should be noted that the content of the second target feature is similar to that of the first target feature and can be simply deduced by analogy. In addition, in actual applications, the number of cascaded local and global feature extraction modules can be any one of 5, 7, 8, etc., which can be simply deduced based on the above content. The difference lies in the number of repetitions of cross-layer feature splicing, which will not be further described in this application.

[0106] Step 130: performing branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature;

[0107] In an embodiment of the present application, the first target feature and the second target feature can be input into a branch fusion module (Branch Fusion Module, BFM) based on a spatial attention mechanism for branch feature fusion processing. The BFM module is used to make up for the semantic gap between the first target feature of the first branch network and the second target feature of the second branch network, so as to obtain the target fusion feature.

[0108] In some embodiments, performing branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature includes:

[0109] Performing feature splicing on the first target feature and the second target feature to obtain a branch splicing feature;

[0110] Performing a dual-branch parallel clustering process on the branch splicing features to obtain a first cluster feature map and a second cluster feature map;

[0111] Performing spatial attention extraction on the first cluster feature map and the second cluster feature map to obtain a spatial attention map;

[0112] According to the spatial attention map, spatial attention fusion is performed on the first target feature and the second target feature to obtain the target fusion feature.

[0113] In the examples of this application, refer to Figure 4 , the first target feature and the second target feature can be concatenated (Concat) by the BFM module to obtain the branch splicing feature; then the branch splicing feature is subjected to the global maximum pooling (GMP) operation and the global average pooling (GAP) operation respectively, so as to obtain the first cluster feature map representing the significant feature distribution of the branch splicing feature on each channel, and the second cluster feature map representing the average feature distribution of the branch splicing feature on each channel; then, the spatial attention representation of the first cluster feature map and the second cluster feature map is learned through a 3×3×3 convolutional layer and a Sigmoid activation function layer to generate a spatial attention map; finally, based on the spatial attention map, the first target feature and the second target feature are respectively element-wise multiplied to suppress the irrelevant information of the first target feature or the second target feature in the key spatial area, and the first target feature and the second target feature after the element-wise multiplication are spliced to obtain the target fusion feature.

[0114] Step 140: Update the parameters of the initialized image segmentation model according to the target fusion features to obtain a trained image segmentation model.

[0115] In an embodiment of the present application, after obtaining the target fusion feature, the image segmentation result of the training image can be obtained by performing threshold processing on the target fusion feature, and then the segmentation accuracy of the image segmentation model is evaluated based on the coronary artery label and image segmentation result of the training image, so as to update the parameters of the model.

[0116] Specifically, the accuracy of the image segmentation results can be measured by a loss function. There are many types of commonly used loss functions, such as the 0-1 loss function, the square loss function, the absolute loss function, the logarithmic loss function, the cross-entropy loss function, etc., which can all be used as loss functions for image segmentation models. They will not be elaborated on here. For a certain round of training, the target loss value of the training data can be determined by the label of a single training image and the segmentation prediction result of the training image by the model, as well as the loss function. Based on the target loss value, the backpropagation algorithm is used to update the parameters of the model. After several rounds of iteration, a trained image segmentation model can be obtained. The specific number of iterations can be set in advance, or the training is considered complete when the test set meets the accuracy requirements.

[0117] It is worth mentioning that according to experimental data, when the image segmentation model is a TransUNet dual-branch network coupled with a CNN-ViT encoder, the DSC value of the trained image segmentation model is 0.81321, the JC value is 0.59381, and the HD value is 10.55173. Among them, the DSC value is the Dice similarity coefficient, which is used to measure the degree of regional overlap between the segmentation result and the true label; the JC value is the Jaccard coefficient, which is used to measure the degree of regional overlap; the HD value is the Hausdorff Distance distance, which is used to measure the maximum mismatch between segmentation boundaries. The larger the HD value, the greater the maximum error on the boundary, the worse the model performance, and the worse the segmentation effect. The DSC value of the image segmentation model trained in this application is 0.83092, the JC value is 0.71194, and the HD value is 7.46226, and the segmentation effect of coronary artery images is better.

[0118] In an embodiment of the present application, a segmentation method of an image segmentation model includes:

[0119] Step 210: Acquire a coronary artery image to be segmented;

[0120] Step 220: Input the coronary artery image into the trained image segmentation model to perform image segmentation, and obtain an image segmentation result of the coronary artery image.

[0121] In an embodiment of the present application, in actual application, a coronary artery image to be segmented provided by a user can be obtained, and the coronary artery image can be input into a trained image segmentation model. The coronary arteries in the coronary artery image can be segmented by the trained image segmentation model to obtain an image segmentation result of the coronary artery image.

[0122] The following describes in detail a training system for an image segmentation model proposed according to an embodiment of the present application with reference to the accompanying drawings.

[0123] Reference Figure 5 , a training system for an image segmentation model proposed in an embodiment of the present application includes:

[0124] A first processing unit 101 is configured to obtain a training image of the coronary arteries;

[0125] a second processing unit 102 configured to perform dual-branch feature extraction on the training image to obtain a first target feature and a second target feature, wherein the feature size of the first target feature is the same as the feature size of the second target feature, the first target feature records local detail information of the training image, and the second target feature records global context information of the training image;

[0126] The third processing unit 103 is configured to perform branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature;

[0127] The fourth processing unit 104 is configured to update parameters of the initialized image segmentation model according to the target fusion features to obtain a trained image segmentation model.

[0128] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0129] Reference Figure 6 , an embodiment of the present application further provides an electronic device, including:

[0130] at least one processor 201;

[0131] At least one memory 202, configured to store at least one program;

[0132] When the at least one program is executed by the at least one processor 201 , the at least one processor 201 implements the above method embodiment.

[0133] Similarly, it can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0134] An embodiment of the present application further provides a computer-readable storage medium, in which a program executable by the processor 201 is stored. The program executable by the processor 201 is used to implement the above-mentioned method embodiment when executed by the processor 201.

[0135] Similarly, the contents of the above method embodiments are applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] In some optional embodiments, the functions / operations mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, the two boxes shown in succession may actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flow chart of the present application are provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logic flows presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0137] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present application. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present application as set forth in the claims using ordinary techniques without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0138] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0140] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0141] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0142] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0143] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0144] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A training method for an image segmentation model, characterized in that: include: Obtain training images of coronary arteries; Performing dual-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network, wherein the network type of the first branch network is different from the network type of the second branch network, the first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image; Performing branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature; According to the target fusion features, the parameters of the initialized image segmentation model are updated to obtain a trained image segmentation model.

2. The method according to claim 1, characterized in that The performing dual-branch feature extraction on the training image by using the first branch network and the second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network includes: Performing local feature extraction on the training image through the first branch network to obtain a plurality of first branch features at different feature levels; Performing global feature extraction on the training image through the second branch network to obtain a plurality of second branch features at different feature levels; Perform inter-layer feature fusion on each third branch feature and the corresponding fourth branch feature to obtain an inter-layer fusion feature corresponding to each feature level, where the third branch feature is any first branch feature except the first branch feature of the last feature level, and the fourth branch feature is a second branch feature having the same feature level as the third branch feature; Performing multi-level cross-layer fusion processing on all the inter-layer fusion features according to the first branch feature of the last feature level and the training image to obtain the first target feature; According to the second branch features of the last feature level and the training image, multi-level cross-layer fusion processing is performed on all the inter-layer fusion features to obtain the second target feature.

3. The method according to claim 2, characterized in that The local feature extraction is performed on the training image to obtain a plurality of first branch features with different feature levels, including: Obtaining an input feature, where the input feature is an image feature of the training image or a first branch feature of a previous feature level; Performing feature convolution extraction on the input features to obtain intermediate features; The intermediate features are downsampled to obtain the first branch features of the current feature level.

4. The method according to claim 2, wherein the global feature extraction is performed on the training image to obtain a plurality of second branch features at different feature levels, comprising: Performing feature convolution block processing on the training image to obtain a second branch feature of the first feature level; Multi-layer global attention extraction is performed on the second branch features of the first feature level to obtain a plurality of fifth branch features, where the fifth branch features are second branch features other than the second branch features of the first feature level.

5. The method according to claim 2, characterized in that The inter-layer feature fusion of each third branch feature and the corresponding fourth branch feature to obtain an inter-layer fusion feature corresponding to each feature level includes: Performing feature splicing on the third branch feature and the fourth branch feature to obtain an inter-layer splicing feature; Performing channel attention extraction on the inter-layer splicing features to obtain a channel attention map; According to the channel attention map, channel attention fusion is performed on the inter-layer splicing features to obtain inter-layer channel features; The inter-layer fusion feature is obtained according to the inter-layer channel feature, the third branch feature and the fourth branch feature.

6. The method according to any one of claims 1 to 5, characterized in that The performing branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature includes: Performing feature splicing on the first target feature and the second target feature to obtain a branch splicing feature; Performing a dual-branch parallel clustering process on the branch splicing features to obtain a first cluster feature map and a second cluster feature map; Performing spatial attention extraction on the first cluster feature map and the second cluster feature map to obtain a spatial attention map; According to the spatial attention map, spatial attention fusion is performed on the first target feature and the second target feature to obtain the target fusion feature.

7. A segmentation method for an image segmentation model, characterized in that: include: Acquire a coronary artery image to be segmented; The coronary artery image is input into the trained image segmentation model according to any one of claims 1 to 6 to perform image segmentation, thereby obtaining an image segmentation result of the coronary artery image.

8. A training system for an image segmentation model, characterized in that: include: a first processing unit, configured to acquire a training image of the coronary arteries; a second processing unit, configured to perform dual-branch feature extraction on the training image through a first branch network and a second branch network to obtain a first target feature output by the first branch network and a second target feature output by the second branch network, wherein the network type of the first branch network is different from the network type of the second branch network, the first branch network is used to extract local feature information of the training image, and the second branch network is used to extract global feature information of the training image; a third processing unit, configured to perform branch feature fusion processing on the first target feature and the second target feature to obtain a target fusion feature; The fourth processing unit is used to update the parameters of the initialized image segmentation model according to the target fusion feature to obtain a trained image segmentation model.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 7 when executed by the processor.

Images