Coronary artery vessel tree segmentation method and device, electronic equipment and storage medium

Abstract

This application relates to the field of medical image processing technology, providing a method, apparatus, electronic device, and storage medium for segmenting coronary artery tree. The method includes: acquiring a sequence of coronary angiography images; performing cross-frame compression on each frame of the coronary angiography image sequence to obtain a corresponding compressed image sequence, achieving the fusion of channel features and temporal features of each frame; extracting multi-dimensional interactive features from the compressed image sequence, including spatial dimension interactive features, channel-height interactive features, and channel-width interactive features; mapping the multi-dimensional interactive features to the segmentation output space to obtain mapped features; and upsampling the mapped features to obtain a binary mask of the coronary artery tree. This invention improves the accuracy and completeness of coronary artery tree binary mask segmentation by fusing multi-dimensional image features and optimizing the segmentation process, achieving precise segmentation of main branches and sub-branches.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method, apparatus, electronic device and storage medium for segmenting coronary artery vascular trees. Background Technology

[0002] Coronary angiography is a widely used routine examination method in the diagnosis of coronary heart disease. Due to its significant advantages of being easy to operate and widely available, it has been widely promoted in clinical diagnosis and treatment.

[0003] With the development of medical technology, computational physiology technology based on coronary angiography has made some progress, enabling non-invasive assessment of some functional parameters and providing some support for the functional diagnosis of coronary heart disease.

[0004] However, when segmenting the binary mask of the coronary artery tree from coronary artery images, only the main branches of the coronary arteries are considered, while the branches are not, resulting in an incomplete binary mask. Summary of the Invention

[0005] In view of this, this application provides a method, apparatus, electronic device and storage medium for segmenting coronary artery vascular tree, which can segment not only the main branches of the coronary artery but also the branches of the coronary artery when segmenting the binary mask of the coronary artery vascular tree from the coronary artery image.

[0006] This application provides a method for segmenting the coronary artery tree through several embodiments. The following description covers various aspects of this application, and the embodiments and beneficial effects described herein can be referenced interchangeably.

[0007] In a first aspect, this application provides a method for segmenting a coronary artery tree, comprising:

[0008] Obtain coronary angiography image sequences;

[0009] Based on cross-frame compression of each frame in the coronary angiography image sequence, a compressed image sequence corresponding to each frame is obtained, so that the channel features and temporal features of each frame are fused.

[0010] Feature extraction is performed based on compressed image sequences to obtain multi-dimensional interactive features, which include spatial dimension interactive features, channel and height interactive features, and channel and width interactive features.

[0011] Multi-dimensional interactive features are mapped onto the segmented output space to obtain the mapped features;

[0012] Upsampling of the mapped features yields a binary mask of the coronary artery tree in the image, enabling coronary artery tree segmentation.

[0013] According to the embodiments of this application, the above-described technical solution of this application has at least the following beneficial effects:

[0014] The coronary artery tree segmentation method of this invention effectively solves the technical problem of inaccurate segmentation caused by existing binary masks that only consider the main branches and ignore the branches. This method achieves comprehensive capture and refined representation of the main branches and branches of the coronary artery tree by performing cross-frame compression and fusion of channel and temporal features on the coronary angiography image sequence and extracting multi-dimensional interactive features. Accurate segmentation is then achieved through feature mapping and upsampling. This improves the segmentation accuracy and spatial structural integrity of the coronary artery tree binary mask, achieving complete and accurate identification of both main branches and branches. The segmented binary mask contains both main branches and branches.

[0015] In one possible implementation of the first aspect above, a compressed image sequence corresponding to each frame of the coronary angiography image sequence is obtained by cross-frame compression based on each frame of the image sequence, including:

[0016] Based on each frame of the image, the coronary angiography image sequence is cropped to obtain time segments;

[0017] Compression across frames is performed based on time segments to obtain a compressed image sequence.

[0018] According to the implementation method of this application, cross-frame compression based on the selected temporal segments can make the fusion of image channel features and temporal features more targeted, accurate and efficient, improve the quality of feature fusion, lay a better temporal feature foundation for the accurate extraction of subsequent multi-dimensional interactive features, and thus ensure the accuracy of subsequent coronary artery tree binary mask segmentation.

[0019] In one possible implementation of the first aspect above, cross-frame compression is performed based on temporal segments to obtain a compressed image sequence, including:

[0020] High-dimensional features are obtained by extracting features based on time-series segments.

[0021] Feature extraction is performed based on high-dimensional features to obtain high-level spatial semantic features;

[0022] Compressed image sequences are obtained by performing nonparametric tensor transformation based on high-level spatial semantic features.

[0023] According to the implementation method of this application, problems such as parameter redundancy and overfitting caused by parametric transformation are avoided. The efficiency and stability of cross-frame compression are improved, and the channel and temporal features of coronary angiography images can be better preserved and fused during the compression process, accurately preserving the key feature information of the main branches and branches of the coronary vascular tree.

[0024] In one possible implementation of the first aspect above, feature extraction is performed based on the compressed image sequence to obtain multi-dimensional interactive features, including:

[0025] By using compressed image sequences as input data for the spatial interaction model, spatial dimension interaction features are obtained.

[0026] By using compressed image sequences as input data for a channel-high interaction model, channel-high interaction features are obtained.

[0027] By using compressed image sequences as input data for the channel-width interaction model, channel-width interaction features are obtained.

[0028] According to the implementation method of this application, the single feature extraction and fusion mode is abandoned, and the feature interaction and correlation information under different dimensions can be specifically mined to achieve accurate, independent and comprehensive capture of interactive features in each dimension, and fully preserve the feature details of the main branches and branches of the coronary artery tree under different dimensions.

[0029] In one possible implementation of the first aspect above, mapping multi-dimensional interaction features to a segmented output space to obtain mapped features further includes:

[0030] By fusing spatial dimension interaction features, channel-height interaction features, and channel-width interaction features, a fused feature is obtained.

[0031] The fused features are mapped onto the segmented output space to obtain the mapped features.

[0032] According to the implementation method of this application, by first fusing the spatial dimension, channel and height, and channel and width interaction features to obtain fused features, and then mapping the fused features to the segmented output space, the organic integration of interaction features of different dimensions is realized, effectively avoiding the limitations of single-dimensional feature mapping.

[0033] In one possible implementation of the first aspect above, the multi-dimensional interaction features are mapped to the segmented output space to obtain the mapped features, including:

[0034] Decoupling and reconstruction are performed based on multi-dimensional interaction features to obtain multiple decoupling features;

[0035] Based on multiple decoupling features, mapping features are obtained.

[0036] According to the implementation method of this application, multiple parallel mapping models are used to decouple and reconstruct the fused features. Each branch independently learns different feature mapping relationships at the same spatial scale, thereby enhancing the response to small blood vessels, main blood vessels, and complex bifurcation regions respectively. Through the fusion of the outputs of parallel branches, the model can maintain spatial consistency while taking into account the segmentation needs of blood vessel structures of different shapes and scales, and effectively improve the integrity and robustness of the segmentation results.

[0037] One possible implementation of the first aspect mentioned above also includes:

[0038] The vascular classification is performed on the binary mask to obtain the aortic region and branch artery region in the binary mask.

[0039] According to the embodiments of this application, the aortic region and branch artery region are accurately divided by performing vascular classification on the binary mask of the segmented coronary artery tree.

[0040] Secondly, this application provides a coronary artery tree segmentation device, comprising:

[0041] The acquisition module is used to acquire coronary angiography image sequences;

[0042] The compression module is used to compress each frame of the coronary angiography image sequence across frames to obtain a compressed image sequence corresponding to each frame, so that the channel features and temporal features of each frame are fused.

[0043] The feature extraction module is used to extract features based on compressed image sequences to obtain multi-dimensional interactive features, including spatial dimension interactive features, channel and height interactive features, and channel and width interactive features.

[0044] The mapping module is used to map multi-dimensional interactive features to the segmented output space to obtain mapped features;

[0045] The upsampling module is used to upsample the mapped features to obtain a binary mask of the coronary artery tree in the image, so as to achieve coronary artery tree segmentation.

[0046] Thirdly, this application provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the coronary artery tree segmentation method disclosed in the first aspect and any possible implementation thereof.

[0047] Fourthly, this application provides a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the coronary artery tree segmentation method disclosed in the first aspect and any possible implementation thereof.

[0048] Fifthly, this application provides a computer program product comprising: computer instructions that, when executed on an electronic device, cause the electronic device to perform the coronary artery tree segmentation method disclosed in the first aspect and any possible implementation thereof.

[0049] The beneficial effects of the second to fifth aspects can be found in the first aspect and the beneficial effects of any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0050] Figure 1 This is a flowchart of coronary artery tree segmentation in an embodiment of this application;

[0051] Figure 2 This is a diagram of the coronary artery vascular tree segmentation model in the embodiments of this application;

[0052] Figure 3 This is a comparison image of coronary angiography and binary mask in the embodiments of this application;

[0053] Figure 4 This is a flowchart illustrating the process of obtaining a compressed image sequence in an embodiment of this application.

[0054] Figure 5 This is a flowchart illustrating the cross-frame compression process in an embodiment of this application;

[0055] Figure 6 This is a flowchart of the multi-dimensional interaction feature extraction steps in the embodiments of this application;

[0056] Figure 7 This is a flowchart illustrating the multi-dimensional interactive feature mapping process in this application embodiment;

[0057] Figure 8 This is a vascular tree grading diagram in the embodiments of this application;

[0058] Figure 9 This is a block diagram of the electronic device in the embodiments of this application;

[0059] Figure 10 This is a block diagram of a system-on-chip (SoC) in the embodiments of this application.

[0060] Marker explanation:

[0061] A. Coronary angiography image; B. Binary mask; 1. Main branch; 2. Branch. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] The technical problems to be solved by the embodiments of this application will be described below.

[0064] As mentioned earlier, in some embodiments, when segmenting the vascular tree binary mask from the coronary artery image, only the main branches of the coronary artery are considered, while the branches are not included in the consideration. As a result, the segmented vascular tree binary mask is incomplete for the entire coronary artery, leading to inaccurate analysis results of the coronary artery based on the binary mask.

[0065] Therefore, to address the aforementioned problems, this application provides a method for segmenting coronary artery tree. The method for segmenting coronary artery tree in this application first acquires a sequence of coronary angiography images. Each frame in the sequence is then compressed across frames to fuse channel and temporal features, resulting in a corresponding compressed image sequence. Next, multi-dimensional interactive features, including spatial dimension features, channel-height interaction features, and channel-width interaction features, are extracted from the compressed image sequence. These multi-dimensional interactive features are then mapped onto the segmentation output space to obtain mapped features. Finally, the mapped features are upsampled to obtain a binary mask for the coronary artery tree.

[0066] The segmentation method of this application effectively solves the problem of inaccurate binary mask segmentation caused by the prior art only considering the main branches of the coronary artery and ignoring the branches. Through multi-step feature processing and transformation, it achieves complete and accurate identification of the main branches and branches of the coronary artery tree, which significantly improves the segmentation accuracy and structural integrity of the binary mask.

[0067] The methods and apparatus of the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0068] Reference Figure 1 , Figure 2 and Figure 3 , Figure 1 This is a flowchart of coronary artery tree segmentation in an embodiment of this application. Figure 2 This is a diagram of the coronary artery tree segmentation model in the embodiments of this application. Figure 3This is a comparison image of the coronary angiography and the binary mask in the embodiments of this application.

[0069] like Figure 1 As shown, the method for segmenting the coronary artery tree in this embodiment includes steps S100-S500.

[0070] S100 is used to acquire coronary angiography image sequences.

[0071] S200, based on cross-frame compression of each frame in the coronary angiography image sequence, obtains a compressed image sequence corresponding to each frame.

[0072] It should be noted that cross-frame compression is performed on each frame of the coronary angiography image sequence. The purpose is to fuse the temporal and channel features of each frame, thereby reducing the number of feature inputs in subsequent feature extraction steps. The aforementioned channel features may include red-green-blue (RGB) color channel features.

[0073] It is understood that the method in this application embodiment performs cross-frame compression on each frame of the coronary angiography image sequence to fuse the channel features and temporal features of the image, fully explores and utilizes the temporal correlation information and channel feature information of the angiography image sequence, breaks through the limitations of image feature extraction, realizes the comprehensive capture of the features of the coronary artery vascular tree (including main branches and branches), builds a complete feature foundation for subsequent accurate segmentation, and avoids the problem of missed identification of branch vessels due to feature loss.

[0074] S300 extracts features from compressed image sequences to obtain multi-dimensional interactive features.

[0075] It should be noted that the above-mentioned multi-dimensional interaction features include spatial dimension interaction features, channel and height interaction features, and channel and width interaction features.

[0076] Understandably, this method extracts multi-dimensional interactive features, including spatial dimension interaction features, channel and height interaction features, and channel and width interaction features. It provides a comprehensive and refined representation of the features of the coronary artery vascular tree from multiple dimensions, overcoming the technical deficiency of existing technologies that struggle to identify branch vessels through single-dimensional feature extraction. It can accurately distinguish and identify the features of branch vessels other than the main coronary artery branches, solving the core problem of existing technologies ignoring branches from the feature extraction level, and achieving complete feature recognition of the main branches and branches of the coronary artery vascular tree.

[0077] S400 maps multi-dimensional interactive features to the segmented output space to obtain mapped features.

[0078] Understandably, this method maps multi-dimensional interactive features to the segmentation output space to obtain mapped features. The core is to accurately transform abstract multi-dimensional features into a feature space adapted to the coronary artery tree segmentation task. This is a crucial intermediate step connecting multi-dimensional interactive feature extraction and subsequent upsampling to generate a binary mask. This operation allows the abstract features, which originally contained information about spatial dimensions, channels and heights, and channels and widths, to match their dimensions and spatial distribution with the segmentation output image space. This ensures that the feature representation accurately corresponds to the vascular regions (including main branches and branches) in the coronary angiography image. Simultaneously, it further integrates the effective information of multi-dimensional interactive features, eliminates redundant features irrelevant to segmentation, and makes the features more closely match the actual spatial structure of the coronary artery tree. This provides a more adaptable and accurate feature foundation for subsequent upsampling steps, ensuring that a coronary artery tree binary mask with complete spatial structure and accurate region identification can be generated after upsampling.

[0079] S500 upsamples the mapped features to obtain a binary mask of the coronary artery tree in the image.

[0080] like Figure 2 As shown, step S500 corresponds to step e, where the mapped features are used as input data to the upsampling model to obtain the aforementioned binary mask. The upsampling model includes two convolutional layers with 3×3 kernels and one activation layer using the rectified linear unit (ReLU) function. After feature extraction by one of the convolutional layers (Conv3×3 in the figure), the output feature data is used as input to the activation layer (ReLU in the figure) to obtain activation data. This activation data is then used as input to the final convolutional layer (Conv3×3 in the figure) for further feature extraction. Finally, the aforementioned binary mask is obtained based on the extracted feature data and the sigmoid function.

[0081] Understandably, this step improves the spatial resolution of the mapping features, restoring the low-resolution abstract mapping features that are adapted to the segmentation output space to a size and pixel distribution that matches the original coronary angiography image. At the same time, during the resolution restoration process, the key spatial structure and feature information of the main branches and branches of the coronary arteries captured in the previous multi-dimensional interactive features are fully preserved. Finally, through pixel-level binary segmentation, the coronary artery vessel regions (including main branches and branches) in the image are clearly distinguished from the background region, generating a coronary tree binary mask with complete structure and accurate region division.

[0082] like Figure 3As shown, the comparison diagram of coronary angiography image and binary mask includes coronary angiography image A and binary mask B. After the coronary artery tree segmentation is completed in the above steps, the coronary angiography image A is used to obtain binary mask B, which includes both the main branch 1 of the coronary artery and the branch 2 of the coronary artery.

[0083] The steps S200-S500 described above are explained in further detail below.

[0084] refer to Figure 4 , Figure 4 A flowchart illustrating the acquisition of compressed image sequences in an embodiment of this application is shown.

[0085] like Figure 4 As shown, in this embodiment of the application, each frame of the coronary angiography image sequence is compressed across frames to obtain a compressed image sequence corresponding to each frame, including steps S210-S220.

[0086] S210: Based on each frame of the image, the coronary angiography image sequence is truncated to obtain a time segment.

[0087] In some embodiments, based on a preset number of samples, image frames of equal number to the number of samples are extracted from the coronary angiography image sequence to construct the aforementioned time segment, with each image frame as a reference.

[0088] For example, if the preset sample size is 5, the first two frames and the last two frames of a certain frame are extracted to form a time segment with a sample size of 5.

[0089] refer to Figure 2 If the total length of the coronary angiography image sequence is T, then there are T time segments, each with a width of w, a height of h, and a sample size of b.

[0090] Since each time series segment has width, height, and number of samples, as well as channel and time features, a time series segment containing five-dimensional features including number of samples, channel, time, height, and width (batch_size, channel, time_clips, height, width, i.e., B, C, T, H, W) can be constructed.

[0091] S220, performs cross-frame compression based on time segments to obtain a compressed image sequence.

[0092] It should be noted that since only time-series segments of four-dimensional features can be processed in subsequent feature extraction, it is necessary to compress the time-series segments from five-dimensional features to four-dimensional features.

[0093] Understandably, by first extracting temporal segments from the coronary angiography image sequence frame by frame, and then performing cross-frame compression based on these temporal segments, we can focus on the effective temporal correlation information of the coronary angiography image sequence, reduce redundant information interference, and make cross-frame compression more targeted. This also improves the efficiency and effectiveness of cross-frame compression, and more accurately achieves the fusion of channel and temporal features of each frame. Fully leveraging the temporal feature value of the angiography image sequence lays a more solid feature foundation for the accurate extraction of subsequent multi-dimensional interactive features and the complete identification of the main branches and branches of the coronary artery tree, further ensuring the accuracy of subsequent binary mask segmentation of the coronary artery tree.

[0094] refer to Figure 5 , Figure 5 A flowchart illustrating the cross-frame compression process in an embodiment of this application is shown.

[0095] like Figure 5 As shown, in some embodiments, the above-mentioned cross-frame compression based on time segments to obtain a compressed image sequence includes:

[0096] S221, feature extraction is performed based on time-series segments to obtain high-dimensional features.

[0097] refer to Figure 2 Step S221 above corresponds to step b1. First, the temporal features are normalized using a layer (i.e., BN in the figure) to obtain normalized data. Then, based on this normalized data, a convolutional mapping is performed using a convolutional layer containing 3×3 kernels (i.e., Conv3×3 in the figure) to enhance the local spatial structure representation of the normalized data, resulting in enhanced data. Finally, based on this enhanced data, a nonlinear mapping is performed using an activation layer (i.e., ReLU in the figure) to improve the expressive power of the enhanced data, resulting in the aforementioned high-dimensional features.

[0098] It should be noted that the activation layer described above uses the ReLU activation function, but other activation functions can also be used, which will not be elaborated here. Normalization is performed on the temporal segment features to eliminate dimensional and distributional biases, ensuring the stability of subsequent processing. A 3×3 convolutional kernel is used to enhance the local spatial structure representation of the features, and then an activation layer constructed with the ReLU activation function is used to achieve nonlinear mapping, overcoming the limitations of linear representation and improving feature representation capabilities. The resulting high-dimensional features encode dynamic information across time frames in the channels, achieving a fusion representation of temporal dynamic information and spatial structure information, laying a solid feature foundation containing multi-dimensional effective information for subsequent high-level feature extraction.

[0099] It is understandable that the channel features of the aforementioned high-dimensional features encode dynamic information across time frames.

[0100] S222, based on high-dimensional features, performs feature extraction to obtain high-level spatial semantic features.

[0101] refer to Figure 2 Step S222 above corresponds to step b2. First, based on the high-dimensional feature, normalization is performed through a layer (i.e., BN in the figure) to obtain new normalized data. Then, based on the normalized data, convolutional mapping is performed through a convolutional layer containing 3×3 convolutional kernels (i.e., Conv3×3 in the figure) to obtain shared spatial semantic features. Finally, based on the shared semantic features, nonlinear mapping is performed through an activation layer (i.e., ReLU in the figure) to obtain the aforementioned high-level spatial semantic features.

[0102] Understandably, normalizing high-dimensional features again further calibrates the data distribution, eliminates feature biases from previous processing, and ensures the accuracy of spatial semantic feature extraction. Shared spatial semantic features are refined and extracted through 3×3 convolutional mapping, effectively stripping away redundant non-semantic information and focusing on core spatial semantics. Then, nonlinear mapping in activation layers deepens the semantic feature expression, achieving the transformation from fused high-dimensional features to specific high-level spatial semantic features. This significantly improves the semantic interpretability and recognizability of features for the target scene, making the features more aligned with the core requirements of subsequent tensor transformations and image compression, and enhancing the overall process's relevance.

[0103] S223, based on high-level spatial semantic features, performs non-parametric tensor transformation to obtain a compressed image sequence.

[0104] refer to Figure 2 Step S223 above corresponds to step b3. The above-mentioned high-level semantic features are subjected to non-parametric tensor transformation through a convolutional layer containing 3×3 convolutional kernels (i.e., Conv3×3 in the figure) to obtain the above-mentioned compressed image sequence.

[0105] Understandably, non-parametric tensor transformations based on high-level spatial semantic features, using 3×3 convolution kernels to perform the operation, ensure that core semantic information is not lost during image compression, thus achieving accurate compression under semantic awareness.

[0106] First, high-dimensional features are extracted from time-series segments to capture their basic feature information. Then, high-level spatial semantic features are further extracted to deeply mine the core spatial semantic association features between the main branches and branches of the coronary artery tree, and invalid feature interference is eliminated.

[0107] Finally, the compressed image sequence is obtained through non-parametric tensor transformation of high-level semantic features, which avoids problems such as parameter redundancy and overfitting caused by parametric transformation. This improves the efficiency and stability of cross-frame compression, and can better preserve and fuse the channel and temporal features of coronary angiography images during the compression process, accurately preserving the key feature information of the main branches and branches of the coronary vascular tree.

[0108] This allows the obtained compressed image sequence to better meet the needs of subsequent feature extraction, laying a solid foundation for the accurate extraction of multi-dimensional interactive features and further improving the accuracy and completeness of subsequent coronary artery tree binary mask segmentation.

[0109] refer to Figure 6 , Figure 6 A flowchart of the multi-dimensional interactive feature extraction steps in an embodiment of this application is shown.

[0110] like Figure 6 As shown, in some embodiments, feature extraction is performed based on compressed image sequences to obtain multi-dimensional interactive features, including steps S310-S330.

[0111] S310 uses the compressed image sequence as input data for the spatial interaction model to obtain spatial dimension interaction features.

[0112] refer to Figure 2 Step S310 above corresponds to step c1. The spatial interaction model includes a pooling layer (i.e., Z-Pool in the figure), a convolutional layer (i.e., Conv7×7 in the figure), and a normalization layer (i.e., BN in the figure). The compressed image sequence with feature scale (B, C×T, H, W) is pooled by the pooling layer of the spatial interaction model to obtain pooled features, with dimensions of (B, 2, H, W). Based on the pooled features, spatial dependency features are extracted through a convolutional layer containing 7×7 kernels in the spatial interaction model, with dimensions of (B, 1, H, W). Subsequently, based on the spatial dependency features, normalization is performed through the normalization layer of the spatial interaction model to obtain spatial features, with dimensions of (B, 1, H, W). Finally, the spatial features are fused with the compressed image sequence to obtain spatial dimension interaction features, with dimensions of (B, C×T, H, W).

[0113] S320 uses the compressed image sequence as input data for the channel and high-level interaction model to obtain the channel and high-level interaction features.

[0114] refer to Figure 2 Step S320 above corresponds to step c2. The channel-height interaction model has the same structure as the spatial interaction model described above. However, when extracting features from the channel-height interaction features, the channel-height interaction model first needs to permutate the height features and channel features of the compressed image sequence to obtain a permutation sequence. This permutation sequence is then used as input data for the channel-height interaction model to obtain the aforementioned channel-height interaction features. The feature scale of the permutation sequence at this point is (B, H, C×T, W). Other steps are detailed above and will not be repeated here.

[0115] S330 uses the compressed image sequence as input data for the channel-width interaction model to obtain the channel-width interaction features.

[0116] refer to Figure 2 Step S330 above corresponds to step c3. The channel-width interaction model has the same structure as the spatial interaction model. Similarly, when the channel-width interaction model extracts features from the channel-width interaction features, it first needs to permutate the width features and channel features of the compressed image sequence to obtain a permuted sequence. The feature scale of this permuted sequence is (B, W, H, C×T). Other steps can be referred to above and will not be repeated here.

[0117] It should be noted that the aforementioned spatial dimension interaction features are used to represent the spatial saliency features of compressed image sequences after cross-channel temporal semantic aggregation, and are used to characterize which locations in two-dimensional space are more likely to correspond to stable and continuous vascular structures under the combined effect of semantics across all time frames and channels. The channel-height interaction features are used to represent the structural consistency features coupled between the channel and height directions, and are used to characterize the stable extension relationship of blood vessels across time frames in the vertical spatial direction. The channel-width interaction features are used to represent the structural consistency features coupled between the channel and width directions, and are used to characterize the trans-temporal continuity and local morphological changes of blood vessels in the horizontal spatial direction.

[0118] Understandably, by inputting compressed image sequences into three dedicated models—spatial interaction, channel and height interaction, and channel and width interaction—independently obtaining interaction features for each corresponding dimension, each model can focus on the deep and precise mining of feature interaction relationships in a single dimension. This effectively avoids the problems of insufficient mining and inter-dimensional feature interference caused by a single model simultaneously processing multiple dimensions of features, and can more fully capture the feature details of the main branches and side branches of the coronary artery tree in each dimension. This makes the representation of interaction features in each dimension purer and more accurate, and the combined multi-dimensional interaction features can more comprehensively and finely characterize the overall features of the coronary artery tree, laying a high-quality feature foundation for subsequent feature mapping and accurate segmentation using binary masks. This further improves the accuracy and completeness of identifying the main branches and side branches of the coronary artery tree.

[0119] refer to Figure 7 , Figure 7 A flowchart illustrating the multi-dimensional interactive feature mapping process in an embodiment of this application is shown.

[0120] like Figure 7 As shown, the multi-dimensional interactive features are mapped to the segmented output space to obtain the mapped features, including steps S410-S420.

[0121] S410, based on multi-dimensional interaction features, performs decoupling reconstruction to obtain multiple decoupling features.

[0122] refer to Figure 2 The above steps correspond to step d, where the multi-dimensional interaction features are first used as input data for four identical mapping models to obtain four decoupled features.

[0123] It should be noted that the above mapping model can include convolutional layers with 3×3 kernels (i.e., Conv3×3 in the figure), activation layers with ReLU activation function (i.e., ReLU in the figure), and normalization layers (i.e., BN in the figure). Taking one of the mapping models as an example, after multi-dimensional interaction features are extracted by the convolutional layers of the mapping model, a multi-dimensional representation is obtained. This multi-dimensional representation is used as the input to the activation layer of the mapping model to obtain the activation representation. Finally, based on this activation representation and the normalization layer, the decoupled features are obtained.

[0124] S420, based on multiple decoupling features, obtains the mapping features.

[0125] In some embodiments, multiple decoupling features are added together to obtain the above-mentioned mapping features.

[0126] refer to Figure 2 The above steps correspond to step d, which fuses the four decoupled features to obtain the above mapping features.

[0127] It is understandable that by first decoupling and reconstructing multi-dimensional interactive features to obtain multiple decoupled features, and then obtaining mapping features based on these decoupled features, it is possible to effectively split and fuse key information of different dimensions and types in the features, eliminate redundant interference and coupling confusion between features, and allow each decoupled feature to focus on specific types of key feature information, thereby improving the purity and accuracy of feature representation.

[0128] Meanwhile, the synergistic effect of multiple decoupled features enables multi-dimensional and hierarchical accurate characterization of coronary artery tree features, making the final mapped features more consistent with the needs of the segmentation output space, further enhancing the ability to accurately convert features into segmentation results, providing better feature support for the upsampling step, thereby further improving the accuracy and structural integrity of coronary artery tree binary mask segmentation, and more effectively avoiding the problem of missed or misidentified branch vessels due to feature coupling.

[0129] In some embodiments, multi-dimensional interaction features are mapped to the segmented output space to obtain mapped features. The method further includes fusing spatial dimension interaction features, channel and height interaction features, and channel and width interaction features to obtain fused features, and mapping the fused features to the segmented output space to obtain mapped features.

[0130] For example, based on preset weighting coefficients, spatial dimension interaction features, channel-height interaction features, and channel-width interaction features are fused to obtain fused features.

[0131] The expression for the fusion feature is:

[0132]

[0133] Where y is the fusion feature, α, β and γ are all weight coefficients, A is the spatial dimension interaction feature, B is the channel and height interaction feature, and C is the channel and width interaction feature.

[0134] It should be noted that during model training, the aforementioned weight coefficients are automatically updated through the backpropagation mechanism, enabling the model to dynamically adjust the importance of features in different dimensions according to the feature distribution of specific samples, thereby achieving an adaptive balance of spatial structure, vertical consistency, and horizontal structure information.

[0135] Understandably, according to the implementation method of this application, by first fusing the interactive features of spatial dimension, channel and height, and channel and width to obtain fused features, and then mapping the fused features to the segmentation output space, the organic integration of interactive features of different dimensions is achieved. This effectively avoids the limitations of single-dimensional feature mapping, allowing the fused features to comprehensively and holistically represent the complete feature information of the main branches and branches of the coronary artery tree in each dimension. This makes the mapped features more in line with the feature requirements of the segmentation output space, further improving the accuracy and completeness of feature representation. This provides a better feature foundation for subsequent upsampling steps, thereby further ensuring the accuracy and structural integrity of the binary mask segmentation of the coronary artery tree and effectively avoiding the problem of missed identification of branch vessels due to the fragmentation of dimensional features.

[0136] refer to Figure 8 , Figure 8 A vascular tree grading diagram from an embodiment of this application is shown.

[0137] In some embodiments, the method further includes vascular grading of the binary mask to obtain the aortic region and branch artery region in the binary mask.

[0138] For example, based on the aforementioned binary mask, a skeletonization algorithm is used to simplify the blood vessels in the binary mask into elongated centerlines. Based on these centerlines, a blood vessel tracing algorithm is used to trace the blood vessel path from its opening to its terminal point. Nodes represent the bifurcation and termination points of the blood vessels, and edges represent the segments connecting them, forming a network model representing the entire blood vessel tree and reflecting the hierarchy and spatial layout of the vascular system. This network model includes the aforementioned aortic region and branch artery regions.

[0139] like Figure 8As shown, each vessel in the network model is labeled with a different color to distinguish between branch artery regions and the aortic region. This provides a crucial prerequisite and accurate regional division basis for calculating the microvascular resistance reserve (MRR) of the main branch or branches separately.

[0140] It should be noted that the skeletonization algorithm and blood vessel tracking algorithm mentioned above can refer to existing technologies, and will not be elaborated here.

[0141] Therefore, compared with existing technologies, the coronary artery tree segmentation method provided in this application significantly improves upon existing methods. By first extracting temporal segments from coronary angiography image sequences and then compressing and fusing channel and temporal features across frames, multi-dimensional interactive features are extracted and fused step-by-step. Decoupling and reconstruction are performed before feature mapping to avoid interference from features of different dimensions. Finally, upsampling is used to obtain an accurate and complete binary mask of the coronary artery tree that considers both main branches and branches. This provides a more reliable technical basis for the functional diagnosis of coronary heart disease and enhances the application value of computational physiology techniques based on coronary angiography in clinical diagnosis and treatment. It solves the problem of inaccurate mask segmentation caused by existing binary mask segmentation of coronary artery trees that only considers main branches and ignores branches.

[0142] This application provides a coronary artery tree segmentation device, comprising:

[0143] The acquisition module is used to acquire coronary angiography image sequences;

[0144] The compression module is used to compress each frame of the coronary angiography image sequence across frames to obtain a compressed image sequence corresponding to each frame, so that the channel features and temporal features of each frame are fused.

[0145] The feature extraction module is used to extract features based on compressed image sequences to obtain multi-dimensional interactive features, including spatial dimension interactive features, channel and height interactive features, and channel and width interactive features.

[0146] The mapping module is used to map multi-dimensional interactive features to the segmented output space to obtain mapped features;

[0147] The upsampling module is used to upsample the mapped features to obtain a binary mask of the coronary artery tree in the image, so as to achieve coronary artery tree segmentation.

[0148] This application provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. When the processor loads and executes the instruction or program, the electronic device performs the coronary artery tree segmentation method described in the above embodiments. Its specific functions and corresponding technical effects can be found in the above embodiments. Figures 1-8 The segmentation method for the coronary artery tree explained above will not be repeated here. The following section will combine... Figure 9 The electronic devices described in the embodiments of this application will be described in detail.

[0149] refer to Figure 9 The diagram shows a block diagram of an electronic device 1200 according to one embodiment of this application. The electronic device 1200 may include one or more processors 1201 coupled to a controller hub 1203. In at least one embodiment, the controller hub 1203 communicates with the processor 1201 via a multi-branch bus such as a front side bus (FSB) 1210, a point-to-point interface such as a quick path interconnect (QPI), or a similar connection. The processor 1201 executes instructions controlling general types of data processing operations. In one embodiment, the controller hub 1203 includes, but is not limited to, a graphics memory controller hub (GMCH) (not shown) and an input / output hub (IOH) (which may be on a separate chip) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.

[0150] Electronic device 1200 may also include a coprocessor 1202 and a memory 1204 coupled to a controller hub 1203. Alternatively, one or both of the memory and the GMCH may be integrated within the processor (as described in this application), with memory 1204 and coprocessor 1202 directly coupled to processor 1201 and controller hub 1203, which resides on a single chip with the IOH. Memory 1204 may be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of both. In one embodiment, coprocessor 1202 is a dedicated processor, such as, for example, a high-throughput MIC (many integerized core) processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose computing on GPU (GPGPU), or an embedded processor, etc. Optional properties of coprocessor 1202 are indicated by dashed lines. Figure 9 middle.

[0151] As a computer-readable storage medium, memory 1204 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. For example, memory 1204 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device such as one or more hard-disk drives (HDDs), one or more compact disc (CD) drives, and / or one or more digital versatile disc (DVD) drives.

[0152] In one embodiment, electronic device 1200 may further include a network interface controller (NIC) 1206. Network interface 1206 may include a transceiver for providing a radio interface for electronic device 1200 to communicate with any other suitable device, such as a front-end module, antenna, etc. In various embodiments, network interface 1206 may be integrated with other components of electronic device 1200. Network interface 1206 can implement the functions of the communication unit in the above embodiments.

[0153] Electronic device 1200 may further include input / output (I / O) device 1205. I / O device 1205 may include: a user interface designed to enable a user to interact with electronic device 1200; a peripheral component interface designed to enable peripheral components to also interact with electronic device 1200; and / or sensors designed to determine environmental conditions and / or location information related to electronic device 1200.

[0154] It is worth noting that, Figure 9 This is merely an example. That is, although... Figure 9 The electronic device 1200 shown includes multiple devices such as a processor 1201, a coprocessor 1202, a controller hub 1203, and a memory 1204. However, in practical applications, devices using the methods of this application may include only a portion of the devices in the electronic device 1200. For example, it may include only the processor 1201 and the network interface 1206. Figure 9 The properties of the optional devices are shown in dashed lines. According to some embodiments of this application, the memory 1204, which is a computer-readable storage medium, stores instructions or programs that, when executed on a computer, perform the coronary artery tree segmentation method described in the above embodiments. Specific details can be found in the methods described in the above embodiments, and will not be repeated here.

[0155] Now for reference Figure 10The diagram shown is a block diagram of a system-on-chip (SoC) 1300 according to an embodiment of this application. Figure 10 In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 10 In this SoC 1300, the following are included: an interconnect unit 1350 coupled to an application processor 1310; a system proxy unit 1380; a bus controller unit 1390; an integrated memory controller unit 1340; a group or one or more coprocessors 1320, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1330; and a direct memory access (DMA) unit 1360. In one embodiment, the coprocessor 1320 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a GPGPU, a high-throughput MIC processor, or an embedded processor.

[0156] The static random access memory (SRAM) cell 1330 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of those instructions. These instructions may include, when executed by at least one unit in the processor, causing the SoC 1300 to perform the coronary artery vascular tree segmentation method according to the above embodiments, as detailed in the methods described above, which will not be repeated here.

[0157] This application provides a computer-readable storage medium storing at least one instruction or at least one program. The instruction or program is loaded and executed by a processor to implement the coronary artery tree segmentation method described in the above embodiments. Its specific functions and corresponding technical effects can be found in the above embodiments. Figures 1-8 The segmentation method for the coronary artery tree explained herein will not be repeated here.

[0158] This application provides a computer program product, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device implements the coronary artery tree segmentation method described in the above embodiments. Its specific functions and corresponding technical effects can be found in the above embodiments. Figures 1-8 The segmentation method for the coronary artery tree explained herein will not be repeated here.

[0159] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0160] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0161] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0162] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0163] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0164] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0165] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0166] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0167] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A method for segmenting a coronary artery tree, characterized in that, include: Obtain coronary angiography image sequences; Based on cross-frame compression of each frame in the coronary angiography image sequence, a compressed image sequence corresponding to each frame is obtained, so that the channel features and temporal features of each frame are fused. Feature extraction is performed based on the compressed image sequence to obtain multi-dimensional interactive features, wherein the multi-dimensional interactive features include spatial dimension interactive features, channel and height interactive features, and channel and width interactive features. The multi-dimensional interactive features are mapped onto the segmentation output space to obtain the mapped features; The mapping features are upsampled to obtain a binary mask of the coronary artery tree of the image, so as to achieve coronary artery tree segmentation.

2. The method according to claim 1, characterized in that, The step of compressing each frame of the coronary angiography image sequence across frames to obtain a compressed image sequence corresponding to each frame includes: Based on each frame of the image, the coronary angiography image sequence is segmented to obtain a time sequence segment; The compressed image sequence is obtained by performing cross-frame compression based on the time segment.

3. The method according to claim 2, characterized in that, The step of performing cross-frame compression based on the time segment to obtain the compressed image sequence includes: High-dimensional features are obtained by extracting features based on the time sequence segments. Based on the high-dimensional features, feature extraction is performed to obtain high-level spatial semantic features; The compressed image sequence is obtained by performing a non-parametric tensor transformation based on the high-level spatial semantic features.

4. The method according to claim 1, characterized in that, The feature extraction based on the compressed image sequence yields multi-dimensional interactive features, including: The compressed image sequence is used as input data for the spatial interaction model to obtain the spatial dimension interaction features; The compressed image sequence is used as input data for the channel-high interaction model to obtain the channel-high interaction features; The compressed image sequence is used as input data for the channel-width interaction model to obtain the channel-width interaction features.

5. The method according to claim 1, characterized in that, The step of mapping the multi-dimensional interaction features to the segmentation output space to obtain the mapped features further includes: The spatial dimension interaction features, the channel and height interaction features, and the channel and width interaction features are fused to obtain the fused features; The fused features are mapped onto the segmented output space to obtain the mapped features.

6. The method according to claim 1, characterized in that, The step of mapping the multi-dimensional interaction features to the segmentation output space to obtain the mapped features includes: Decoupling and reconstruction are performed based on the multi-dimensional interaction features to obtain multiple decoupling features; The mapping feature is obtained based on the multiple decoupling features.

7. The method according to claim 1, characterized in that, Also includes: The binary mask is used to classify blood vessels to obtain the aortic region and branch artery region in the binary mask.

8. A segmentation device for coronary artery tree, characterized in that, include: The acquisition module is used to acquire coronary angiography image sequences; The compression module is used to compress each frame of the coronary angiography image sequence across frames to obtain a compressed image sequence corresponding to each frame of the image, so that the channel features and temporal features of each frame of the image are fused. The feature extraction module is used to extract features based on the compressed image sequence to obtain multi-dimensional interactive features, wherein the multi-dimensional interactive features include spatial dimension interactive features, channel and height interactive features, and channel and width interactive features. The mapping module is used to map the multi-dimensional interactive features to the segmentation output space to obtain the mapped features; An upsampling module is used to upsample the mapped features to obtain a binary mask of the coronary artery tree of the image, so as to achieve coronary artery tree segmentation.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the coronary artery tree segmentation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the coronary artery tree segmentation method as described in any one of claims 1 to 7.

Images