A semi-supervised coronary artery segmentation method based on frequency domain information fusion

Abstract

The present invention discloses a semi-supervised coronary artery segmentation method based on frequency domain information fusion. The method first obtains a coronary computed tomography angiography (CCTA) image and stores it in NIFTI format; then manually annotates the NIFTI format data to obtain the specific structure of the coronary artery and constructs a coronary artery dataset; inputs the coronary artery dataset into a semi-supervised segmentation network based on frequency domain information fusion for training to obtain a training model; and uses the obtained training model to predict new coronary artery data to obtain a coronary artery segmentation mask. The present invention proposes a semi-supervised coronary artery segmentation method based on frequency domain information fusion, which can obtain highly accurate output results with a small amount of manual annotation, while solving the problem of difficult coronary artery segmentation. The present invention can directly obtain a coronary artery segmentation mask through a trained deep learning segmentation model, providing technical support for clinicians' imaging diagnosis work.

Description

Technical Field

[0001] The present invention relates to a semi-supervised coronary artery segmentation method based on frequency domain information fusion, and belongs to the technical field of medical image processing. Background Art

[0002] Currently, the application of artificial intelligence technology in the medical field has been widely explored and has made significant progress. Computer vision technology, as a branch of artificial intelligence, plays a vital role in medical image processing. Utilizing technologies such as deep learning, computers can automatically classify, identify, and analyze structures and lesions in medical images, providing assistance and advice to clinicians in diagnosis and treatment, and significantly improving the efficiency and accuracy of medical imaging diagnosis.

[0003] The heart is a vital organ in the human body. Its primary function is to power blood flow and supply blood to all tissues and organs throughout the body. The heart is connected to the rest of the body through a series of blood vessels, of which the coronary arteries are important blood vessels that supply nutrients to the heart itself. The coronary arteries originate in the aortic sinus at the root of the aorta and are generally divided into two major branches: the left coronary artery and the right coronary artery. The branches of the coronary arteries form a complex network on the surface of the heart to ensure that the heart muscle receives sufficient blood and nutrients to maintain its normal contraction and relaxation functions. Coronary artery disease (CAD) is the most common cardiovascular disease. Its main cause is the accumulation of plaque in the heart arteries (atherosclerosis), which reduces blood flow to the myocardium and causes insufficient or interrupted blood supply to the coronary arteries. Clinical symptoms mainly include chest pain (angina pectoris), dyspnea, myocardial infarction, arrhythmia, heart failure, etc.

[0004] Coronary computed tomography angiography (CCTA), as a non-invasive imaging modality, has been widely used in clinical diagnosis. Processing and reconstruction of high-resolution CCTA images can assist physicians in assessing coronary artery morphology, stenosis severity, and plaque distribution, facilitating subsequent diagnosis and treatment planning. Accurately extracting coronary artery structure from CCTA images is crucial for clinicians to assess stenosis severity and diagnose plaque morphology. Currently, manual or semi-automatic extraction by expert physicians yields the most accurate results, but this segmentation approach is time-consuming and relies on subjective experience, leading to discrepancies in segmentation results. Deep learning-based computer vision, a key branch of artificial intelligence, has played a significant role in medical image processing in recent years. However, deep learning-based models typically require large amounts of accurately labeled data for training, and the complex anatomy of coronary arteries, with their narrow, long, and tortuous branches, can easily lead to fragmented segmentation results. Considering the reality that it is difficult to obtain a large number of accurate labels for CCTA images, a targeted design of a semi-supervised segmentation method that can be trained based on a small amount of labeled data is used to accurately extract the coronary artery structure, which is of great value for imaging diagnosis and clinical decision-making. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology by taking into account the difficulty of obtaining a large number of accurate labels for medical images. A semi-supervised coronary artery segmentation method based on frequency domain information fusion is provided. The frequency domain information of the original CCTA image is extracted by discrete wavelet transform and then fused with the features extracted by the convolutional network. The information between high-frequency and low-frequency frequency domains is mutually constrained to guide the network to fully utilize unlabeled data to learn CCTA image features with a small number of labels, thereby obtaining accurate coronary artery segmentation results and providing technical support for clinicians to carry out diagnostic work.

[0006] To solve the above technical problems, the present invention provides a semi-supervised coronary artery segmentation method based on frequency domain information fusion, comprising the following steps:

[0007] (1) CCTA imaging data acquisition and storage: Collect the patient's CCTA imaging DICOM data and store the data in NIFTI format using 3DSlicer software;

[0008] (2) Coronary artery annotation and dataset construction: 3D Slicer software was used to extract regions of interest from the acquired CCTA image data, and the coronary artery structure of the CCTA image data was obtained through manual annotation. The manual annotation results were stored in NIFTI format, and the CCTA image data and coronary artery labels were combined to form a coronary artery dataset;

[0009] (3) Deep learning segmentation model training: The coronary artery dataset obtained in step (2) is randomly divided into four equal parts. The training is performed using a four-fold cross-validation method. Each part of the data is used as a validation set, and the remaining three parts of the data are input into the network as training sets. The network is trained four times so that each divided data is verified, thereby ensuring the accuracy of the model evaluation. In each fold of training, only a small amount of coronary artery labels corresponding to CCTA image data are set to enter the network, and the remaining data is input into the deep learning segmentation model in an unlabeled form for training;

[0010] (4) Coronary artery segmentation prediction: Use the segmentation model trained in step (3) to perform coronary artery segmentation on CCTA image data.

[0011] Specifically, the deep learning segmentation model in step (3) includes an encoder, a decoder, a jump connection branch, a frequency domain information fusion module, and a multi-scale global feature perception module; wherein the encoder is composed of a convolution block and a pooling layer, and the encoder part contains a total of 5 convolution blocks and 4 pooling layers. The image input to the network obtains feature maps of different scales through convolution and pooling operations; a multi-scale global feature perception module is used between the encoder and the decoder to enhance the network's ability to capture spatial information and channel information, so as to enhance the extraction effect of global features; the jump connection branch is used to connect the encoder and the decoder, and splice the feature map extracted by the convolution block in the encoder with the feature map obtained by the decoder, which helps the network enhance its ability to understand information at different levels while avoiding the gradient loss problem of the deep network; the frequency domain information fusion module is divided into two parts, high frequency and low frequency, and is used to introduce the frequency domain information of the original image at the jump connection branch. First, the multi-scale low frequency and high frequency information of the original image are obtained by wavelet decomposition, and the convolution features obtained by the convolution block are fused with the low frequency features or high frequency features obtained by wavelet decomposition, and then spliced ​​with the subsequent decoder part through the jump connection branch. The decoder part is divided into two different branches, each decoder branch consists of 4 convolution blocks and 4 upsampling layers, where decoder one is used to receive the low-frequency fusion features transmitted from the frequency domain information fusion module (low frequency), and decoder two is used to receive the high-frequency fusion features transmitted from the frequency domain information fusion module (high frequency). The outputs of the two decoders are finally converted into two prediction results focusing on different frequency domains through the output convolution layer respectively; the labeled data and unlabeled data input into the network are predicted through the network to obtain the prediction results, where the prediction results obtained by the labeled data generate the supervised probability through the softmax function and compare it with the true label for calculating the supervised loss, and the prediction results obtained by the unlabeled data generate the unsupervised probability and pseudo label through the softmax function and argmax function respectively for calculating the cross loss function.

[0012] In step (3), the data input to the deep learning segmentation model needs to obtain features through the encoder. The data input to the network is first extracted by a convolution block to extract the feature map F1. The dimension of the feature map F1 is defined as (W, H, D, C), where W, H, D represent the width, height, and depth information of the image spatial dimension respectively, and C represents the number of channels, W = H = 112, D = 80, and C = 16. Subsequently, the feature maps F2, F3, and F4 are extracted in sequence through a combination of convolution blocks and pooling layers. Each time the feature is extracted, the spatial dimension is reduced to half of the feature map of the previous layer, and the channel dimension is expanded to 4 times the number of channels of the feature map of the previous layer. Each convolution block consists of two groups of convolution layers with the same structure. Each group of convolution layers contains a three-dimensional convolution (Conv3d) with a convolution kernel of (3, 3, 1), a three-dimensional instance normalization (InstanceNorm3d), and an activation function (ReLU). The pooling layer uses a three-dimensional maximum pooling (MaxPool3d) with a kernel of (2, 2, 2).

[0013] The multi-scale global feature perception module in step (3) processes the features extracted by the encoder to enhance the network's ability to perceive global features. The specific process can be expressed as follows:

[0014] F spatial =conv dp3 (conv dp2 (conv dp1 (F4)))

[0015] F5=BN(GELU(conv pw (F spatial )))+F4

[0016] The multi-scale global feature perception module can be specifically divided into two parts: the multi-scale spatial information perception module and the channel information perception module. The multi-scale spatial information perception module is constructed by three depthwise convolutions with different hole rates. dp1 ,conv dp2 ,conv dp3 Operate the feature map F4 obtained in the last layer of the encoder to aggregate spatial features, conv dp1 ,conv dp2 and conv dp3 The void rates are 1, 2, and 3 respectively. The feature F after spatial information aggregation spatial Then pass Pointwise convolution conv pw The activation function GELU and batch normalization (BatchNorm, BN) operate to aggregate channel features, and introduce residual connections, which are added to the feature map F4 before the input module to enhance the performance of information transmission.

[0017] The frequency domain information fusion module in step (3) extracts low-frequency and high-frequency information from the original image, enhancing the deep learning model's ability to extract different information while assisting in building a differentiated decoder for semi-supervised learning. The frequency domain information fusion module processes features as follows:

[0018]

[0019] The frequency domain information fusion module receives the feature maps F1, F2, F3, and F4 obtained from the convolution block in the encoder, and fuses them with the low-frequency information and high-frequency information obtained by discrete wavelet transform (DWT) of the original image. The fused features will be transmitted to decoder 1 and decoder 2 respectively in the subsequent process. The feature map F1 does not add additional wavelet features when passing through the frequency domain information fusion module, which is equivalent to directly connecting with the decoder feature map through a jump connection. and The up-sampled features are fused. Trans Represents a three-dimensional convolution with a convolution kernel of (1,1,1), which is used to convert the wavelet information obtained after discrete wavelet decomposition into the same channel dimension as the convolution feature. ReLU represents the activation function. Conv Vote Represents the voting convolution block, which contains a three-dimensional convolution with a convolution kernel of (1,1,1), a batch normalization (BatchNorm, BN) operation and an activation function Sigmoid. and They represent the low-frequency fusion features and high-frequency fusion features output by the low-frequency information fusion module respectively.

[0020] The decoder part in step (3) includes two different decoders. The features of the encoder and the multi-scale global feature perception module are gradually upsampled and connected through the features transmitted by the jump connection after passing through the frequency domain fusion module to generate the final prediction result. Decoder 1 and decoder 2 respectively receive the feature map F5 obtained after passing through the multi-scale global feature perception module. The feature transmission process of decoder 1 and decoder 2 is similar, only the features from different frequency domain fusion modules are different. Taking decoder 1 as an example, the feature transmission process is as follows:

[0021]

[0022] Among them, Conv represents the convolution block, Concat represents the feature splicing operation, and Upsample represents the upsampling operation, which is specifically performed using trilinear interpolation (Trilinear Interpolation). represents the features obtained by the decoder, Specifically represents the features of the previous layer obtained by the decoder, Fi+1 In the formula, it specifically refers to the feature map F5 obtained after the multi-scale global feature perception module, Represents the low-frequency features obtained by the frequency domain fusion module. The feature transmission process of decoder 2 is the same as that of decoder 1, only the low-frequency features of the input are Replace with high-frequency features

[0023] After the dual decoder of the deep learning segmentation model, the labeled data and unlabeled data of the input network are used to obtain the prediction results, where the prediction results obtained by the labeled data are used to generate the supervision probability P through the softmax function. sup1 and P sup2 , and perform supervision loss L between the real label G sup The prediction results obtained by unlabeled data are calculated by the softmax function and the argmax function to generate the unsupervised probability P unsup1 and P unsup2 , and perform unsupervised cross loss function L with pseudo labels PL1 and PL2 unsup Calculation of the total loss function L all The specific calculation process is as follows:

[0024] L all =L sup +λL unsup

[0025] =L CM (P sup1 ,G)+L CM (P sup2 ,G)+λ(L CE (P unsup1 ,PL2)+L CE (P unsup2 ,PL1))

[0026] Among them, L CM represents the coronary morphology loss function (CMLoss), L CE represents cross entropy loss (CELoss), Where t represents the number of iterations of the current model training, t max Indicates the total number of model training iterations.

[0027] L CM By the regional loss function L Region And the boundary loss function L Boundary The specific calculation process is as follows:

[0028] L CM =LRegion +αL Boundary

[0029]

[0030] Among them, C represents the number of categories to be segmented, N represents the total number of pixels in the 3D target, represents the true label of the n-th voxel in the c-th category, It represents the corresponding prediction result, and Respectively express and The corresponding distance transform map is squared. α is a dynamic balancing weight. Since fitting the boundary is a difficult task at the beginning of model training, when epoch < 100, α is set to 0, so that only the regional loss function L is used. Region Constraints are made to obtain the initial segmentation mask stably and quickly. When epoch>100, α is increased by 0.02 after each iteration of the model until it reaches 1 and then no longer changes. This enables the model to use region and boundary information as joint constraints, making the prediction results more continuous and closer to the true target.

[0031] Compared to existing technologies, the present invention has the following beneficial effects: It provides a semi-supervised segmentation method based on frequency domain information fusion, which can learn rich image information using a small amount of labeled and unlabeled data; it uses an encoder-decoder format to extract multi-scale features, and combines this with a multi-scale global feature perception module to further enhance the model's ability to learn global features at different resolution scales; the model constructs a frequency domain information fusion module to guide the decoder to improve its perception of features in different frequency domains of the original CCTA, and on this basis, constructs two different decoders. The prediction results of the two decoders are cross-constrained through the calculation of a loss function, thereby further improving the final effect of the semi-supervised deep learning segmentation model, helping clinicians obtain patients' coronary artery morphology information, and assisting in subsequent diagnosis and treatment plan formulation. This method addresses the difficulty of obtaining a large number of accurate labels for medical images, overcomes the shortcomings of existing technologies, and provides technical support for clinicians in conducting diagnostic work. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0033] Figure 2 This is a schematic diagram of the deep learning segmentation network architecture of the present invention;

[0034] Figure 3 Schematic diagram of the multi-scale global feature perception module structure in the deep learning segmentation network;

[0035] Figure 4 This is a schematic diagram of the frequency domain information fusion module structure in the deep learning segmentation network;

[0036] Figure 5 Schematic diagram of the segmentation result of the present invention. DETAILED DESCRIPTION

[0037] The embodiments of the present invention are described in further detail below with reference to the accompanying drawings.

[0038] Example 1: Figure 1 A semi-supervised coronary artery segmentation method based on frequency domain information fusion is shown, comprising the following steps:

[0039] (1) CCTA imaging data acquisition and storage: Collect the patient's CCTA imaging DICOM data and store the data in NIFTI format using 3DSlicer software;

[0040] (2) Coronary artery annotation and dataset construction: 3D Slicer software was used to extract regions of interest from the acquired CCTA image data, and the coronary artery structure of the CCTA image data was obtained through manual annotation. The manual annotation results were stored in NIFTI format, and the CCTA image data and coronary artery labels were combined to form a coronary artery dataset;

[0041] (3) Deep learning segmentation model training: The coronary artery dataset obtained in step (2) is randomly divided into four equal parts. The training is performed using a four-fold cross-validation method. Each part of the data is used as a validation set, and the remaining three parts of the data are input into the network as training sets. The network is trained four times so that each divided data is verified, thereby ensuring the accuracy of the model evaluation. In each fold of training, only a small amount of coronary artery labels corresponding to CCTA image data are set to enter the network, and the remaining data is input into the deep learning segmentation model in an unlabeled form for training;

[0042] (4) Coronary artery segmentation prediction: Use the segmentation model trained in step (3) to perform coronary artery segmentation on CCTA image data.

[0043] like Figure 2As shown in FIG, the deep learning segmentation model in step (3) includes an encoder, a decoder (decoder 1 and decoder 2), a jump connection branch, a frequency domain information fusion module (low frequency / high frequency), and a multi-scale global feature perception module; wherein the encoder is composed of a convolution block and a pooling layer, and the encoder part contains a total of 5 convolution blocks and 4 pooling layers. The image input to the network is subjected to convolution and pooling operations to obtain feature maps of different scales; a multi-scale global feature perception module is used between the encoder and the decoder to enhance the network's ability to capture spatial information and channel information, so as to enhance the extraction effect of global features; the jump connection branch is used to Connecting the encoder and decoder, splicing the feature map extracted by the convolution block in the encoder with the feature map obtained by the decoder, helps the network enhance its ability to understand information at different levels while avoiding the gradient loss problem of deep networks; the frequency domain information fusion module is divided into two parts, high frequency and low frequency, and is used to introduce the frequency domain information of the original image at the jump connection branch. First, the multi-scale low-frequency and high-frequency information of the original image is obtained through wavelet decomposition, and the convolution features obtained by the convolution block are fused with the low-frequency features or high-frequency features obtained by wavelet decomposition, and then spliced ​​with the subsequent decoder part through the jump connection branch. The decoder part is divided into two different branches, each decoder branch consists of 4 convolution blocks and 4 upsampling layers, where decoder one is used to receive the low-frequency fusion features transmitted from the frequency domain information fusion module (low frequency), and decoder two is used to receive the high-frequency fusion features transmitted from the frequency domain information fusion module (high frequency). The outputs of the two decoders are finally converted into two prediction results focusing on different frequency domains through the output convolution layer respectively; the labeled data and unlabeled data input into the network are predicted through the network to obtain the prediction results, where the prediction results obtained by the labeled data generate the supervised probability through the softmax function and compare it with the true label for calculating the supervised loss, and the prediction results obtained by the unlabeled data generate the unsupervised probability and pseudo label through the softmax function and argmax function respectively for calculating the cross loss function.

[0044] In step (3), the data input to the deep learning segmentation model needs to obtain features through the encoder. The data input to the network is first extracted by a convolution block to extract the feature map F1. The dimension of the feature map F1 is defined as (W, H, D, C), where W, H, D represent the width, height, and depth information of the image spatial dimension respectively, and C represents the number of channels, W = H = 112, D = 80, and C = 16. Subsequently, the feature maps F2, F3, and F4 are extracted in sequence through a combination of convolution blocks and pooling layers. Each time the feature is extracted, the spatial dimension is reduced to half of the feature map of the previous layer, and the channel dimension is expanded to 4 times the number of channels of the feature map of the previous layer. Each convolution block consists of two groups of convolution layers with the same structure. Each group of convolution layers contains a three-dimensional convolution (Conv3d) with a convolution kernel of (3, 3, 1), a three-dimensional instance normalization (InstanceNorm3d), and an activation function (ReLU). The pooling layer uses a three-dimensional maximum pooling (MaxPool3d) with a kernel of (2, 2, 2).

[0045] like Figure 3 As shown in FIG, the multi-scale global feature perception module in step (3) processes the features extracted by the encoder to enhance the network's perception of global features. The specific process can be expressed as follows:

[0046] F spatial =conv dp3 (conv dp2 (conv dp1 (F4)))

[0047] F5=BN(GELU(conv pw (F spatial )))+F4

[0048] The multi-scale global feature perception module can be specifically divided into two parts: the multi-scale spatial information perception module and the channel information perception module. The multi-scale spatial information perception module is constructed by three depthwise convolutions with different hole rates. dp1 ,conv dp2 ,conv dp3 Operate the feature map F4 obtained in the last layer of the encoder to aggregate spatial features, conv dp1 ,conv dp2 and conv dp3 The void rates are 1, 2, and 3 respectively. The feature F after spatial information aggregation spatial Then pass Pointwise convolution conv pw The activation function GELU and batch normalization (BatchNorm, BN) operate to aggregate channel features, and introduce residual connections, which are added to the feature map F4 before the input module to enhance the performance of information transmission.

[0049] like Figure 4 As shown in Figure 3, the frequency domain information fusion module in step (3) extracts the low-frequency and high-frequency information of the original image, enhancing the deep learning model's ability to extract different information while assisting in building a differentiated decoder for semi-supervised learning. The frequency domain information fusion module processes the features as follows:

[0050]

[0051] The frequency domain information fusion module receives the feature maps F1, F2, F3, and F4 obtained from the convolution block in the encoder, and fuses them with the low-frequency information and high-frequency information obtained by discrete wavelet transform (DWT) of the original image. The fused features will be transmitted to decoder 1 and decoder 2 respectively in the subsequent process. The feature map F1 does not add additional wavelet features when passing through the frequency domain information fusion module, which is equivalent to directly connecting with the decoder feature map through a jump connection. and The up-sampled features are fused. Trans Represents a three-dimensional convolution with a convolution kernel of (1,1,1), which is used to convert the wavelet information obtained after discrete wavelet decomposition into the same channel dimension as the convolution feature. ReLU represents the activation function. Conv Vote Represents the voting convolution block, which contains a three-dimensional convolution with a convolution kernel of (1,1,1), a batch normalization (BatchNorm, BN) operation and an activation function Sigmoid. and They represent the low-frequency fusion features and high-frequency fusion features output by the low-frequency information fusion module respectively.

[0052] like Figure 2 As shown, the decoder part in step (3) includes two different decoders. The features after the encoder and the multi-scale global feature perception module are gradually upsampled, and after passing through the frequency domain fusion module, the features transmitted through the jump connection are connected to generate the final prediction result. Decoder 1 and decoder 2 respectively receive the feature map F5 obtained after passing through the multi-scale global feature perception module. The feature transmission process of decoder 1 and decoder 2 is similar, only the features from different frequency domain fusion modules are different. Taking decoder 1 as an example, the feature transmission process is as follows:

[0053]

[0054] Among them, Conv represents the convolution block, Concat represents the feature splicing operation, and Upsample represents the upsampling operation, which is specifically performed using trilinear interpolation (Trilinear Interpolation). represents the features obtained by the decoder, Specifically represents the features of the previous layer obtained by the decoder, F i+1 In the formula, it specifically refers to the feature map F5 obtained after the multi-scale global feature perception module, Represents the low-frequency features obtained by the frequency domain fusion module. The feature transmission process of decoder 2 is the same as that of decoder 1, only the low-frequency features of the input are Replace with high-frequency features

[0055] like Figure 2 As shown, after the dual decoder of the deep learning segmentation model, the labeled data and unlabeled data of the input network are used to obtain the prediction results, where the prediction results obtained by the labeled data are used to generate the supervision probability P through the softmax function. sup1 and P sup2 , and perform supervision loss L between the real label G sup The prediction results obtained by unlabeled data are calculated by the softmax function and the argmax function to generate the unsupervised probability P unsup1 and P unsup2 , and perform unsupervised cross loss function L with pseudo labels PL1 and PL2 unsup Calculation of the total loss function L all The specific calculation process is as follows:

[0056] L all =L sup +λL unsup

[0057] =L CM (P sup1 ,G)+L CM (P sup2 ,G)+λ(L CE (P unsup1 ,PL2)+L CE (P unsup2 ,PL1))

[0058] Among them, L CM represents the coronary morphology loss function (CMLoss), L CE represents cross entropy loss (CELoss), Where t represents the number of iterations of the current model training, t max Indicates the total number of model training iterations.

[0059] L CM By the regional loss function L RegionAnd the boundary loss function L Boundary The specific calculation process is as follows:

[0060] L CM =L Region +αL Boundary

[0061]

[0062] Among them, C represents the number of categories to be segmented, N represents the total number of pixels in the 3D target, represents the true label of the n-th voxel in the c-th category, It represents the corresponding prediction result, and Respectively express and The corresponding distance transform map is squared. α is a dynamic balancing weight. Since fitting the boundary is a difficult task at the beginning of model training, when epoch < 100, α is set to 0, so that only the regional loss function L is used. Region Constraints are imposed to obtain the initial segmentation mask stably and quickly. When epoch>100, α is increased by 0.02 after each model iteration until it reaches 1 and then no longer changes. This enables the model to use region and boundary information as joint constraints, making the prediction results more continuous and closer to the true target.

[0063] Input the CCTA image data for testing, and use the deep learning segmentation model trained as described in step (3) to perform coronary artery segmentation on the CCTA image data. The generated coronary artery segmentation results are as follows: Figure 5 shown.

[0064] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions fall within the scope of protection of the claims of the present invention.

Claims

1. A semi-supervised coronary artery segmentation method based on frequency domain information fusion, characterized in that: include: (1) CCTA imaging data acquisition and storage: Collect the patient's CCTA imaging DICOM data and store the data in NIFTI format using 3DSlicer software; (2) Coronary artery annotation and dataset construction: 3D Slicer software was used to extract regions of interest from the acquired CCTA image data, and the coronary artery structure of the CCTA image data was obtained through manual annotation. The manual annotation results were stored in NIFTI format, and the CCTA image data and coronary artery labels were combined to form a coronary artery dataset; (3) Deep learning segmentation model training: The coronary artery dataset obtained in step (2) is randomly divided into four equal parts, and the training is performed using a four-fold cross-validation method. Each part of the data is used as a validation set in turn, and the remaining three parts of the data are input into the network as training sets. The network is trained four times so that each divided data is verified, thereby ensuring the accuracy of the model evaluation. In each fold of training, only a small amount of coronary artery labels corresponding to CCTA image data are set to enter the network, and the remaining data are input into the deep learning segmentation model in an unlabeled form for training; (4) Coronary artery segmentation prediction: Use the segmentation model trained in step (3) to perform coronary artery segmentation on CCTA image data; The deep learning segmentation model in step (3) includes an encoder, a decoder, a skip connection branch, a frequency domain information fusion module, and a multi-scale global feature perception module; The encoder consists of convolution blocks and pooling layers. The encoder part contains 5 convolution blocks and 4 pooling layers. The image input to the network obtains feature maps of different scales through convolution and pooling operations; a multi-scale global feature perception module is used between the encoder and the decoder to enhance the network's ability to capture spatial information and channel information, so as to enhance the extraction effect of global features; the jump connection branch is used to connect the encoder and the decoder, and the feature map extracted by the convolution block in the encoder is spliced ​​with the feature map obtained by the decoder, which helps the network to enhance its ability to understand information at different levels while avoiding the gradient loss problem of deep networks; the frequency domain information fusion module is divided into high-frequency and low-frequency parts, which are used to introduce the frequency domain information of the original image at the jump connection branch. First, the multi-scale low-frequency and high-frequency information of the original image are obtained by wavelet decomposition, and the convolution features obtained by the convolution block are fused with the low-frequency features or high-frequency features obtained by wavelet decomposition. The decoder part is divided into two different branches, each of which is composed of 4 convolution blocks and 4 upsampling layers. Decoder 1 is used to receive the low-frequency fusion features transmitted from the low-frequency part of the frequency domain information fusion module, and decoder 2 is used to receive the high-frequency fusion features transmitted from the high-frequency part of the frequency domain information fusion module. The outputs of the two decoders are finally converted into two prediction results focusing on different frequency domains through the output convolution layer respectively; the labeled data and unlabeled data input into the network are predicted by the network to obtain the prediction results, wherein the prediction results obtained by the labeled data generate the supervised probability through the softmax function and compare it with the real label for calculating the supervised loss, and the prediction results obtained by the unlabeled data generate the unsupervised probability and pseudo label through the softmax function and argmax function respectively for calculating the cross loss function.

2. The semi-supervised coronary artery segmentation method based on frequency domain information fusion according to claim 1, characterized in that: In the encoder, the data input to the network is first extracted from a feature map F1 by a convolution block. The dimension of the feature map F1 is defined as (W, H, D, C), where W, H, and D represent the width, height, and depth information of the image space dimension, respectively, and C represents the number of channels, W = H = 112, D = 80, and C = 16. Subsequently, the feature maps F2, F3, and F4 are extracted in sequence through a combination of convolution blocks and pooling layers. Each time a feature is extracted, the spatial dimension is reduced to half of the feature map of the previous layer, and the channel dimension is expanded to four times the number of channels of the feature map of the previous layer. Each convolution block consists of two groups of convolution layers with the same structure. Each group of convolution layers sequentially contains a three-dimensional convolution Conv3d with a convolution kernel of (3, 3, 1), a three-dimensional instance normalization, and an activation function ReLU. The pooling layer uses a three-dimensional maximum pooling MaxPool3d with a kernel of (2, 2, 2).

3. The semi-supervised coronary artery segmentation method based on frequency domain information fusion according to claim 2, characterized in that: The multi-scale global feature perception module processes features as follows: F spatial =conv dp3 (conv dp2 (conv dp1 (F4))) F5=BN(GEL(conv pw (F spatial )))+F4 The multi-scale global feature perception module can be specifically divided into two parts: the multi-scale spatial information perception module and the channel information perception module. The multi-scale spatial information perception module is constructed by three depthwise convolutions with different hole rates. dp1 ,conv dp2 ,conv dp3 Operate the feature map F4 obtained in the last layer of the encoder to aggregate spatial features, conv dp1 ,conv dp2 and conv dp3 The void rates are 1, 2, and 3 respectively. After spatial information aggregation, the feature F spatial Then pass Pointwise convolution conv pw The activation function GELU and batch normalization operation aggregate channel features, and introduce residual connections to add them to the feature map F4 before the input module, which enhances the performance of information transmission.

4. The semi-supervised coronary artery segmentation method based on frequency domain information fusion according to claim 2, characterized in that: The frequency domain information fusion module processes the features as follows: F i L =F i ,if i=1 F i H =F i ,if i=1 The frequency domain information fusion module receives the feature maps F1, F2, F3, and F4 obtained from the convolution block in the encoder, and fuses them with the low-frequency information and high-frequency information obtained by discrete wavelet decomposition of the original image. The fused features will be transmitted to decoder 1 and decoder 2 respectively in the subsequent process. The feature map F1 does not add additional wavelet features when passing through the frequency domain information fusion module, which is equivalent to directly connecting with the decoder feature map through a jump connection. and The features obtained by upsampling are fused, conv Trans Represents a three-dimensional convolution with a convolution kernel of (1,1,1), which is used to convert the wavelet information obtained after discrete wavelet decomposition into the same channel dimension as the convolution feature. ReLU represents the activation function. Conv Vote Represents the voting convolution block, which contains a three-dimensional convolution with a convolution kernel of (1,1,1), a batch normalization operation and an activation function Sigmoid, F i L and F i H They represent the low-frequency fusion features and high-frequency fusion features output by the low-frequency information fusion module respectively.

5. The semi-supervised coronary artery segmentation method based on frequency domain information fusion according to claim 2, characterized in that: In the decoder, the features passed through the encoder and the multi-scale global feature perception module are gradually upsampled and connected through the features transmitted by the jump connection after passing through the frequency domain information fusion module to generate the final prediction result. Decoder 1 and decoder 2 respectively receive the feature map F5 obtained after passing through the multi-scale global feature perception module. The feature transmission process of decoder 1 is as follows: F i D1 =Conv(Concat(Upsample(F i+1 ),F i L )),if i=4 Among them, Conv represents the convolution block, Concat represents the feature splicing operation, Upsample represents the upsampling operation, and the upsampling operation is specifically performed using trilinear interpolation. i D1 represents the features obtained by the decoder, Specifically represents the features of the previous layer obtained by the decoder, F i+1 In the formula, it specifically refers to the feature map F5 obtained after the multi-scale global feature perception module, F i L It represents the low-frequency features obtained by the frequency domain fusion module. The feature transmission process of decoder 2 is the same as that of decoder 1, and only the input low-frequency features F i L Replaced by high-frequency feature F i H .

6. The semi-supervised coronary artery segmentation method based on frequency domain information fusion according to claim 2, characterized in that: After the dual decoder of the deep learning segmentation model, the labeled data and unlabeled data of the input network are used to obtain the prediction results, where the prediction results obtained by the labeled data are used to generate the supervision probability P through the softmax function. sup1 and P sup2 , and perform supervision loss L between the real label G sup The prediction results obtained by unlabeled data are calculated by the softmax function and the argmax function to generate the unsupervised probability P unsup1 and P unsup2 , and perform unsupervised cross loss function L with pseudo labels PL1 and PL2 unsup Calculation of the total loss function L all The specific calculation process is as follows: THE all =L sup +λL unsup =L CM (P sup1 ,G)+L CM (P sup2 ,G)+λ(L CE (P unsup1 ,PL2)+L CE (P unsup2 ,PL1)) Among them, L CM represents the coronary morphology loss function (CMLoss), L CE represents cross entropy loss (CELoss), Where t represents the number of iterations of the current model training, t max represents the total number of model training iterations, L CM By the regional loss function L Region And the boundary loss function L Boundary The specific calculation process is as follows: L CM =L Region +αL Boundary Among them, C represents the number of categories to be segmented, N represents the total number of pixels in the 3D target, represents the true label of the n-th voxel in the c-th category, It represents the corresponding prediction result, and Respectively express and The corresponding distance transformation map is squared, and α is a dynamic balance weight. Since fitting the boundary is a difficult task at the beginning of model training, when epoch < 100, α is set to 0, so that only the regional loss function L is used. Region Constraints are imposed to obtain the initial segmentation mask stably and quickly. When epoch>100, α is increased by 0.02 after each iteration of the model until it reaches 1 and then no longer changes. This enables the model to use region and boundary information as joint constraints, making the prediction results more continuous and closer to the true target.

Images