A three-dimensional multi-scale image segmentation model and an image segmentation method, application and system based thereon

Abstract

The application relates to the technical field of image segmentation, and discloses a three-dimensional multi-scale image segmentation model and an image segmentation method, application and system based on the same, wherein the model comprises a Swin Transformer encoder, a large-core three-dimensional convolutional encoder and two decoders based on a space converter; the Swin Transformer encoder and the large-core three-dimensional convolutional encoder are respectively connected with the two decoders based on the space converter; the encoding layers of the Swin Transformer encoder and the encoding layers of the large-core three-dimensional convolutional encoder are connected one by one; the output of the decoder corresponding to the Swin Transformer encoder is connected with the input of the large-core three-dimensional convolutional encoder; and each decoding layer of the decoder based on the space converter is provided with one space converter block. The application solves the problem that the prior art is not applicable to the segmentation of three-dimensional images with large individual differences, and has the characteristics of being applicable to large-scale and foreground-background proportionally unbalanced images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, and more particularly to a three-dimensional multi-scale image segmentation model and an image segmentation method, application and system based thereon. BACKGROUND

[0002] Coronary computed tomography angiography (CCTA) as a key auxiliary tool is extremely useful for clinicians to assess coronary stenosis, which is a highly efficient non-invasive technique with high sensitivity and specificity. At the same time, CCTA helps to assess other risk factors of coronary heart disease, such as measuring epicardial adipose tissue. However, traditional coronary artery automatic segmentation techniques, such as region growing, centerline tracking, threshold processing and Hessian matrix analysis, usually require preliminary guidance from medical staff to optimize the processing of vascular images. These techniques often show low robustness when facing different individual data sets or the same sample at different stages of disease progression, and their segmentation results may vary depending on the situation. These coronary artery lesion diagnoses usually require experienced doctors to manually extract coronary artery images, which not only consumes time but is also susceptible to observer differences. Therefore, it is particularly important to develop objective and fully automatic coronary artery segmentation techniques.

[0003] Fully automatic coronary artery segmentation is a difficult point. First, the interference of surrounding tissues such as ribs and spine poses a challenge to segmentation. Second, other blood vessels that are similar in appearance and morphology to coronary arteries can lead to misidentification. Third, the segmentation method needs to address the significant imbalance between foreground and background. Finally, due to the diversity of coronary artery tree morphology in different patients and the difference in local context information of different coronary arteries, accurate positioning of coronary arteries becomes complex.

[0004] Coronary artery segmentation techniques based on deep learning are mostly based on convolutional neural network models and Transformer models. Convolutional network models are limited by the receptive field of the convolution kernel, resulting in insufficient modeling ability of long-distance context information. Although the Transformer model can capture global context information well, it is weak in acquiring local information.

[0005] As the existing one kind based on SwinTransformer fusion multi-scale feature and multi-attention mechanism's medical image segmentation method abstract, provides a kind of based on SwinTransformer fusion multi-scale feature and multi-attention mechanism's medical image segmentation method.It includes S1: based on SwinTransformer network is established with multiple down-sampling layer's encoding module, to sample medical image is successively down-sampled, obtains four different scale sample image features;S2: based on the four different scale sample image features are multiplied and fused, obtain four multi-scale fused sample image features;S3: based on the attention module of combining space and channel is established with multiple up-sampling layer's decoder;Based on the four fused sample image features described above, successively up-sampling coding is carried out, and the corresponding segmentation result of sample image is obtained.

[0006] In summary, the prior art method lacks adaptability for three-dimensional image segmentation with large individual differences, so how to invent a method suitable for individuals is a technical problem that needs to be solved in the technical field. SUMMARY

[0007] The present application provides a three-dimensional multi-scale image segmentation model and an image segmentation method, application and system based thereon, which is suitable for large-scale and foreground-background imbalance images.

[0008] To achieve the above-mentioned purposes of the present application, the technical solutions adopted are as follows:

[0009] A three-dimensional multi-scale image segmentation model, comprising a Swin Transformer encoder, a large-core three-dimensional convolutional encoder, and two decoders based on a space transformer; the Swin Transformer encoder and the large-core three-dimensional convolutional encoder are respectively connected to the two decoders based on the space transformer; the encoding layers of the Swin Transformer encoder are one-to-one connected to the encoding layers of the large-core three-dimensional convolutional encoder; the output of the decoder corresponding to the Swin Transformer encoder is connected to the input of the large-core three-dimensional convolutional encoder; each decoding layer of the decoder based on the space transformer is provided with a space transformer block.

[0010] Preferably, the Swin Transformer encoder comprises four cascaded encoding layers, each of which is composed of two Swin Transformer blocks; the large-core three-dimensional convolutional encoder has four encoding layers, each of which is composed of two large-core convolutional blocks, each of which includes a 7x7 convolution, a BN normalization and a ReLU activation function, and each 7x7 convolution is re-parameterized by a spatial frequency matrix.

[0011] Further, the decoder comprises four decoding layers connected in cascade, each decoding layer being connected with the encoding layer residual of the corresponding encoder; each decoding layer of the decoder comprises an up-sampling block, a residual block, and a spatial transformer block connected in cascade.

[0012] A three-dimensional multi-scale image segmentation method based on the three-dimensional multi-scale image segmentation model comprises the following specific steps:

[0013] Obtaining a three-dimensional image dataset for training and preprocessing the same;

[0014] Cutting the preprocessed three-dimensional image dataset and inputting the same into the three-dimensional multi-scale image segmentation model for multiple iterative training;

[0015] Inputting a three-dimensional image data to be segmented into the trained three-dimensional multi-scale image segmentation model after preprocessing to perform pixel-level classification and complete automatic segmentation.

[0016] Preferably, the three-dimensional image dataset for training is obtained and preprocessed, and the specific steps are as follows:

[0017] Obtaining a three-dimensional image dataset for training and dividing the same into a training set, a validation set, and a test set at a preset ratio;

[0018] Preprocessing the obtained training set, validation set, and test set respectively; during preprocessing, all three-dimensional image volumes are resampled to a preset isotropic voxel spacing, and then three-dimensional images of different resolutions are sampled according to foreground / background random patches to obtain uniform resolution images.

[0019] Further, the three-dimensional multi-scale image segmentation model uses the sum of a soft dice coefficient loss and a cross-entropy loss as a loss function:

[0020]

[0021] wherein I represents the number of classes; J represents the number of voxels; and respectively represent the true label and the output probability of the i-th class voxel at the j-th voxel.

[0022] Further, the preprocessed three-dimensional image dataset is cut and inputted into the three-dimensional multi-scale image segmentation model for multiple iterative training, and the specific steps are as follows:

[0023] Cutting the preprocessed three-dimensional image dataset into a plurality of three-dimensional square images of a preset size;

[0024] ​​The three-dimensional block image is input into a Swin Transformer encoder to extract global information of the input image;

[0025] The feature information generated in each stage of the Swin Transformer encoder is input into a corresponding decoder based on a spatial transformer for decoding; in each decoding, the spatial information retained in the spatial path is fused through a spatial transformer block;

[0026] The decoded semantic information output by the decoder and the spatial information generated by the spatial path are fused and subjected to convolution operation to obtain an output output1; the output1 is output through a Sigmoid activation function to obtain a weight mask of the three-dimensional image;

[0027] The obtained weight mask of the three-dimensional image is multiplied with the input image at the same position, and input into a large-core three-dimensional convolution encoder to further extract local information of the coronary artery;

[0028] The feature information output by each layer of the large-core three-dimensional convolution encoder is added to the feature information output by the corresponding encoding layer of the Swin Transformer encoder at the same position, and then input into a corresponding decoder based on a spatial transformer for decoding; in each decoding, the spatial information retained in the spatial path is fused through a spatial transformer block;

[0029] The decoded semantic information and the spatial information generated by the spatial path are fused and subjected to convolution operation to obtain an output output2;

[0030] The output2 and the output1 are fused by the same position addition, and the fused feature information is output as a three-dimensional image segmentation result through a segmentation head;

[0031] Based on the obtained three-dimensional image segmentation result, a three-dimensional multi-scale image segmentation model is iteratively trained until a preset end iteration condition is reached.

[0032] Further, after the input feature information X is input into the spatial transformer block, X is divided into output x1 and output x2 through 1x1x1 convolution operation; output x1 is subjected to convolution operation according to H, W and D directions respectively to generate respective static context information representations x H , x W , x D ; after x H , x W , x D are fused, two 1x1 convolution operations are performed to dynamically learn a multi-head attention matrix; the obtained output feature x oMultiplication in the same position with x2 obtains a dynamic context representation xo2 based on different spatial state bases, and xo2 is added to X in the same position to obtain the output y.

[0033] An application of the image segmentation method based on the three-dimensional multi-scale cascade, which is used for segmentation of three-dimensional coronary artery CT images.

[0034] A three-dimensional multi-scale image segmentation system for implementing the image segmentation method, comprising an image processing module, a model training module and an automatic segmentation module.

[0035] The image processing module is used to obtain and preprocess three-dimensional image data sets for training and segmentation.

[0036] The model training module is used to crop and input the preprocessed three-dimensional image data sets into the three-dimensional multi-scale image segmentation model for multiple iterations of training.

[0037] The automatic segmentation module is used to input the preprocessed image into the trained three-dimensional multi-scale image segmentation model for pixel-level classification to complete automatic segmentation.

[0038] The beneficial effects of the present application are as follows:

[0039] The present application provides a three-dimensional multi-scale image segmentation model, which realizes long-distance modeling of context information of three-dimensional images through a Swin Transformer encoder, extracts local information of three-dimensional images through a large-core three-dimensional convolutional encoder, and obtains spatial information of input data images through a decoder based on a spatial transformer, thereby effectively improving the segmentation effect of three-dimensional images. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a schematic diagram of the three-dimensional multi-scale image segmentation model of the present application.

[0041] Figure 2 is a schematic diagram of the Swin Transformer encoder in the three-dimensional multi-scale image segmentation model of the present application.

[0042] Figure 3 is a schematic diagram of the large-core three-dimensional convolutional encoder in the three-dimensional multi-scale image segmentation model of the present application.

[0043] Figure 4 is a schematic diagram of the three-dimensional multi-scale image segmentation method flow of the present application.

[0044] Figure 5 is a specific flowchart of the flow of the three-dimensional multi-scale image segmentation method of the present application in Example 3.

[0045] Figure 6 is a spatial transformer based decoder of the three-dimensional multi-scale image segmentation method of the application in embodiment 3.

[0046] Figure 7 is a flowchart of the spatial transformer in embodiment 3.

[0047] Figure 8 is a schematic diagram of coronary CT, representing coronary artery true label, representing coronary artery predicted label in embodiment 4. DETAILED DESCRIPTION

[0048] The application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Embodiment 1

[0050] As shown in Figure 1 , a three-dimensional multi-scale image segmentation model, characterized by comprising a Swin Transformer encoder, a large-core three-dimensional convolutional encoder, and two spatial transformer based decoders; the Swin Transformer encoder and the large-core three-dimensional convolutional encoder are respectively connected to the two spatial transformer based decoders; the encoding layers of the Swin Transformer encoder are one-to-one connected to the encoding layers of the large-core three-dimensional convolutional encoder; the output of the decoder corresponding to the Swin Transformer encoder is connected to the input of the large-core three-dimensional convolutional encoder; each decoding layer of the spatial transformer based decoder is provided with a spatial transformer block.

[0051] In a specific implementation, the Swin Transformer encoder comprises four cascaded encoding layers, each of which is composed of two Swin Transformer blocks;

[0052] In this embodiment, as shown in Figure 2 , the Swin Transformer block comprises a first layer normalization layer LN, a window multi-head self-attention layer W-MSA, and the output of the W-MSA is connected to the input of the LN in residual error, and then the output is input to the second LN and a normalization and multi-layer perceptron MLP which are cascaded, and the output of the MLP is connected to the input of the second LN in residual error and then output.

[0053] In this embodiment, the patch size of the Swin Transformer encoder is 2 × 2 × 2, and the feature dimension is 2 × 2 × 2 × 1 = 8. The embedding space C in the Swin Transformer encoder is set to 48. To maintain the hierarchical structure of the encoder, a patch merging layer is used at the end of each encoding layer to reduce the resolution of the feature representation by a factor of 2. Furthermore, the patch merging layer groups patches with a resolution of 2 × 2 × 2 and concatenates them to produce a 4C feature embedding, which is then reduced to 2C using a linear layer.

[0054] In this embodiment, as Figure 3 As shown, the large-kernel 3D convolutional encoder has four coding layers, each consisting of two large-kernel convolutional blocks. Each large-kernel convolutional block includes a 7x7 convolution, BN normalization, and a ReLU activation function, and each 7x7 convolution is reparameterized using a spatial frequency matrix. With each coding layer, the feature image resolution is halved, while the number of channels doubles. The spatial frequency matrix is ​​obtained based on requirements through Fourier transform or other frequency domain transformations, representing the convolution kernel in the frequency domain, effectively capturing different frequency components of the input signal. During reparameterization, the original 7x7 convolution kernel is converted into frequency form, and new convolution kernels are constructed using these frequency components. These frequency domain information is then used to recombine the new convolution kernels. The reparameterization process essentially involves manipulating information in the frequency domain and then transforming it back to the spatial domain to obtain the convolution kernel used for convolution. Reparameterization can make convolution calculations in the frequency domain more efficient than direct calculations in the spatial domain in some cases, especially when dealing with large convolution kernels. Reparameterization can help reduce computation and speed up inference. By processing spatial frequency, it can better capture periodicity and texture information in the data, enabling convolutional layers to learn richer features. Operating convolutional kernels in the frequency domain allows for more flexible variations. Furthermore, the reparameterized convolutional kernels can adaptively adjust to different input features, thereby improving the model's robustness. Using frequency components to reparameterize convolutional kernels can help the model focus on more regular features rather than local noise, potentially reducing the risk of overfitting to some extent.

[0055] In a specific implementation, such as Figure 4 As shown, the decoder includes four cascaded decoding stages. i Each decoding layer is residually connected to the encoding layer of its corresponding encoder; each decoding layer of the decoder includes cascaded upsampling blocks, residual blocks, and spatial converter blocks.

[0056] Example 2

[0057] likeFigure 5 As shown, a three-dimensional multi-scale image segmentation method based on the aforementioned three-dimensional multi-scale image segmentation model is characterized by the following specific steps:

[0058] Obtain the 3D image dataset for training and preprocess it;

[0059] The preprocessed 3D image dataset is cropped and input into a 3D multi-scale image segmentation model for multiple iterative training steps.

[0060] The 3D image data to be segmented is preprocessed and then input into a pre-trained 3D multi-scale image segmentation model for pixel-level classification, thus completing automatic segmentation.

[0061] In one specific implementation, the 3D image dataset used for training is acquired and preprocessed, with the following steps:

[0062] Obtain the 3D image dataset for training and divide it into training set, validation set and test set according to a preset ratio;

[0063] The obtained training set, validation set and test set are preprocessed respectively. During preprocessing, all 3D image volumes are resampled into the preset isotropic voxel spacing. Then, 3D images of different resolutions are randomly sampled into images of uniform size and resolution by random foreground / background patches.

[0064] In this embodiment, all 3D image volumes are resampled to an isotropic voxel spacing of 0.5 mm², and the intensity of all non-zero voxels in the image is normalized to the range of [0,1]. 3D coronary CT images of different resolutions are randomly sampled at a ratio of 1:1 using random foreground / background patches to obtain uniform-sized [128, 128, 128] resolution images;

[0065] In one specific implementation, the aforementioned three-dimensional multi-scale image segmentation model uses the sum of soft Dice coefficient loss and cross-entropy loss as the loss function:

[0066]

[0067] Where I represents the number of classes; J represents the number of voxels; and They represent the first voxels The true label and output probability at the given location.

[0068] In a specific implementation, such as Figure 6 As shown, the preprocessed 3D image dataset is cropped and input into the 3D multi-scale image segmentation model for multiple iterative training steps. The specific steps are as follows:

[0069] cropping the pre-processed three-dimensional image dataset into a plurality of three-dimensional square images of a preset size;

[0070] inputting the three-dimensional square images into a Swin Transformer encoder to extract features of global information of the input images;

[0071] inputting the feature information generated in each stage of the Swin Transformer encoder into a corresponding decoder based on a spatial transformer for decoding; at each decoding, the spatial information retained in the spatial path is fused through a spatial transformer block;

[0072] fusing the decoded semantic information output by the decoder and the spatial information generated by the spatial path and performing convolution operation to obtain an output output1; performing 1x1 convolution, Sigmoid activation function and residual block on the output1 to output a weight mask of the three-dimensional image;

[0073] multiplying the obtained weight mask of the three-dimensional image with the input image at the same position, inputting into a large-core three-dimensional convolutional encoder to further extract local information of the coronary artery;

[0074] adding the feature information output by each layer of the large-core three-dimensional convolutional encoder and the feature output by the corresponding encoding layer of the Swin Transformer encoder at the same position, and inputting into a corresponding decoder based on a spatial transformer for decoding; at each decoding, the spatial information retained in the spatial path is fused through a spatial transformer block;

[0075] In this embodiment, the decoded semantic information and the spatial information generated by the spatial path are fused and convolution operation is performed to obtain an output output2;

[0076] fusing the output2 and the output1 by the same position addition, and outputting a segmentation result with a channel number of 2 by using 1x1 convolution operation after the fused feature information passes through a residual block and a segmentation head head;

[0077] iteratively training the three-dimensional multi-scale image segmentation model based on the obtained three-dimensional image segmentation result until a preset end iteration condition is reached.

[0078] In this embodiment, the output resolution of each encoding layer of the Swin transformer encoder and the large-core three-dimensional convolutional encoder is shown in Table 1:

[0079] Table 1

[0080]

[0081] In one specific implementation, after the input feature information X is input to the space converter block, X is divided into output x1 and output x2 through 1x1x1 convolution operation; output x1 is subjected to convolution operation according to H, W and D three directions respectively, to generate respective static context information representation x H W D ; x H W D is fused and then passed through two 1x1 convolution operations to dynamically learn a multi-head attention matrix; the obtained output feature x o is multiplied with x2 at the same position to obtain dynamic context representation xo2 based on different spatial state bases, and xo2 is added with X at the same position to obtain output y.

[0082] In this embodiment, all three-dimensional image volumes are resampled to an isotropic voxel spacing of 0.5 mm2, and all non-zero voxel intensities of the image are normalized to the range of [0, 1]. The three-dimensional images of different resolutions are sampled according to the random patches of foreground / background at a ratio of 1:1, and the random sampling is a uniform size [128, 128, 128] resolution image; the residual block is introduced to retain the spatial information of the initial three-dimensional image; the SwinTransformer in the cascaded encoder models the long-distance context global information of the three-dimensional image, and further fuses the spatial information retained in the spatial path through the decoder based on the space converter module to preliminarily generate the weight mask of the three-dimensional image. The product of the weight mask and the three-dimensional image is taken as the input of the large-core three-dimensional convolution encoder in the cascaded encoder, the local information is obtained on the basis of long-distance modeling, the large-core convolution operation is re-parameterized through the spatial frequency matrix to improve the local detail capture capability, and the feature encoding of the Swin Transformer is fused and sent to the decoder for decoding; the feature representations of the two decodings of the cascaded network and the spatial information retained in the spatial path are fused, and the final three-dimensional image segmentation result is obtained through the segmentation head prediction. Thus, through the establishment of the three-dimensional multi-scale cascaded network model, the three-dimensional image is subjected to single-stage and end-to-end segmentation, and high segmentation accuracy and stable segmentation performance are achieved.

[0083] Embodiment 3

[0084] An application of the three-dimensional multi-scale cascaded image segmentation method, which is used for segmentation of three-dimensional coronary artery CT images.

[0085] ​​​​In this embodiment, the data for training the model comes from real clinical cases of Guangdong General Hospital, consisting of 3D CTA images collected by Siemens 128-slice dual-source scanner from 1000 patients with coronary artery disease. The size of the CTA image is 512x512x(206-275) voxels, the plane resolution is 0.29 mm2-0.43 mm2, and the interval is 0.25 mm-0.45 mm. The patients collected are 414 females and 586 males, with an average age of 59.98 years and 57.68 years respectively. Patients aged 18 years or older, with a history of ischemic stroke, transient ischemic attack or peripheral arterial disease meet the inclusion criteria. Patients with low CTA imaging quality (assessed by three radiologists) that may affect coronary artery function are excluded. The left and right coronary arteries in each image are independently labeled by two radiologists, and the results are cross-validated. If there is a difference, a third radiologist will annotate, and the final annotation is based on consensus.

[0086] In this embodiment, after obtaining the three-dimensional image data set for training, the data set is divided into training set, validation set and test set according to the ratio of 8:1:1. The epoch setting for all model training is 50 rounds (about 40000 iterations), the training batch size is 1, the learning rate is set to 0.0001, the Adamw optimizer is used, and the entire training process uses linear preheating and cosine annealing learning rate scheduler training, and DiceCELoss is used as the loss function. For adjacent voxels, a sliding window method with an overlap of 0.5 is used, and the evaluation index of the final prediction result mainly uses Dice similarity coefficient evaluation, HD95 Hausdorff distance, recall rate and precision rate.

[0087] In this embodiment, the average dice coefficient reaches 81.24, and the Hausdorff distance HD95 is 14.28. The three-dimensional coronary artery segmentation results are as shown in Figure 8 , wherein from left to right are coronary artery CT, coronary artery true label, and coronary artery predicted label.

[0088] In this embodiment, the performance of the model on the test set is compared with other segmentation methods. The compared methods are common 3D medical image segmentation models, including 3D Unet based on convolutional neural network, SegResNet, 3DUXNET and nnFormer based on Transformer architecture, UNETR, SwinUNETR, etc. The comparison results are shown in Table 2:

[0089] Table 2

[0090]

[0091] In summary, the present application obtains three-dimensional coronary artery CT three-dimensional image data set and pre-processes; establishes a multi-scale cascaded coronary artery segmentation model based on Swin Transformer and large-core three-dimensional convolution; cuts the pre-processed coronary artery CT three-dimensional image data set into a plurality of three-dimensional blocks of a preset size, and inputs all the three-dimensional blocks into the multi-scale cascaded coronary artery segmentation model for multiple iterative training; inputs the coronary artery CT three-dimensional image to be segmented into the trained multi-scale cascaded coronary artery segmentation model for pixel-level classification, and completes automatic segmentation of the three-dimensional coronary artery; the present application proposes a multi-scale cascaded encoding mode based on Swin Transformer and large-core three-dimensional convolution for global feature and local detail feature extraction in encoding and decoding, and introduces a 3D space converter module in the decoder, which can extract correlation information between different spatial states, effectively improve the decoding ability and visual representation ability of the decoder, and improve the accuracy of three-dimensional coronary artery segmentation. Compared with the prior art, the present application can overcome the limitations caused by the dependence on artificial intervention, the foreground-background imbalance of the coronary artery image, the large individual difference, the complex and tiny blood vessel structure, and the like.

[0092] Example 4

[0093] A three-dimensional multi-scale image segmentation system for implementing the image segmentation method comprises an image processing module, a model training module, and an automatic segmentation module.

[0094] The image processing module is used to obtain and pre-process three-dimensional image data sets for training and segmentation.

[0095] The model training module is used to cut the pre-processed three-dimensional image data set and input it into the three-dimensional multi-scale image segmentation model for multiple iterative training.

[0096] The automatic segmentation module is used to input the pre-processed image into the trained three-dimensional multi-scale image segmentation model for pixel-level classification and complete automatic segmentation.

[0097] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. Any modification, equivalent replacement, and improvement within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A three-dimensional multi-scale image segmentation method based on a three-dimensional multi-scale image segmentation model, characterized in that: The three-dimensional multi-scale image segmentation model comprises a Swin Transformer encoder, a large-core three-dimensional convolutional encoder, and two decoders based on a space transformer; the Swin Transformer encoder and the large-core three-dimensional convolutional encoder are respectively connected to the two decoders based on the space transformer; the encoding layers of the Swin Transformer encoder are connected to the encoding layers of the large-core three-dimensional convolutional encoder one by one; the output of the decoder corresponding to the Swin Transformer encoder is connected to the input of the large-core three-dimensional convolutional encoder; each decoding layer of the decoder based on the space transformer is provided with a space transformer block; The method comprises the following specific steps: obtaining a three-dimensional image dataset for training and preprocessing; cropping the preprocessed three-dimensional image dataset and inputting it into a three-dimensional multi-scale image segmentation model for multiple iterations of training, and the specific steps are as follows: cropping the preprocessed three-dimensional image dataset into a plurality of three-dimensional square images of a preset size; inputting the three-dimensional square images into the Swin Transformer encoder to extract features of global information of the input images; inputting the feature information generated in each stage of the Swin Transformer encoder into the corresponding decoder based on the space transformer for decoding; in each decoding, the space information retained in the space path is fused through the space transformer block; fusing the decoded semantic information output by the decoder and the space information generated by the space path and performing convolution operation to obtain output1; outputting the weight mask of the three-dimensional image through the Sigmoid activation function of output1; multiplying the obtained weight mask of the three-dimensional image with the input image at the same position, inputting it into the large-core three-dimensional convolutional encoder, and further extracting local information of the coronary artery; adding the feature information output by each layer of the encoder in the large-core three-dimensional convolutional encoder to the feature output by the corresponding encoding layer of the Swin Transformer encoder at the same position, and inputting it into the corresponding decoder based on the space transformer for decoding; in each decoding, the space information retained in the space path is fused through the space transformer block; fusing the decoded semantic information and the space information generated by the space path and performing convolution operation to obtain output2; fusing output2 and output1 by the same position addition method, and outputting the three-dimensional image segmentation result through a segmentation head after the fused feature information; iteratively training the three-dimensional multi-scale image segmentation model based on the obtained three-dimensional image segmentation result until a preset end iteration condition is reached; inputting the three-dimensional image data to be segmented into the trained three-dimensional multi-scale image segmentation model after preprocessing to perform pixel-level classification and complete automatic segmentation.

2. The method of claim 1, wherein: The Swin Transformer encoder includes 4 encoding layers in cascade, each of which is composed of 2 Swin Transformer blocks; the large-core three-dimensional convolutional encoder has 4 encoding layers, each of which is composed of 2 large-core convolutional blocks, each of which includes a 7x7 convolution, BN normalization, and a ReLU activation function, and each 7x7 convolution is re-parameterized by a spatial frequency matrix.

3. The method of claim 2, wherein: The decoder includes 4 decoding layers in cascade, each of which is connected in residual connection with the encoding layer of the corresponding encoder; each decoding layer of the decoder includes cascaded upsampling blocks, residual blocks, and spatial transformer blocks.

4. The method of claim 3, wherein: A three-dimensional image dataset for training is obtained and preprocessed, and the specific steps are as follows: A three-dimensional image dataset for training is obtained, and is divided into a training set, a validation set and a test set at a preset ratio. The obtained training set, validation set and test set are preprocessed respectively; during preprocessing, all three-dimensional image volumes are resampled to a preset isotropic voxel spacing, and then three-dimensional images of different resolutions are sampled according to foreground / background random patches, and random sampling is a resolution image of uniform size.

5. The method of claim 4, wherein: The three-dimensional multi-scale image segmentation model uses the sum of a soft dice coefficient loss and a cross-entropy loss as a loss function: where I denotes the number of classes; J denotes the number of voxels; and denote the true label and output probability at the class voxel respectively.

6. The method of claim 1, wherein: After inputting the input feature information X into the space converter block, X is divided into output x1 and output x2 through 1x1x1 convolution operation; output x1 is subjected to convolution operation according to three directions of H, W and D to generate respective static context information representations x H , x W , x D ; after fusing x H , x W , x D , the dynamic multi-head attention matrix is dynamically learned through two 1x1 convolution operations once; the obtained output feature x o is multiplied with x2 at the same position to obtain a dynamic context representation xo2 based on different space state bases, and xo2 is added with X at the same position to obtain the output y.

7. Use of a method for segmenting a three-dimensional multiscale image according to any one of claims 1 to 6, characterized in that: The method is used for segmentation of three-dimensional coronary CT images.

8. A three-dimensional multi-scale image segmentation system for implementing the image segmentation method of any one of claims 1 to 6, characterized in that: It comprises an image processing module, a model training module and an automatic segmentation module. The image processing module is used to obtain a three-dimensional image dataset for training and a three-dimensional image data to be segmented and pre-process them. The model training module is used to crop the pre-processed three-dimensional image dataset for training and input it into a three-dimensional multi-scale image segmentation model for multiple iterations of training. The automatic segmentation module is used to input the pre-processed three-dimensional image data to be segmented into the trained three-dimensional multi-scale image segmentation model for pixel-level classification to complete automatic segmentation.

Images