Cerebral vascular segmentation method and neural network segmentation device based on multi-center TOF-MRA images

Abstract

The present invention relates to a cerebral vascular segmentation method and a neural network segmentation model based on multi-center TOF-MRA images. By constructing a gold standard database, the problems of poor generalization ability and low robustness of existing cerebral vascular segmentation methods are alleviated. By establishing a signed distance field of cerebral vessels in 3D space and calculating the cost function between the cerebral vascular segmentation signed distance field and the gold standard signed distance field, the robustness of cerebral vascular segmentation from TOF-MRA images is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a cerebral blood vessel segmentation method and a neural network segmentation device based on multi-center TOF-MRA images. Background Art

[0002] Cerebral arterial vascular assessment is crucial to the diagnostic process for patients with cerebral vascular diseases. Stroke is one of the leading causes of death worldwide, with 15 million people suffering from stroke each year. Stroke is a cerebrovascular disease characterized by changes in the cerebral arteries, such as stenosis and occlusion. Studies have shown that morphological information of arterial vessels is crucial to the diagnostic process. In chronic cerebrovascular diseases, cerebral vascular morphological parameters can be used as a biomarker. In acute clinical settings, cerebral vascular morphological parameters provide key auxiliary information for stroke physicians. For example, these parameters can help physicians diagnose acute arterial occlusive diseases in advance, thereby allowing their patients to undergo mechanical thrombectomy, which is currently the best stroke reperfusion therapy.

[0003] Magnetic resonance imaging (MRA) time-of-flight (TOF) imaging is a non-invasive neuroimaging technology used in routine clinical practice to model arteries and capture the morphology of intracranial cerebral blood vessels. However, these imaging techniques can only visually evaluate blood vessels, and the precise quantification of cerebral arteries relies solely on the operator's clinical experience and skill familiarity, which largely introduces a large amount of uncertainty into the quantitative analysis of cerebral arteries. In addition, the analysis of brain function and structure requires a large amount of cerebral artery data as support. However, relying on manual visual reconstruction and analysis of cerebral arteries requires a lot of laborious work. Even though TOF MRA imaging is fast, does not involve ionizing radiation exposure, and can characterize arterial vessels in high detail, the lack of fully automatic vascular segmentation methods means that clinical interpretation of vascular imaging is based solely on visual judgment. Extensive, time-consuming, and non-standardized image post-processing is a major obstacle to widespread clinical application. Summary of the Invention

[0004] The problem solved by the present invention is how to make the cerebral blood vessel segmentation algorithm obtain better generalization problem, so as to realize the automatic segmentation of cerebral blood vessels, thereby promoting the identification of valuable biomarkers related to cerebrovascular diseases.

[0005] To solve the above problems, the present invention provides a cerebral vascular segmentation method based on multi-center TOF-MRA images, comprising:

[0006] Step 1: A CNN encoder consisting of four layers of residual convolutional modules is used to extract cerebral vascular feature maps from TOF-MRA images;

[0007] Step 2: Dimensionally map the cerebrovascular feature map extracted by the last two layers of the residual convolution module of the CNN encoder; stretch the dimensionally mapped cerebrovascular feature map to form two one-dimensional feature vectors; then concatenate the two one-dimensional feature vectors into a one-dimensional feature vector rich in voxel information;

[0008] Step 3: Position encoding is performed on the cerebrovascular feature map extracted by the last two layers of residual convolution modules of the CNN encoder to restore the spatial position information of the cerebrovascular feature map;

[0009] Step 4: Add the position code and the one-dimensional feature vector pixel by pixel and input them into the Transformer neural network to extract the long-range dependency feature vector of the cerebral blood vessels;

[0010] Step 5: Split, upsample, and reshape the long-range feature vector to obtain a long-range feature map;

[0011] Step 6: Set up a CNN decoder. The CNN decoder has four residual convolution layers corresponding to the CNN encoder. The CNN decoder processes the long-distance feature map to obtain the cerebrovascular structure.

[0012] The beneficial effect of the present invention is: by integrating the Transformer neural network to fully explore the long-distance dependency features between different frames in TOF-MRA images, thereby adapting to the ability to extract cerebral vascular features from TOF-MRA images taken by different devices, and improving the generalization ability of the cerebral vascular segmentation algorithm.

[0013] Preferably, the formula for position encoding the cerebral vascular characteristic map in step 3 is expressed as:

[0014] ;

[0015] Where, Indicates the position of the feature point in the cerebral vascular feature map, Represents the dimensions of the last two layers of residual convolution modules of the CNN encoder after dimension mapping.

[0016] Preferably, the cerebral blood vessel segmentation method further includes a training method for performing TOF-MRA image segmentation on steps 1 to 6, specifically comprising:

[0017] Step 7: Obtain TOF-MRA images captured by multiple centers and multiple devices, perform pre-segmentation on the TOF-MRA images, and establish a gold standard dataset for cerebral vascular segmentation training;

[0018] Step 8: The cerebral vascular structure in step 6 and the corresponding pre-segmented cerebral vascular structure in the gold standard dataset are transformed using a signed distance function to obtain a cerebral vascular segmentation signed distance field and a gold standard signed distance field;

[0019] Step 9: Calculate the error loss between the cerebral vascular segmentation signed distance field and the gold standard signed distance field using a mean square error function;

[0020] By constructing a gold standard database, the problems of poor generalization ability and low robustness of existing cerebral vascular segmentation methods are alleviated; and by establishing a signed distance field of cerebral vessels in 3D space, the robustness of cerebral vascular segmentation from TOF-MRA images is further improved by calculating the cost function between the signed distance field of cerebral vascular segmentation and the gold standard signed distance field.

[0021] Preferably, the pre-segmentation process in step 7 specifically includes:

[0022] Step 701: Use a local threshold method to normalize the intensity value of the TOF-MRA image and remove the skull, segment the aorta vessel labels, and delete the incorrectly segmented voxel blocks;

[0023] Step 702: The aorta labels generated in step 701 are segmented again through steps 1 to 6, and incorrectly segmented voxel blocks are deleted. A determination is then made as to whether the segmented cerebral vascular voxel blocks reach 0.3 mm. If so, the process proceeds to step 703; otherwise, the process returns to step 701.

[0024] Step 703: The cerebral vessels larger than 0.3 mm in the marked portion of the cerebral vascular structure generated in step 702 are refined, and the broken cerebral vessels are outlined to establish a gold standard data set.

[0025] Preferably, the cerebral vascular segmentation method also includes a surface modeling method based on the cerebral vascular structure, specifically including: the cerebral vascular structure obtained in step 6 is a preliminary representation of the position information of the cerebral vascular voxels in three-dimensional space, the marching cubes algorithm is used to perform voxel rendering on the cerebral vascular structure to generate cerebral vascular surface mesh data, and the surface is smoothed to obtain a cerebral vascular smooth surface mesh for assisting in the calculation of cerebral vascular morphological parameters.

[0026] A neural network segmentation device for implementing the multi-center TOF-MRA image cerebral blood vessel segmentation method includes:

[0027] CNN encoder: It consists of four layers of residual convolution modules. Each residual convolution module includes two residual convolution groups. Each residual convolution group includes a 3×3×3 convolution layer, a normalization layer, and a ReLU activation layer. Each residual convolution module is connected to a pooling layer that reduces the resolution of the cerebrovascular feature map by half.

[0028] Dimensional mapping layer: used to perform dimension mapping on the cerebrovascular feature maps of the last two layers of residual convolution modules of the CNN encoder to form two one-dimensional feature vectors; and then concatenate the two one-dimensional feature vectors into a one-dimensional feature vector rich in voxel information;

[0029] Position encoder: used to perform position encoding on the cerebrovascular feature map and restore the spatial structure of the cerebrovascular features;

[0030] Transformer neural network, used to improve the extraction of long-range dependency features of cerebral blood vessels at different spatial locations through the Transformer attention mechanism;

[0031] Reshaping layer: It is used to split the long-distance dependency feature vector of cerebral blood vessels extracted by the Transformer neural network into two feature vectors according to the dimensions before mapping by the dimension mapping layer, and reshape it after upsampling to obtain a long-distance feature map;

[0032] CNN decoder: used to restore the resolution of cerebral vascular features. The CNN decoder includes four layers of residual convolution modules. The last residual convolution module of the CNN decoder includes two groups of residual convolution groups, one of which includes a 1×1×1 convolution layer, a normalization layer, and a ReLU activation layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the process of the specific embodiment 1 of the present invention;

[0034] Figure 2 A schematic diagram of the cerebral blood vessel signed distance field and a real simulation effect diagram in specific embodiment 2 of the present invention;

[0035] Figure 3 This is the cerebral blood vessel smooth surface mesh data model of specific embodiment 3 of the present invention. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiment 1:

[0038] Brain vessel segmentation methods based on multi-center TOF-MRA images, such as Figure 1 Shown, including:

[0039] Step 1: A CNN encoder composed of four layers of residual convolution modules is used to extract a cerebral vascular feature map from the TOF-MRA image to obtain high-dimensional semantic information of the cerebral vessels. In this specific embodiment, the size of the cerebral vascular feature map output by the CNN encoder is ,in, Indicates the number of dimensions of the feature, , and Respectively represent the length, width and height (layer) of the feature map. In this specific embodiment, the CNN encoder output feature size is: 8×8×4×512;

[0040] Step 2: Dimension mapping is performed on the cerebrovascular feature maps extracted by the last two layers of the residual convolution modules of the CNN encoder. In this specific embodiment, two cerebrovascular feature maps with dimensions of 16×16×8×240 and 8×8×4×240 are obtained after dimension mapping. The dimensionally mapped cerebrovascular feature maps are then stretched to form two one-dimensional feature vectors. The two one-dimensional feature vectors are then concatenated into a one-dimensional feature vector rich in voxel information. However, because the stretching operation causes the cerebrovascular feature map to lose spatial position information, the process proceeds to step 3.

[0041] Step 3: Position encoding is performed on the cerebral vascular feature map extracted by the last two layers of the residual convolution module of the CNN encoder to restore the spatial position information of the cerebral vascular feature map; the position encoding formula is expressed as:

[0042] ;

[0043] Where, Indicates the position of the feature point in the cerebral vascular feature map, Represents the dimensions of the last two layers of residual convolution modules of the CNN encoder after dimension mapping;

[0044] Step 4: The position code and the one-dimensional feature vector are added pixel by pixel, and then input into the Transformer neural network to extract the long-range dependency feature vector of the cerebral blood vessels. The Transformer neural network in this specific embodiment is prior art and will not be described in detail here. After feature extraction through position coding, the cerebral vascular feature map becomes a high-dimensional abstract semantic feature. The Transformer neural network can establish the correlation between the various semantics in the high-dimensional semantic features, thereby activating the cerebral vascular feature values ​​while suppressing the interference of background features.

[0045] Step 5: Split, upsample, and reshape the long-range feature vector to obtain a long-range feature map;

[0046] Step 6. Because the two feature vectors output by the Transformer neural network can only represent high-level abstract information of cerebral blood vessels, but cannot accurately reflect the specific voxel distribution of cerebral blood vessels in TOF-MRA images, this specific embodiment sets a CNN decoder to restore the extracted cerebral vascular features to the corresponding positions in the TOF-MRA image, and gradually restore the resolution of the cerebral vascular features. The CNN decoder is provided with four layers of residual convolution layers with the same structure as the CNN encoder but different sizes, so that the CNN decoding can decode cerebral vascular voxel information at different scales; at the same time, in order to further compensate for the spatial position information lost due to the pooling layer, the cerebral vascular feature maps of the corresponding layers of the CNN encoder and CNN decoder in this specific embodiment are stacked; the CNN decoder processes the long-distance feature map to obtain the cerebral vascular structure. Specific embodiment 2:

[0048] This specific embodiment is based on specific embodiment 1. The cerebral blood vessel segmentation method further includes a training method for TOF-MRA image segmentation in steps 1 to 6, specifically including:

[0049] Step 7: Obtain TOF-MRA images taken by multiple different brands of equipment from different R&D centers, perform pre-segmentation on the TOF-MRA images, and establish a gold standard dataset for cerebral vascular segmentation training; this includes:

[0050] Step 701: Use the local threshold method to normalize the intensity value of the TOF-MRA image and remove the skull, segment the aorta vessel labels, and manually delete the incorrectly segmented voxel blocks;

[0051] Step 702: The aorta labels generated in step 701 are segmented again through steps 1 to 6, and incorrectly segmented voxel blocks are manually deleted. A determination is then made as to whether the segmented cerebral vascular voxel blocks reach 0.3 mm. If so, the process proceeds to step 703; otherwise, the process returns to step 701.

[0052] Step 703: The cerebral vessels with a diameter of more than 0.3 mm in the marked position in the cerebral vascular structure generated in step 702 are finely delineated, as well as the broken cerebral vessels, to establish a gold standard data set;

[0053] Step 8: The cerebral vascular structure in step 6 and the corresponding pre-segmented cerebral vascular structure in the gold standard dataset are transformed using the signed distance function to obtain the cerebral vascular segmentation signed distance field and the gold standard signed distance field; Figure 2 As shown, in this specific embodiment, the cerebral blood vessel wall is regarded as the boundary, the distance inside the cerebral blood vessel is a negative number, and the distance on the cerebral blood vessel wall is 0;

[0054] Step 9: Using the mean square error function to calculate the error loss between the cerebral blood vessel segmentation signed distance field and the gold standard signed distance field, thereby achieving accurate segmentation of the cerebral blood vessel wall by predicting the signed distance field of the cerebral blood vessels;

[0055] By constructing a gold standard database, the problems of poor generalization and low robustness of existing cerebral vascular segmentation methods are alleviated; and by establishing a signed distance field of cerebral blood vessels in 3D space, the cerebral vascular voxel classification problem is transformed into a signed distance field regression problem, enhancing the ability of the segmentation model to extract the universal characteristics of cerebral blood vessels, thereby achieving a segmentation method with stronger generalization ability. Specific embodiment 3:

[0057] This specific embodiment is based on specific embodiment 1 or specific embodiment 2 and adds the following:

[0058] The cerebral vascular segmentation method also includes a surface modeling method based on the cerebral vascular structure, specifically including: the cerebral vascular structure obtained in step 6 is a preliminary representation of the position information of the cerebral vascular voxels in the three-dimensional space, and cannot fully represent the topological structure information of the cerebral vessels. In order to facilitate the subsequent analysis of the cerebral vascular topological structure information, Figure 3 As shown, this specific embodiment uses the marching cubes algorithm to perform voxel rendering on the cerebral vascular structure to generate cerebral vascular surface mesh data, and performs surface smoothing processing to obtain a cerebral vascular smooth surface mesh for assisting in the calculation of cerebral vascular morphological parameters. The morphological parameters of this specific embodiment include vascular centerline, vascular radius, vascular bifurcation point, etc. Specific embodiment 4:

[0060] A neural network segmentation device for implementing the multi-center TOF-MRA image cerebral blood vessel segmentation method, comprising:

[0061] CNN encoder: It consists of four layers of residual convolution modules. Each residual convolution module includes two residual convolution groups. Each residual convolution group includes a 3×3×3 convolution layer, a normalization layer, and a ReLU activation layer. Each residual convolution module is connected to a pooling layer that reduces the resolution of the cerebrovascular feature map by half. While reducing the resolution of the extracted cerebrovascular feature map by half, it also reduces the computational complexity and obtains high-order semantic features of the cerebrovascular.

[0062] Dimensional mapping layer: used to perform dimension mapping on the cerebrovascular feature maps of the last two layers of residual convolution modules of the CNN encoder to form two one-dimensional feature vectors; and then concatenate the two one-dimensional feature vectors into a one-dimensional feature vector rich in voxel information;

[0063] Position encoder: used to perform position encoding on the cerebrovascular feature map and restore the spatial structure of the cerebrovascular features;

[0064] Transformer neural network, used to improve the extraction of long-range dependency features of cerebral blood vessels at different spatial locations through the Transformer attention mechanism;

[0065] Reshaping layer: It is used to split the long-distance dependency feature vector of cerebral blood vessels extracted by the Transformer neural network into two feature vectors according to the dimensions before mapping by the dimension mapping layer, and reshape it after upsampling to obtain a long-distance feature map;

[0066] CNN decoder: used to restore the resolution of cerebral vascular features. The CNN decoder includes a four-layer residual convolution module. The last layer of the CNN decoder residual convolution module includes two groups of residual convolution groups, one of which includes a 1×1×1 convolution layer, a normalization layer, and a ReLU activation layer. The 1×1×1 convolution layer is set to reduce the 32-dimensional cerebral vascular feature map to 2 dimensions to represent the cerebral vascular voxels in the TOF-MRA image. In this specific embodiment, the 0th dimension represents the background voxels other than the cerebral blood vessels, and the 1st dimension represents the cerebral vascular voxels.

[0067] Although the present disclosure is disclosed as above, the protection scope of the present disclosure is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A cerebral vascular segmentation method based on multi-center TOF-MRA images, characterized in that: include: Step 1: A CNN encoder consisting of four layers of residual convolutional modules is used to extract cerebral vascular feature maps from TOF-MRA images; Step 2: Dimensionally map the cerebrovascular feature map extracted by the last two layers of the residual convolution module of the CNN encoder; stretch the dimensionally mapped cerebrovascular feature map to form two one-dimensional feature vectors; then concatenate the two one-dimensional feature vectors into a one-dimensional feature vector rich in voxel information; Step 3: Position encoding is performed on the cerebrovascular feature map extracted by the last two layers of the residual convolution module of the CNN encoder to restore the spatial position information of the cerebrovascular feature map. The formula for position encoding of the cerebrovascular feature map is expressed as: ； Where, Indicates the location of the feature points in the cerebral vascular feature map, Represents the dimensions of the last two layers of residual convolution modules of the CNN encoder after dimension mapping; Step 4: Add the position code and the one-dimensional feature vector pixel by pixel and input them into the Transformer neural network to extract the long-range dependency feature vector of the cerebral blood vessels; Step 5: Split, upsample, and reshape the long-range feature vector to obtain a long-range feature map; Step 6: Setting a CNN decoder. The CNN decoder is provided with four residual convolution layers corresponding to the CNN encoder. The CNN decoder processes the long-range feature map to obtain the cerebral vascular structure. The method also includes a training method for TOF-MRA image segmentation for steps 1 to 6, specifically including: Step 7: Obtain TOF-MRA images captured by multiple centers and multiple devices, perform pre-segmentation on the TOF-MRA images, and establish a gold standard dataset for cerebral vascular segmentation training; Step 8: The cerebral vascular structure in step 6 and the corresponding pre-segmented cerebral vascular structure in the gold standard dataset are transformed using a signed distance function to obtain a cerebral vascular segmentation signed distance field and a gold standard signed distance field; Step 9: Use the mean square error function to calculate the error loss between the cerebral blood vessel segmentation signed distance field and the gold standard signed distance field.

2. The cerebral vascular segmentation method based on multi-center TOF-MRA images according to claim 1, characterized in that: The pre-segmentation process in step 7 specifically includes: Step 701: Use a local threshold method to normalize the intensity value of the TOF-MRA image and remove the skull, segment the aorta vessel labels, and delete the incorrectly segmented voxel blocks; Step 702: The aorta labels generated in step 701 are segmented again through steps 1 to 6, and incorrectly segmented voxel blocks are deleted. A determination is then made as to whether the segmented cerebral vascular voxel blocks reach 0.3 mm. If so, the process proceeds to step 703; otherwise, the process returns to step 701. Step 703: Mark the cerebral blood vessels larger than 0.3 mm in the cerebral blood vessel structure generated in step 702, and perform fine delineation on the broken cerebral blood vessels to establish a gold standard data set.

3. The cerebral vascular segmentation method based on multi-center TOF-MRA images according to claim 1, characterized in that: The cerebral vascular segmentation method also includes a surface modeling method based on the cerebral vascular structure, specifically including: the cerebral vascular structure obtained in step 6 is a preliminary representation of the position information of the cerebral vascular voxels in three-dimensional space, the cerebral vascular structure is voxel-rendered using the marching cubes algorithm to generate cerebral vascular surface mesh data, and the surface is smoothed to obtain a cerebral vascular smooth surface mesh for assisting in the calculation of cerebral vascular morphological parameters.

4. A neural network segmentation device for implementing the cerebral vascular segmentation method of multi-center TOF-MRA images according to claim 1, characterized in that: include: CNN encoder: It consists of four layers of residual convolution modules. Each residual convolution module includes two residual convolution groups. Each residual convolution group includes a 3×3×3 convolution layer, a normalization layer, and a ReLU activation layer. Each residual convolution module is connected to a pooling layer that reduces the resolution of the cerebrovascular feature map by half. Dimensional mapping layer: used to perform dimension mapping on the cerebrovascular feature maps of the last two layers of residual convolution modules of the CNN encoder to form two one-dimensional feature vectors; and then concatenate the two one-dimensional feature vectors into a one-dimensional feature vector rich in voxel information; Position encoder: used to perform position encoding on the cerebrovascular feature map and restore the spatial structure of the cerebrovascular features; Transformer neural network, used to improve the extraction of long-range dependency features of cerebral blood vessels at different spatial locations through the Transformer attention mechanism; Reshaping layer: It is used to split the long-distance dependency feature vector of cerebral blood vessels extracted by the Transformer neural network into two feature vectors according to the dimensions before mapping by the dimension mapping layer, and reshape it after upsampling to obtain a long-distance feature map; CNN decoder: used to restore the resolution of cerebral vascular features. The CNN decoder includes four layers of residual convolution modules. The last residual convolution module of the CNN decoder includes two groups of residual convolution groups, one of which includes a 1×1×1 convolution layer, a normalization layer, and a ReLU activation layer.

Images