A multi-modal cerebral vascular segmentation algorithm

Abstract

The invention discloses a cerebrovascular segmentation algorithm applicable to multiple modes, belonging to the technical field of medical image segmentation. The cerebrovascular segmentation algorithm applicable to multiple modes comprises the following steps: obtaining clinical cases of real patients and performing data preprocessing, respectively processing slice data of three viewing angles, binarizing the slice data and performing region of interest segmentation to obtain a roughly extracted binary image of the cerebrovascular, obtaining a roughly extracted cerebrovascular edge through algorithm processing, obtaining a point on a centerline of the roughly extracted cerebrovascular, spatially aligning the preprocessed MRA and CTA data with the roughly extracted centerline, registering the three-dimensional roughly extracted centerline to the brain DICOM data of the corresponding volume, using multiple images as input of a network model, constructing a cerebrovascular segmentation model applicable to multiple modes, and performing three-dimensional reconstruction on the obtained segmentation results. The invention has the advantages of improving the precision and accuracy of vascular segmentation, placing vascular fractures, and enhancing the robustness and reusability of the network model.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a cerebral blood vessel segmentation algorithm applicable to multiple modalities. Background Art

[0002] In recent years, cerebrovascular disease has become the leading cause of death among adults in my country. It has attracted widespread attention from the society due to its high incidence, high disability rate, and high mortality rate. Neurointerventional technology has emerged as the times require. With the assistance of digital subtraction angiography, selective angiography, embolization, dilation and other cerebrovascular disease treatments are achieved through catheters in cerebral blood vessels. Therefore, cerebrovascular segmentation has become the technical basis for neurointerventional surgery. At the same time, cerebrovascular segmentation of images is also an important prerequisite for various applications such as vascular image registration, motion estimation, three-dimensional reconstruction, and path planning. The results can also be used to quantitatively describe the different characteristics of vascular structures, observe and compare the geometric changes of blood vessels, and determine the location of vascular lesions. It is an important reference for doctors' preoperative diagnosis, preoperative planning, and postoperative treatment.

[0003] At present, when performing medical image segmentation, the existing technology can indeed solve some segmentation problems, but when facing more complex segmentation tasks, both traditional segmentation methods and deep learning-based segmentation methods have their own limitations. For traditional segmentation algorithms: it is difficult to uniformly process different types of medical images (such as MRA, CT, etc.); and in the process of image feature extraction, there is more manual participation and the degree of automation is low. In this regard, the segmentation method based on deep learning shows good characteristics, and can basically achieve semi-automatic or even fully automated segmentation in many segmentation tasks. At present, most of the segmentation tasks of deep learning-based segmentation methods are tissues and organs such as tumors, prostates, and aortic vessels, and have achieved relatively good segmentation results, but most of the segmentation targets have the characteristics of high structural integrity, almost no branches, and relatively large volumes, which reduces the difficulty of segmentation to a certain extent. In the segmentation of cerebral blood vessels, due to the characteristics of small brain blood vessels and many branches, the current well-performing network model has problems such as broken blood vessels and difficulty in extracting small blood vessels when it is transferred to cerebral blood vessel images for training.

[0004] Therefore, it is necessary to provide a multi-modal cerebral vascular segmentation algorithm to solve the above problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, an object of the embodiments of the present invention is to provide a cerebral blood vessel segmentation algorithm applicable to multiple modalities to solve the problems in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A multi-modal cerebral blood vessel segmentation algorithm includes the following steps:

[0008] S1. Obtain 60 sets of mixed MRA and CTA clinical cases of real patients;

[0009] S2. performing data preprocessing on the 60 groups of cases, wherein the data preprocessing includes vascular contrast adjustment, region of interest screening and Ants bias correction;

[0010] S3. Export each group of cases in S2, obtain slice data of three viewing angles, namely, the transverse, sagittal and coronal planes, perform binarization processing and segment the region of interest, and obtain a roughly extracted binary image of the cerebral blood vessels;

[0011] S4. Use the Canny algorithm to process the data obtained in S3 to obtain a rough extraction of the cerebral blood vessel edge;

[0012] S5. Calculate the centroid of the geometric figure composed of the blood vessel edge obtained in S4, which is the point on the roughly extracted cerebral blood vessel centerline;

[0013] S6. spatially align the preprocessed MRA and CTA data with the coarsely extracted centerline of the corresponding patient obtained in S5, register the three-dimensional coarsely extracted centerline to the brain DICOM data of the corresponding volume, and use the multiple images as the input of the network model;

[0014] S7. Build a multi-modal cerebrovascular segmentation model;

[0015] S8. Use the MC 3D reconstruction algorithm packaged in Python to perform 3D reconstruction on the segmentation results obtained in S7.

[0016] As a further solution of the present invention, the vascular contrast adjustment in S2 includes modifying the threshold of the MRA image and the HU value of the CTA image, and the region of interest screening method in S2 is to segment each group of data before training and screen out unnecessary parts.

[0017] As a further solution of the present invention, the steps of the Canny algorithm in step S4 are specifically as follows:

[0018] S1. Use Gaussian filtering to smooth the image and remove noise;

[0019] S2. Calculate the gradient strength and direction;

[0020] S3. Non-maximum suppression;

[0021] S4.Dual threshold detection;

[0022] S5. Isolated weak edge suppression.

[0023] As a further solution of the present invention, the centroid calculation formula in S5 is:

[0024]

[0025]

[0026]

[0027]

[0028] V(i,j) is the grayscale value of point (i,j). In the binary image, white is 1 and black is 0. 00 The result is the sum of the pixel values ​​of all white areas, where M 10 is the sum of the x coordinates of all white area pixels, M 01 is the sum of the y coordinates of all white areas, C x , C y The coordinates of the center of mass.

[0029] As a further solution of the present invention, the MRA, CTA data and the roughly extracted cerebral vascular centerline in S6 need to perform convolution operations, activation functions, and pooling operations on each layer of the neural network, and perform contact operations through skip links and right-side upsampling to ensure the consistency of the MRA, CTA data and the roughly extracted cerebral vascular centerline during the training process of the neural network, and provide conditions for calculating the consistency loss of the target area.

[0030] As a further solution of the present invention, the consistency loss between the MRA and CTA data and the roughly extracted cerebral vascular centerline in the target segmentation area is calculated as follows:

[0031]

[0032] Loss(F)=λLoss(U)+σLoss(x,y),

[0033] Loss(x,y) is the loss function for determining the consistency between the MRA and CTA data and the roughly extracted cerebral vascular centerline in the target segmentation area.

[0034] As a further solution of the present invention, when the coarsely extracted cerebral blood vessel center point in the target area is consistent with the blood vessel segmentation of MRA and CTA data, the value of Loss(x,y) is 0, and when the two are inconsistent, the value of Loss(x,y) is 1, Loss(U) is the original 3D-UNet loss function, λ and σ are the weights of Loss(x,y) and Loss(U) in Loss(F), and λ+σ=1.

[0035] As a further solution of the present invention, the parameter setting of the cerebrovascular segmentation model in S7 includes: inputting 60 groups of patient MRA, CTA data and the corresponding roughly extracted cerebrovascular centerlines into the network model, respectively passing through two downsampling channels through 3×3×3 convolutions, each convolution is followed by a Relu activation function and a dropout operation, and then using a 2×2×2 maximum pooling with a stride of 2. Each layer is connected to the upsampling module of the same layer through a contact operation. During the upsampling process, each layer is composed of a 2×2×2 upconvolution and two 3×3×3 convolutions, and each convolution surface is followed by a Relu activation function and a dropout operation.

[0036] In summary, compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0037] 1. By integrating the traditional segmentation algorithm with the deep learning-based segmentation algorithm, the roughly extracted cerebral vascular centerline is embedded in the 3D-UNet network preprocessing and training module, which improves the accuracy and precision of vascular segmentation and fundamentally solves the problem of vascular rupture caused by small and branched cerebral vessels during medical reconstruction;

[0038] 2. A data integration module suitable for input, storage and processing of multimodal image types is implemented. Through the data integration module, the robustness and reusability of the network model are enhanced;

[0039] 3. By introducing the dropout method into the network model, the gradient problem caused by the increase of parameters in most network models is avoided. While ensuring the effect of brain blood vessel segmentation, the calculation speed is improved to ensure real-time performance.

[0040] In order to more clearly illustrate the structural features and effects of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of data preprocessing according to an embodiment of the invention.

[0042] Figure 2 It is a schematic diagram of the structure of a multi-modal cerebral blood vessel segmentation network model in an embodiment of the invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0045] In one embodiment of the present invention, see Figure 1 and Figure 2 The multi-modal cerebral vascular segmentation algorithm comprises the following steps:

[0046] S1. Obtain 60 sets of mixed MRA and CTA clinical cases of real patients;

[0047] S2. The 60 groups of cases were subjected to data preprocessing, which included vascular contrast adjustment, region of interest screening, and Ants bias correction. In this step, vascular contrast adjustment included modifying the threshold of MRA images and the HU value of CTA images, which was conducive to improving network training and learning, and the region of interest screening method was to segment each group of data before training and screen out unnecessary parts, such as the neck, which could reduce model training pressure and improve training efficiency;

[0048] S3. Export each group of cases in S2, obtain slice data of three perspectives, namely, slice data of three perspectives, namely, cross-section, sagittal and coronal, perform binarization processing and region of interest segmentation on the slice data, and obtain a roughly extracted binary image of cerebral blood vessels;

[0049] S4. Use the Canny algorithm to process the data obtained in S3 to obtain the rough extraction of cerebral vascular edges. This step includes the following sub-steps:

[0050] S1. Use Gaussian filtering to smooth the image and remove noise. The specific steps are:

[0051]

[0052] Since the edge detection operator cannot work well on unprocessed images, the original image needs to be convolved with a Gaussian kernel. The Gaussian kernel used in Gaussian filtering is a Gaussian function H(x,y) with two dimensions of x and y.

[0053] S2. Calculate the gradient strength and direction. The specific steps are:

[0054]

[0055] θ=arc tan(G y / G x ) (2)

[0056] The Canny algorithm uses edge detection operators to detect horizontal, vertical, and diagonal edges in an image and returns the first-order derivative values ​​in the horizontal Gx and vertical Gy directions, from which the gradient G and gradient direction θ of the pixel point can be determined;

[0057] S3. Non-maximum suppression. Relying solely on gradients cannot determine edges well. To make edges clearer, it is necessary to retain the local maximum gradient of the pixel. The specific steps are as follows:

[0058] a. Compare the gradient strength of the current pixel with two pixels along the positive and negative gradient directions.

[0059] b. If the gradient strength of the current pixel is the largest compared to the other two pixels, the pixel is retained as an edge point, otherwise the pixel will be suppressed;

[0060] S4. Double threshold detection, define a high threshold and a low threshold. Pixels with gradient strength lower than the low threshold are suppressed and not considered as edge points; pixels higher than the high threshold are defined as strong edges and retained as edge points; pixels between the high and low thresholds are defined as weak edges and reserved for further processing;

[0061] S5. Isolated weak edge suppression. Generally speaking, weak edge pixels caused by real edges will be connected to strong edge pixels, while noise responses are not connected. By checking the weak edge pixels and their eight neighboring pixels, we can make a judgment based on their connection with the strong edge. Generally, we can define that as long as one of the neighboring pixels is a strong edge pixel, the weak edge can be retained as a strong edge, i.e., a real edge point.

[0062] S5. Obtain the centroid of the geometric figure composed of the blood vessel edge obtained in S4, which is the point on the roughly extracted cerebral blood vessel centerline. Since the blood vessel slices are mostly circular or nearly circular, and the midline point of the blood vessel section is always located at the geometric center of the section, this algorithm assumes that the intersection of the cerebral blood vessel centerline and the image slice is always located at the centroid of the blood vessel section. That is, obtain the centroid of the geometric figure composed of the blood vessel edge obtained in step 4, which is the point on the roughly extracted cerebral blood vessel centerline;

[0063] S6. spatially align the preprocessed MRA and CTA data with the coarsely extracted centerline of the corresponding patient obtained in S5, register the three-dimensional coarsely extracted centerline to the brain DICOM data of the corresponding volume, and use the multiple images as the input of the network model;

[0064] S7. Build a multi-modal cerebrovascular segmentation model;

[0065] S8. Use the MC 3D reconstruction algorithm packaged in Python to perform 3D reconstruction on the segmentation results obtained in S7.

[0066] In one embodiment of the present invention, see Figure 1 and Figure 2 , the centroid calculation formula in S5 is:

[0067]

[0068]

[0069]

[0070]

[0071] V(i,j) is the grayscale value of point (i,j). In the binary image, white is 1 and black is 0. 00 The result is the sum of the pixel values ​​of all white areas, where M 10 is the sum of the x coordinates of all white area pixels, M 01 is the sum of the y coordinates of all white areas, C x , C y The coordinates of the center of mass.

[0072] In one embodiment of the present invention, see Figure 1 and Figure 2 On the basis of the original 3D-UNet network model, the traditional segmentation method is integrated to be applicable to the multimodal cerebrovascular segmentation model. In the multimodal cerebrovascular segmentation model, the application scope of the network model can be enhanced by adopting multi-type image input; by adopting a new downsampling mode, that is, two downsampling paths, the feature extraction in the downsampling process can be enhanced; by adding dropout after the activation function Relu, the model can be simplified, overfitting can be avoided, and the generalization ability of the two models can be improved; by adding the consistency loss of the center point on the coarse extraction center line and the corresponding regional MRA and CTA data in the loss function, the recognition of small blood vessels is improved, the image loss is reduced, and the breakage of blood vessels is avoided, realizing the integration of traditional methods and deep learning, which specifically includes the following contents:

[0073] a. The roughly extracted cerebral vascular centerline and the MRA and CTA data obtained in S2 are simultaneously used as the input of the network model. In each layer of the neural network, the roughly extracted cerebral vascular centerline and the MRA and CTA data need to be convolved, activated and pooled at each layer, and contacted through skip links and right upsampling. This ensures the consistency of the roughly extracted cerebral vascular centerline and the MRA and CTA data during the training of the neural network, and provides the basic conditions for the subsequent calculation of the consistency loss of the two in the target area. At the same time, during the training process, a suitable convolution kernel is selected to act on the roughly extracted cerebral vascular centerline obtained in S5. The convolution kernel continuously strengthens the centroid feature to ensure that the roughly extracted cerebral vascular centerline will not disappear due to convolution and pooling in the training of each layer of the neural network, and at the same time ensure that its spatial structure and the relative position of the MRA and CTA data at the same layer in the neural network remain unchanged;

[0074] b. Loss function: The weighted sigmoid loss function used in the original 3D-UNet network is denoted as Loss(U). In the present invention, in order to solve the problems of disappearance and breakage of small blood vessels in the segmentation process of the network model, the consistency loss of the coarsely extracted cerebral vascular centerline in the target segmentation area and MRA and CTA is added to the loss function:

[0075]

[0076] Loss(F)=λLoss(U)+σLoss(x,y) (4)

[0077] Loss(x,y) is a loss function for judging the consistency between the coarsely extracted cerebral vascular centerline and the MRA and CTA data in the target segmentation area. When the coarsely extracted cerebral vascular center point in the target area is consistent with the vascular segmentation of the MRA and CTA data, the value of Loss(x,y) is 0. When the two are inconsistent, the value of Loss(x,y) is 1. Loss(U) is the original 3D-UNet loss function. λ and σ are the weights of Loss(x,y) and Loss(U) in Loss(F), and λ+σ=1. By adjusting the different proportions of λ and σ, the performance of the network model can be optimized.

[0078] c. Parameter setting: 60 groups of patient MRA, CTA data and the corresponding roughly extracted cerebral vascular centerlines were input into the network model, and then passed through two downsampling channels through 3×3×3 convolutions. Each convolution was followed by a Relu activation function and a dropout operation, followed by a 2×2×2 maximum pooling with a stride of 2. Each layer was connected to the upsampling module of the same layer through a contact operation. During the upsampling process, each layer was composed of a 2×2×2 upconvolution and two 3×3×3 convolutions. Each convolution surface was followed by a Relu activation function and a dropout operation.

[0079] In this embodiment, this algorithm has important practical value in the preoperative diagnosis, preoperative planning, and postoperative treatment of brain diseases in actual neurointerventional surgery. It not only realizes the segmentation of complex medical tissue structures of brain blood vessels, but also solves the problem of blood vessel rupture caused by small cerebral blood vessels. It integrates traditional methods with deep learning to reduce feature loss. At the same time, it realizes the segmentation of the same image of multiple types of medical images. This algorithm obtains 60 MRA and CTA clinical cases of real patients, trains them, and finally reconstructs and displays the segmented cerebral vascular dicom data in three dimensions, so as to dynamically observe the three-dimensional structure of cerebral blood vessels and the continuity of small blood vessels.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-modal cerebral vascular segmentation algorithm, characterized in that: The following steps are involved: S1. Obtain 60 sets of mixed MRA and CTA clinical cases of real patients; S2. performing data preprocessing on the 60 groups of cases, wherein the data preprocessing includes vascular contrast adjustment, region of interest screening and Ants bias correction; S3. Export each group of cases in S2, obtain slice data of three viewing angles, namely, the transverse, sagittal and coronal planes, perform binarization processing and segment the region of interest, and obtain a roughly extracted binary image of the cerebral blood vessels; S4. Use the Canny algorithm to process the data obtained in S3 to obtain a rough extraction of the cerebral blood vessel edge; S5. Calculate the centroid of the geometric figure composed of the blood vessel edge obtained in S4, which is the point on the roughly extracted cerebral blood vessel centerline; S6. spatially align the preprocessed MRA and CTA data with the coarsely extracted centerline of the corresponding patient obtained in S5, register the three-dimensional coarsely extracted centerline to the brain DICOM data of the corresponding volume, and use the multiple images as the input of the network model; S7. Build a multi-modal cerebrovascular segmentation model; S8. Use the MC 3D reconstruction algorithm packaged in Python to perform 3D reconstruction on the segmentation results obtained in S7.

2. The multi-modal cerebral vessel segmentation algorithm according to claim 1, characterized in that: The vascular contrast adjustment in S2 includes modifying the threshold of the MRA image and the HU value of the CTA image. The region of interest screening method in S2 is to segment each group of data and screen out unnecessary parts before training.

3. The multi-modal cerebral vessel segmentation algorithm according to claim 1, characterized in that: The steps of the Canny algorithm in step S4 are specifically as follows: S1. Use Gaussian filtering to smooth the image and remove noise; S2. Calculate the gradient strength and direction; S3. Non-maximum suppression; S4.Dual threshold detection; S5. Isolated weak edge suppression.

4. The multi-modal cerebral vessel segmentation algorithm according to claim 1, characterized in that: The centroid calculation formula in S5 is: V(i,j) is the grayscale value of point (i,j). In the binary image, white is 1 and black is 0. 00 The result is the sum of the pixel values ​​of all white areas, where M 10 is the sum of the x coordinates of all white area pixels, M 01 is the sum of the y coordinates of all white areas, C x , C y The coordinates of the center of mass.

5. The multi-modal cerebral vessel segmentation algorithm according to claim 1, characterized in that: The MRA, CTA data and the roughly extracted cerebral vascular centerline in S6 need to perform convolution operations, activation functions, and pooling operations on each layer of the neural network, and perform contact operations through skip links and right upsampling to ensure the consistency of the MRA, CTA data and the roughly extracted cerebral vascular centerline during the training process of the neural network, and provide conditions for calculating the consistency loss of the target area.

6. The multi-modal cerebral vessel segmentation algorithm according to claim 5, characterized in that: The calculation method for the consistency loss between the MRA and CTA data and the roughly extracted cerebral vascular centerline in the target segmentation area is: Loss(F)=λLoss(U)+σLoss(x,y), Loss(x,y) is the loss function for determining the consistency between the MRA and CTA data and the roughly extracted cerebral vascular centerline in the target segmentation area.

7. The multi-modal cerebral blood vessel segmentation algorithm according to claim 6, characterized in that: When the roughly extracted cerebral vascular center point in the target area is consistent with the vascular segmentation of MRA and CTA data, the value of Loss(x,y) is 0. When the two are inconsistent, the value of Loss(x,y) is 1. Loss(U) is the original 3D-UNet loss function, λ and σ are the weights of Loss(x,y) and Loss(U) in Loss(F), and λ+σ=1.

8. The multi-modal cerebral blood vessel segmentation algorithm according to claim 7, characterized in that: The parameter setting of the cerebrovascular segmentation model in S7 includes: inputting 60 groups of patient MRA, CTA data and the corresponding roughly extracted cerebrovascular centerlines into the network model, respectively passing through two downsampling channels through 3×3×3 convolutions, each convolution is followed by a Relu activation function and a dropout operation, then using a 2×2×2 maximum pooling with a stride of 2, each layer is connected to the upsampling module of the same layer through a contact operation, and during the upsampling process, each layer is composed of a 2×2×2 upconvolution and two 3×3×3 convolutions, and each convolution surface is followed by a Relu activation function and a dropout operation.

Images