Coronary artery image segmentation method based on improved nnUnet

Abstract

The invention provides a coronary artery image segmentation method based on improved nnUnet, and relates to the field of medical image segmentation. The method comprises the following steps: acquiring a training data set containing a CTA image of a normal coronary artery blood vessel and a CTA image of a lesion coronary artery blood vessel; adjusting the window width and the window level of the image; re-sampling the interlayer spacing of all the images in the training data set; a preprocessing module is constructed, the preprocessing module comprises an equalization sub-module and a filtering sub-module, and each layer of two-dimensional image of all CTA images in the training data set is preprocessed through the preprocessing module; constructing a segmentation model, and training the segmentation model by using the preprocessed image; and after a to-be-processed CTA image passes through the preprocessing module and the trained segmentation model, a segmentation result image is obtained. According to the method, the accuracy and continuity of coronary artery segmentation are improved, and the requirement of three-dimensional reconstruction of the coronary artery CTA image can be widely met.

Description

Technical Field

[0001] The present invention relates to the field of medical image segmentation, in particular to a coronary artery image segmentation method based on an improved nnUnet. Background Art

[0002] Cardiovascular disease is the leading cause of death worldwide, causing approximately 170,000 deaths each year, accounting for 30% of all deaths. Cardiovascular and cerebrovascular diseases are characterized by high morbidity, high disability, high recurrence and high mortality, bringing heavy social and economic burdens.

[0003] Accurately assessing the structure and function of the heart and its surrounding blood vessels is essential for the early diagnosis, treatment and prevention of cardiovascular disease. Some imaging methods, such as computed tomography (CT) and magnetic resonance imaging (MRI), provide fast and non-invasive means of examination. With their advantages of low storage space requirements and fast analysis speed, they have become important tools for assessing cardiovascular disease. Combining these images with medical image processing methods can achieve three-dimensional visual reconstruction of the heart including its accessory parts, convey potential information about the tissue structure to the user, and improve the patient's medical experience.

[0004] CTA imaging technology can provide high-resolution three-dimensional images of the heart and its surrounding blood vessels. When these cross-sectional slice images are used, the information between layers is often ignored. Analyzing these large-scale three-dimensional data is a tedious and time-consuming task, and is easily affected by operator subjective factors, which limits the comprehensive and accurate evaluation of the complex heart and its accessory vascular structures.

[0005] As an emerging artificial intelligence technology, deep learning has made significant breakthroughs in the field of image recognition and segmentation. Through deep learning models, features can be learned and extracted from complex CTA images to achieve automatic segmentation of the heart and its surrounding blood vessels. This will greatly reduce the workload of doctors, improve the accuracy and efficiency of diagnosis, and provide strong support for clinical decision-making. In addition, this segmentation method can also provide a basis for other related research and applications. For example, it can be used for heart and large blood vessel morphology analysis, cardiac function assessment, and cardiac surgery planning. By accurately extracting the anatomical structure of the heart and blood vessels, we can better understand the development process of the disease, evaluate the treatment effect, and provide support for precision medicine. However, the current deep learning model is prone to miss the thinner coronary artery terminal vessels when used for the CTA image segmentation task of coronary arteries, and the segmentation accuracy, continuity, and robustness of the coronary arteries need to be improved. Summary of the invention

[0006] In order to solve the problems in the prior art, the present invention proposes a coronary artery image segmentation method based on improved nnUnet.

[0007] To achieve the above purpose, the technical solution proposed by the present invention is as follows:

[0008] A coronary artery image segmentation method based on improved nnUnet, comprising:

[0009] S1: obtaining a training data set, the training data set including CTA images of normal coronary arteries and CTA images of diseased coronary arteries; adjusting the window width and window position of all images in the training data set so as to enhance the contrast between the coronary artery part and other parts in each image;

[0010] S2: resample the inter-layer spacing of all images in the training dataset to uniform voxels, and after unification, each image contains multiple layers of two-dimensional images;

[0011] S3: construct a preprocessing module, and preprocess the two-dimensional images of each layer of all CTA images in the training data set through the preprocessing module; the preprocessing module includes an equalization submodule and a filtering submodule, the equalization submodule is used to perform CLAHE processing on the two-dimensional images of each layer of the CTA image to obtain an enhanced image, and the filtering submodule is used to perform Frangi filtering on the enhanced image;

[0012] S4: construct a segmentation model for coronary artery image segmentation, and use the preprocessed images to train the segmentation model;

[0013] S5: Obtain the CTA image to be processed. After the CTA image to be processed passes through the preprocessing module and the trained segmentation model, a segmentation result image is obtained.

[0014] The present invention also proposes a coronary artery image segmentation system based on an improved nnUnet, which is used to implement the above method.

[0015] Compared with the existing methods, the present invention has the following significant improvements:

[0016] 1. The present invention adopts a new preprocessing module consisting of the CLAHE algorithm and Frangi filtering, which enhances the contrast of CTA images and makes the difference between the vascular area and the surrounding tissue more obvious. Finally, the vascular structure is highlighted through filtering to reduce the interference of background noise and non-vascular tissue.

[0017] 2. The present invention is based on the nnUnet architecture and applied to the field of coronary artery segmentation. The training set includes both normal and diseased stenotic vessels, which improves the robustness of the coronary artery segmentation network.

[0018] 3. The present invention improves the accuracy and continuity of coronary artery segmentation to a certain extent, and the proposed method can widely meet the needs of three-dimensional reconstruction of coronary artery CTA images. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is an overall flow chart of a coronary artery image segmentation method based on an improved nnUnet in one embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of a CLAHE in one embodiment of the present invention;

[0021] Figure 3 This is a diagram of the nnUnet network architecture in one embodiment of the present invention;

[0022] Figure 4 A comparison diagram of the segmentation results of an image to be detected between the embodiment of the present invention and the comparative example;

[0023] Figure 5 The figure is a comparison diagram of the segmentation results of another image to be detected between the embodiment of the present invention and the comparative example. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0025] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0026] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0027] The CTA image described in the present invention is CT angiography, which is a three-dimensional image of the coronary artery formed by multi-slice spiral CT through fast multi-slice scanning and image reconstruction technology after intravenous injection of contrast agent, and can obtain multi-layer two-dimensional images by sampling in a certain direction.

[0028] The coronary artery described in the present invention is a blood vessel wrapped around the periphery of the heart to provide the heart with its own blood supply. The entire coronary artery tree has a complex topological structure, with many small branches, and its diameter generally ranges from 2mm to 7mm. The main characteristic of the diseased coronary artery described in the present invention is that the lumen area is reduced, and it is narrower than the normal coronary artery in the CTA image.

[0029] The nnUnet model described in the present invention is a deep learning model based on the U-Net architecture, which can automatically adjust the model structure, training parameters, etc. according to the specific medical image segmentation task, and is suitable for medical image segmentation tasks.

[0030] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the following description, many specific details are described to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below. The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are only exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0032] Example 1

[0033] This embodiment provides a coronary artery segmentation method based on an improved nnUnet, comprising the following steps:

[0034] The first step is to obtain a training data set, which includes CTA images of normal coronary arteries and abnormal stenotic coronary arteries. The window width and window position of the CTA images of normal coronary arteries and stenotic coronary artery vessels in the data set are adjusted to highlight the coronary artery vessels. In this embodiment, the training data set comes from the 2020 ASOCA (Automated Segmentation of Coronary Arteries Post Challenge), and the image adjustment parameters in the data set are set as follows: the window width is 750 and the window position is 450.

[0035] The second step is to resample the inter-layer spacing of the CTA image to (1,1,2). The CTA image is composed of several layers of data. If the size of an image is , which can be considered as The layer size is The two-dimensional image composition. They represent the number of voxels in the X-axis, Y-axis, and Z-axis respectively. This data is recorded in the Size parameter of the original image.

[0036] Spacing parameter in CTA image header file , Respectively represent the size of the voxel in the X-axis, Y-axis and Z-axis directions. According to the Spacing parameter and the Size parameter, the size of the real 3D image can be calculated as The resampling process refers to modifying the size of voxels to achieve the purpose of unifying the data. The Spacing of the training data is unified to (1, 1, 2), while keeping the real image size unchanged. Correspondingly, this means that the Size parameter will change to:

[0037]

[0038] The third step is to build a special preprocessing module. The CTA image is processed by the preprocessing module. Specifically, the CTA image is first subjected to contrast-limited adaptive histogram equalization to enhance the image contrast. For each pixel of the enhanced CTA image, the Hessian matrix is ​​calculated and its eigenvalue is obtained. and , and then the plate-like variable and scale variables , each pixel and Substitute into the filter response function , calculate the filter response value corresponding to the pixel. By iterating the entire image matrix, performing such calculation operations on each pixel, a new, filtered image is finally obtained.

[0039] The preprocessing module is mainly composed of a contrast-limited adaptive histogram equalization (CLAHE) submodule and a Frangi filter submodule based on the Hessian matrix. CLAHE is an image enhancement method that introduces contrast limitation based on adaptive histogram equalization (AHE). The specific steps are as follows:

[0040] Block division: for each layer of the CTA image in the z-axis direction, in this embodiment, the size of each layer of the image is 512*512, and each layer of the image is divided into multiple sub-regions, the size of the sub-region is set to 4*4, and then discretized into a pixel matrix;

[0041] Calculate histogram: Calculate the histogram of each sub-region , the histogram represents the distribution of different pixel values ​​in the sub-region;

[0042] Histogram equalization: Figure 2 As shown in the figure, for the histogram of each sub-area, a uniform setting value Cliplimit is set to clip the part of the histogram that is higher than the setting value Cliplimit, and the clipped part is evenly distributed to other gray levels to keep the overall contrast of the image within a reasonable range. Histogram after histogram equalization It can be expressed as:

[0043]

[0044] in It is the average value of pixels above the set value Cliplimit, and the set value Cliplimit is set to 2 in this embodiment.

[0045] Interpolation reconstruction: After performing histogram equalization on each sub-region, bilinear interpolation is used to reconstruct the grayscale values ​​of adjacent sub-regions to make the images of adjacent sub-regions continuous, and the reconstructed image is output.

[0046] Frangi filtering is based on the eigenvalues ​​of the Hessian matrix to enhance blood vessels. The Hessian matrix is ​​a second-order partial derivative matrix. For a two-dimensional image function , whose Hessian matrix is defined as follows:

[0047]

[0048] in yes right The second-order partial derivative of yes First Find the partial derivative, and then Find the partial derivative. According to the properties of partial derivatives, we know that , yes right The second-order partial derivative of .

[0049] According to the concept of scale space, a two-dimensional image The scale space can be expressed as , in It is the scale space factor that determines the degree to which the image is smoothed. The construction of the scale space uses the Gaussian function to perform convolution operations. The two-dimensional Gaussian function The expression is as follows:

[0050]

[0051] The second-order derivative of an image at a certain point can be obtained by convolving the image with a Gaussian function, so it can be calculated The expression is:

[0052]

[0053]

[0054]

[0055] Combine the following two formulas to calculate the eigenvalue of the matrix:

[0056]

[0057]

[0058] in is the identity matrix, is the Hessian matrix of the two-dimensional image.

[0059] In the above formula There are two solutions, namely the eigenvalues ​​of the calculated two-dimensional Hessian matrix ,and The eigenvalues ​​and eigenvectors of the Hessian matrix can describe the local shape information of an image at a certain point, representing the size and direction of the curvature of the image along a certain direction at that point. For the detection of tubular structures, different eigenvalue combinations can distinguish different types of structures.

[0060] Then two variables were constructed, namely the "plate-like" variable With the "scale" variable ,when When it is larger, it means that the structure is more tubular ( ), the larger Indicates that the structure has a large local curvature change, which may correspond to a thicker tubular structure:

[0061]

[0062] Subsequent use and Construct the filter response function:

[0063]

[0064] in is the filter response value, and It is a hyperparameter that can be set. When performing Frangi filtering, the convolution scale also needs to be specified. The closer the convolution scale is to the blood vessel diameter, the better the filtering effect. The enumeration method is used to calculate multiple filtered images for the same point within the set range, and then the result with the highest response value is selected. In this embodiment, the relevant parameters are set as , the convolution scale is enumerated in the range of [1,2.5] with a step size of 0.5.

[0065] The filter response value of each pixel of the CTA image is calculated to replace the original pixel value, and finally the filtered image is obtained.

[0066] The fourth step is to build and train the segmentation model. The preprocessed CTA images are fed into the segmentation model. The model uses a total loss function that combines the cross-entropy loss function (Cross-Entropy Loss) with the Dice loss function ((Dice Loss)). The calculation of the cross-entropy loss function is as follows:

[0067]

[0068] in, is the cross entropy loss, y is the true label, the value is 0 or 1; is the predicted probability, in the range [0,1]. The Dice loss function is calculated as follows:

[0069]

[0070] in, is the Dice loss, i is the index number, is the predicted segmentation result, in the range of [0,1]; is the true segmentation label, with a value of 0 or 1. The total loss function used in segmentation model training is a linear combination of the cross entropy loss function and the Dice loss function, and the two have the same weight. The expression is as follows:

[0071]

[0072] The segmentation model is a nnUnet model, such as Figure 3As shown. The nnUnet model is an improved network model based on the Unet model. The improvement is that the leaky ReLU activation function is used to replace the ReLU function in the Unet model. In the preprocessing process, the non-zero area of ​​the provided training set data is first cropped. Based on the cropped training data, a dataset fingerprint is then created to record all relevant parameters and attributes, such as the image size before and after cropping (i.e., the number of voxels in each spatial dimension), image spacing (i.e., the physical size of the voxel), image modality (read from metadata), number of label categories, and number of training datasets. In addition, the interlayer size of the training data is also recorded. All models resample the interlayer spacing of the dataset to the median of all data interlayer spacings. The original data uses third-order spline interpolation, and the label uses the nearest neighbor interpolation method. When normalizing the CT modality data, the HU value range of the pixels in the label in the entire dataset is counted, and the pixels in the range of 0.5% to 99.5% are retained, and then normalized using the standard deviation normalization method.

[0073] nnUnet creates several Unet configurations for each dataset based on the dataset fingerprint: 2d configuration for 2D data, 3d fullres configuration for 3D high-resolution data, 3d lowres configuration for 3D low-resolution data, and 3d lowres and 3d_cascade_fullres cascade configuration for 3D full-resolution data. During training, the 3d fullres configuration for 3D high-resolution data is selected.

[0074] In the fifth step, the CTA image to be processed is preprocessed by the preprocessing module, and the preprocessed CTA image to be processed is sent to the segmentation model for segmentation of the coronary artery to generate and save the segmentation result map, such as Figure 4 and Figure 5 As shown in the middle sub-figure.

[0075] Example 2

[0076] In this embodiment, a coronary artery image segmentation system based on an improved nnUnet is also provided, and the system is used to implement the above embodiment. The terms "module", "unit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the system described in the following embodiments is preferably implemented in software, it is also possible to implement hardware, or a combination of software and hardware.

[0077] A data acquisition module, used to acquire training data sets and CTA images to be processed;

[0078] A window width and window level adjustment module is used to adjust the window width and window level of all images in the training data set;

[0079] The resampling module is used to resample the layer spacing of all images in the training dataset;

[0080] A preprocessing module, used for preprocessing all CTA images in the training data set and the two-dimensional images of each layer of the CTA images to be processed;

[0081] A model training module, used to train the segmentation model using preprocessed images in the training data set;

[0082] The segmentation model module is used to construct a segmentation model and use the segmentation model to perform coronary artery image segmentation on the image output by the preprocessing module.

[0083] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0084] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The system embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capabilities and run.

[0085] Comparative Example 1

[0086] For comparison, this embodiment uses the same training data set and segmentation model, trains the segmentation model according to the method described in Example 1, but does not perform the preprocessing in step 3. The trained segmentation model is used to perform coronary artery segmentation on the CTA image to be processed, and generates and saves the segmentation result map, such as Figure 4 and Figure 5 As shown in the left sub-figure of .

[0087] The difference between the segmentation result obtained in Example 1 and the segmentation result obtained in Comparative Example 1 is calculated. Figure 4 and Figure 5As shown in the right sub-figure of , it can be seen that the image segmentation method of the present invention makes the segmentation of the coronary artery more detailed, improves the accuracy and continuity of the coronary artery segmentation, and especially shows outstanding performance at the end of the arterial blood vessels which are easily missed by the traditional method.

[0088] The five-fold cross validation method and the pseudo-dice coefficient were used to evaluate the segmentation performance and effect of the methods described in comparative example 1 and embodiment 1. Specifically, the data set was divided into 5 parts, 4 of which were used as training data and 1 as verification data in turn for experiments. The pseudo-dice coefficient was used as an evaluation indicator of the segmentation accuracy in each experiment. The pseudo-dice coefficients of the 5 experiments are shown in the following table.

[0089] Table 1

[0090] Test No. Comparative Example 1 Example 1 0 0.81035 0.83321 1 0.80024 0.81322 2 0.84747 0.85867 3 0.84452 0.85247 4 0.84348 0.84781

[0091] The results show that the pseudo-dice coefficient of the results obtained in each test example 1 is higher than that in the comparative example 1, which indicates that the method described in example 1 of the present invention has a higher accuracy rate for segmenting coronary artery images, and the method of the present invention has good robustness.

[0092] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A coronary artery image segmentation method based on improved nnUnet, characterized in that: include: S1: obtaining a training data set, the training data set including CTA images of normal coronary arteries and CTA images of diseased coronary arteries; adjusting the window width and window position of all images in the training data set so as to enhance the contrast between the coronary artery part and other parts in each image; S2: resample the inter-layer spacing of all images in the training dataset to uniform voxels, and after unification, each image contains multiple layers of two-dimensional images; S3: construct a preprocessing module, and preprocess the two-dimensional images of each layer of all CTA images in the training data set through the preprocessing module; The preprocessing module includes an equalization submodule and a filtering submodule, wherein the equalization submodule is used to perform CLAHE processing on the two-dimensional images of each layer of the CTA image to obtain an enhanced image, and the filtering submodule is used to perform Frangi filtering on the enhanced image; S4: construct a segmentation model for coronary artery image segmentation, and use the preprocessed images to train the segmentation model; S5: Obtain the CTA image to be processed. After the CTA image to be processed passes through the preprocessing module and the trained segmentation model, a segmentation result image is obtained.

2. The coronary artery image segmentation method based on improved nnUnet according to claim 1, characterized in that: The two-dimensional images of each layer of the CTA image are two-dimensional images of each layer of the CTA image in the z-axis direction.

3. The coronary artery image segmentation method based on improved nnUnet according to claim 1, characterized in that: The CLAHE processing of each layer of the two-dimensional image of the CTA image specifically includes: Each layer of the two-dimensional image is divided into multiple sub-regions, and each sub-region is discretized into a pixel matrix; the histogram of each sub-region is calculated; the histogram of each sub-region is equalized; the grayscale values ​​of adjacent sub-regions are reconstructed by bilinear interpolation to make the images of each adjacent sub-region continuous, and the reconstructed sub-region images are spliced ​​to obtain the enhanced image.

4. The coronary artery image segmentation method based on improved nnUnet according to claim 3, characterized in that: The histogram equalization is performed on the histogram of each sub-region, and the expression is as follows: ； in, is the histogram after equalization, is the unbalanced histogram, is the set value, It is the average value of the two-dimensional image pixels above the set value Cliplimit.

5. The coronary artery image segmentation method based on improved nnUnet according to claim 1, characterized in that: The Frangi filter is a Frangi filter based on the Hessian matrix.

6. The coronary artery image segmentation method based on improved nnUnet according to claim 3, characterized in that: The Frangi filtering of the enhanced image is specifically performed as follows: Based on each layer of enhanced images in the CTA image, calculate the Hessian matrix of each pixel and find the eigenvalue of the Hessian matrix , based on the eigenvalue Obtaining the plate-like variable and scale variables ; Construct a filter response function to transform the plate-likeness variable of each pixel into and scale variables Substitute the filter response function to calculate the filter response value corresponding to the pixel point, and use the corresponding filter response value to replace the value of the original pixel point. After all pixels are replaced, the filtered image is obtained.

7. The coronary artery image segmentation method based on improved nnUnet according to claim 6, characterized in that: The Hessian matrix of each pixel is: ； in is the function of the enhanced image right The second-order partial derivative of yes First Find the partial derivative and then Find the partial derivative, yes First Find the partial derivative and then Find the partial derivative, yes right The second-order partial derivative of ; Partial derivative at the current pixel , , and The expression is: ； ； ； ； in is the Gaussian function of the enhanced image, is the scale space factor of the enhanced image, represents convolution, is a natural number; The eigenvalues ​​of the Hessian matrix , which can be obtained by solving the following expressions: ； ； Combined The two solutions of are the eigenvalues ​​of the Hessian matrix ; The eigenvalue-based Obtaining the plate-like variable and scale variables , the expression is: ； in ; The filter response function is: ； in is the filter response value, and is a hyperparameter.

8. The coronary artery image segmentation method based on improved nnUnet according to claim 1, characterized in that: The segmentation model is a nnUnet model.

9. A coronary artery image segmentation system based on improved nnUnet, characterized in that: For implementing the coronary artery image segmentation method based on improved nnUnet as described in claim 1, the system comprises: A data acquisition module, used to acquire training data sets and CTA images to be processed; A window adjustment module is used to adjust the window width and window position of all images in the training data set; The resampling module is used to resample the layer spacing of all images in the training dataset; A preprocessing module, used for preprocessing all CTA images in the training data set and the two-dimensional images of each layer of the CTA images to be processed; A model training module, used to train the segmentation model using preprocessed images in the training data set; The segmentation model module is used to construct a segmentation model and use the segmentation model to perform coronary artery image segmentation on the image output by the preprocessing module.

Images