Aortic dissection segmentation method based on edge response and nonlinear loss
By combining a deep edge response supervision module and an adaptive logarithmic loss function, along with gamma grayscale transformation and a 3D cascaded segmentation model, the problems of edge discontinuity and class imbalance in aortic dissection segmentation are solved, thereby improving segmentation accuracy.
Patent Information
- Application Number
- CN202210019396.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-07
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-01-07
AI Technical Summary
Existing technologies suffer from discontinuities in edge regions and class imbalances in aortic dissection segmentation, resulting in poor segmentation performance, especially in the segmentation of the true lumen, false lumen, and the entire aorta, where accuracy is insufficient.
We designed a deep edge response supervision module and an adaptive logarithmic loss function, combined with gamma grayscale transformation to enhance aortic dissection features. Through a three-dimensional cascaded segmentation model based on deep edge response, we alleviated the problems of discontinuity in edge regions and class imbalance, thereby improving segmentation accuracy.
By enhancing attention to the marginal region of aortic dissection and balancing the loss contributions of different categories, the segmentation accuracy of the whole aorta, true lumen, and false lumen of aortic dissection was significantly improved, and the problems of edge discontinuity and category imbalance were resolved.
Smart Images

Figure CN114359308B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of medical image processing, and in particular relates to an aortic dissection segmentation method based on edge response and nonlinear loss. Background Art
[0002] Aortic dissection (AD) is the aortic disease with the highest mortality rate. It has complex clinical manifestations and lacks specific symptoms and signs. The diagnosis of this disease requires a high level of professionalism from doctors. Non-cardiovascular doctors who do not have a good understanding of AD are prone to misdiagnosis. AD should be classified according to the source of the intimal rupture and the extent of the dissection. According to the Stanford classification method, it can be divided into type A and type B. Each type of dissection has a corresponding professional treatment method. However, the surgical treatment of type A and type B dissections requires the physician to accurately grasp the location of the rupture and the extent of the dissection before surgery. Mastering this necessary information is of great significance in preoperative diagnosis, surgical planning and postoperative treatment, ensuring timely and effective treatment for patients. Automatic and accurate segmentation of the true lumen, false lumen and entire aorta of aortic dissection can save valuable time for patients and provide assistance to doctors in surgical planning and postoperative treatment. At present, the segmentation of aortic dissection mainly includes model-based methods, wavelet analysis-based methods and deep learning-based methods.
[0003] Model-based method: The circular prior knowledge of the aorta and the Hough transform are used to segment the aorta without considering the uneven distribution of the intimal flap and contrast agent in the false lumen. There are three main steps: segmenting the entire aortic lumen, detecting the intimal flap, and identifying the true lumen and false lumen. Since each step is highly dependent on the previous step, and the method largely utilizes the prior assumptions about the morphology and characteristics of the aortic dissection in CT images. Therefore, the model-based method can easily fail on irregularly shaped aortas, such as aortas with aneurysms and severe stenosis.
[0004] Wavelet analysis-based method: Combine wavelet analysis method and generative-discriminative model matching learning method to segment the true lumen and false lumen of aortic dissection. First, a semi-automatic method is used to detect the complete boundary of the aorta. Second, the inner edge of the aortic dissection is detected using wavelet analysis. Then, the detected edge is refined using a discriminative learning method to predict only the lumen edge points. Finally, a generative learning method is used to distinguish the true lumen from the false lumen by estimating the density distribution within the aortic boundary.
[0005] Methods based on deep learning: Segmentation of aortic dissection based on deep learning is the current trend. At present, researchers have proposed the following methods for segmenting aortic dissection based on deep learning: 1. Segment the aortic dissection CT image into true and false lumens on multi-planar reconstruction images perpendicular to the aortic centerline, and obtain quantitative morphological features, especially the aortic diameter and the cross-sectional area of the true and false lumens. The design includes two convolutional neural network (CNN) segmentation algorithms, which are used to obtain the aortic centerline, generate MPRs orthogonal to the centerline, and segment the true and false lumens. 2. Aortic dissection segmentation model based on fully convolutional neural network. Combining local information and global information to improve segmentation accuracy, and then passing through a fully connected random field, the network can pay attention to the detailed information of the edges of the true and false lumens in the aortic dissection image. 3. Use two independent two-dimensional cascaded CNN networks to extract the contours of the entire aortic lumen and true lumen respectively. The false lumen contour can be obtained by converting from the contours of the entire aortic lumen and true lumen. 4. Use contrast-enhanced CT images to segment aortic dissection. First, a U-Net-based semantic segmentation architecture is constructed and applied to contrast-enhanced CT images to segment the true lumen of aortic dissection. Then, the segmentation results are used to perform aortic roundness analysis to obtain slice-level detection results.
[0006] There is a class imbalance problem in both the full aortic dissection segmentation task and the true and false lumen segmentation task in aortic dissection, that is, the frequency of occurrence of different categories during segmentation is quite different, which leads to the network model not being able to converge well on all categories at the same time, resulting in inconsistent segmentation effects on each category and poor segmentation effects. At the same time, due to the special structure of aortic dissection, there is a thin film, i.e., a dissection, between the true lumen and the false lumen of aortic dissection, which will lead to discontinuity in the aortic dissection area, and there is also discontinuity in the edge area of aortic dissection. These problems are huge obstacles to the segmentation of the full aorta and the true and false lumen of aortic dissection. However, current research does not take into account the regional discontinuity and class imbalance problems in the segmentation process when segmenting the true and false lumen and the full aorta of aortic dissection. Therefore, the edges of the segmentation results are often not smooth enough and the segmentation accuracy is not high. Summary of the invention
[0007] The present invention proposes a method for segmenting aortic dissection based on edge response and nonlinear loss. The purpose is to provide a solution to the discontinuity of edge regions and the imbalance of categories in the segmentation of aortic dissection. First, a deep edge response supervision module and a loss function based on edge response are designed to alleviate the problem of rough edge segmentation caused by the discontinuity of edge regions in segmentation. Secondly, an adaptive logarithmic loss function is designed to address the category imbalance problem in the segmentation of the true and false lumens and the whole aorta of aortic dissection. A combined loss function is obtained based on the two loss functions and applied to the subnetwork. Finally, a deep edge response three-dimensional cascade segmentation model for aortic dissection is designed, and the first subnetwork is used to segment the whole aorta of aortic dissection to eliminate background interference. The second subnetwork is used to segment the true lumen and false lumen of aortic dissection. The present invention can better alleviate the problems of regional discontinuity and category imbalance in the segmentation of aortic dissection, and improve the segmentation accuracy of the true lumen, false lumen and the whole aorta.
[0008] To achieve the above object, the present invention provides an aortic dissection segmentation method based on edge response and nonlinear loss, comprising the following steps:
[0009] Preprocessing the aortic dissection CT images, and dividing the preprocessed aortic dissection CT images into a training set and a validation set;
[0010] Establish a three-dimensional cascade segmentation model of aortic dissection depth edge response;
[0011] Assume a combined loss function based on edge response loss and adaptive logarithmic loss;
[0012] The aortic dissection depth edge response three-dimensional cascade segmentation model is trained based on the combined loss function and the training set, and the aortic dissection segmentation is completed using the trained aortic dissection depth edge response three-dimensional cascade segmentation model.
[0013] Optionally, the preprocessing includes data format conversion processing, gamma enhancement processing, data geometric transformation augmentation processing and standardization processing.
[0014] Optionally, the data format conversion process is to convert the aortic dissection CT image from a two-dimensional format to a three-dimensional format;
[0015] The gamma enhancement process is to perform gamma grayscale transformation on the three-dimensional aortic dissection CT image, and the change formula of the gamma grayscale transformation is:
[0016] CT out =2CT in 0.65
[0017] Among them, CT inis the gray value of the image before gamma grayscale transformation, CT out is the gray value of the image after gamma grayscale transformation, 2 is the grayscale scaling factor, and 0.65 is the gamma factor value;
[0018] The data geometric transformation augmentation process is to use translation, rotation, flipping, and mirroring geometric transformations of the image to increase the data volume of the aortic dissection CT image after the gamma enhancement process;
[0019] The standardization process is to standardize the resolution and grayscale value of the aortic dissection CT image after the data geometric transformation and augmentation process, the resolution is processed to 256×128×128, and the grayscale value is processed to between 0-1.
[0020] Optionally, the aortic dissection depth edge response three-dimensional cascade segmentation model includes a first sub-network and a second sub-network;
[0021] The first sub-network is used to segment the entire aorta region of aortic dissection;
[0022] The second sub-network is used to segment the true lumen and false lumen of aortic dissection;
[0023] The first sub-network and the second sub-network both include an encoder, a decoder, and a plurality of hop connections.
[0024] Optionally, both the first sub-network and the second sub-network further include a deep edge response supervision module;
[0025] The deep edge response supervision module is used to process the output feature maps sampled at each layer in the first sub-network and the second sub-network.
[0026] Optionally, the method for processing the upsampled output feature map of each layer includes:
[0027] Calculate the gradient magnitude of each pixel in the feature map;
[0028] Calculate the edge response coefficient matrix of the feature map;
[0029] A processed output feature map is obtained based on the gradient amplitude and the edge response coefficient matrix, and deep supervision is performed on the processed output feature map.
[0030] Optionally, the edge response loss function formula is:
[0031]
[0032] Among them, DC loss is the additional loss value of the edge area, n c Indicates the total number of categories, N cis the number of pixels belonging to category c in the image, img_pb cn is the predicted value of the nth pixel of category c in image img_pb, img_tb cn is the true value of the nth pixel of category c in the image img_tb; img_pb is the predicted image containing only the edge area, and img_tb is the true label image containing only the edge area;
[0033] The adaptive logarithmic loss function formula is:
[0034]
[0035] Among them, HE loss is the adaptive logarithmic loss function value, n c is the total number of categories, w c is the loss weight coefficient corresponding to category c, DL c is the Dice Loss value corresponding to category c;
[0036] The DL c The formula is:
[0037]
[0038] Among them, r cn is the true value of the nth pixel of category c, p cn is the predicted value of the nth pixel of category c, N c is the number of pixels belonging to category c in the image;
[0039] The w c The formula is:
[0040]
[0041] Optionally, the method for training the aortic dissection depth edge response three-dimensional cascade segmentation model is:
[0042] Inputting the training set into the first sub-network, calculating the loss using the combined loss function, and adjusting the hyperparameters using the validation set to complete the training of the first sub-network and obtain the segmentation result of the first sub-network;
[0043] An intersection operation is performed on the segmentation result of the first sub-network and the training set, and the operation result is input into the second sub-network as the training set of the second sub-network, the loss is calculated using the combined loss function, and the hyperparameters are adjusted using the validation set to complete the training of the second sub-network.
[0044] Compared with the prior art, the present invention has the following advantages and technical effects:
[0045] 1. The present invention aims to solve the problem that the CT image of aortic dissection is dark, and the contrast between the aortic dissection area and the background area in the image is not high, which makes it difficult to distinguish the boundary area of the aortic dissection. A method for enhancing the CT image of aortic dissection based on gamma grayscale transformation is proposed to enhance the features of aortic dissection in CT images. The overall brightness of the CT image is greatly improved by gamma grayscale transformation, and the contrast is also greatly improved. It makes it easier for the segmentation model to learn the features of aortic dissection, laying a good foundation for subsequent segmentation tasks.
[0046] 2. Aiming at the problem of low segmentation accuracy caused by discontinuous edge areas in aortic dissection segmentation, the present invention proposes a deep edge response supervision module. This module allows the model to focus on the edge area of the aortic dissection, thereby improving the accuracy of the model segmentation. The deep edge response supervision module will set a larger edge response coefficient for the pixels in the edge area of the aortic dissection in the CT image. Therefore, during the training process, the model will pay more attention to the pixels in the edge area, thereby improving the accuracy of the model segmentation.
[0047] 3. Aiming at the problems of category imbalance and regional discontinuity in the segmentation of aortic dissection, the present invention proposes a combined loss function based on edge response loss and adaptive logarithmic loss. The ability of the model to accurately extract features is enhanced, and the segmentation accuracy of the model is improved. Among them, the loss function based on edge response calculates additional losses for pixels in the edge area of the aortic dissection. The adaptive logarithmic loss function does not depend on the size of the target in the image and can balance the loss contribution of small category targets. The application of this combined loss function can improve the segmentation accuracy of the model for the entire aorta, true lumen, and false lumen of aortic dissection. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0049] Figure 1 A schematic flow chart of an aortic dissection segmentation method based on edge response and nonlinear loss according to an embodiment of the present invention;
[0050] Figure 2 Schematic diagram of the network structure of the first sub-network and the second sub-network according to an embodiment of the present invention. DETAILED DESCRIPTION
[0051] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0052] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0053] Example
[0054] like Figure 1 As shown, this embodiment provides an aortic dissection segmentation method based on edge response and nonlinear loss, including:
[0055] The aortic dissection CT images were preprocessed and divided into a training set and a validation set.
[0056] In this embodiment, since the original aortic dissection CT image is easily affected by factors such as noise and volume effect, compared with ordinary images, the CT image has the disadvantages of blurred target edges and poor light-dark contrast. Therefore, before the aortic dissection CT image is used for model training, the original image data needs to be preprocessed. The preprocessing includes data format conversion, gamma enhancement, data geometric transformation and augmentation, and resolution and gray value standardization.
[0057] (1) Data format conversion processing
[0058] The original data is 120 cases of abdominal aortic dissection CT images, all of which were taken during the hospitalization of patients with aortic dissection in a certain hospital. The CT scan image of each case is stored in a folder, which contains multiple DICOM files, and each DICOM file consists of a two-dimensional scan image and the image data header information.
[0059] For CT images, the segmentation effect of two-dimensional network is usually worse than that of three-dimensional network. Because the three-dimensional network can extract the spatial position information between CT image slices, while the two-dimensional segmentation network ignores the positional relationship between different slices and cannot use this information, and the images at different positions are often quite different, which is not conducive to network learning of general features. Therefore, the aortic dissection segmentation model of the present invention will be based on a three-dimensional convolutional neural network, and its convolution layer is also three-dimensional, requiring the input to be a three-dimensional tensor. Therefore, the DICOM format data should be parsed and all the two-dimensional images in each case folder should be reconstructed into three-dimensional .nii format images.
[0060] (2) Gamma enhancement processing
[0061] Each pixel position in the CT image has a grayscale value, and different grayscale values represent different degrees of absorption of radiation by various parts of the human body. The areas where the image appears black, such as the air part in the abdominal cavity, indicate that the absorption degree is low. The areas where the image appears white, such as bones, indicate that the absorption degree is very high and are high-density areas. By statistically analyzing the aortic dissection data set, it is known that the grayscale range of the aortic dissection area in the thoracic and abdominal CT images is greater than 0, while the grayscale values of some background areas such as air are much less than 0. Therefore, the CT image is thresholded first. That is, all grayscale values less than 0 in the CT image are set to 0 to eliminate interference from complex background areas. Then perform a gamma grayscale transform on the CT image, and the formula is as follows:
[0062] s=cr γ (1)
[0063] Where r is the grayscale value of the input grayscale image, s is the output grayscale value after the gamma grayscale transformation, c is the grayscale scaling factor, and γ is the gamma factor size, which controls the scaling degree of the transformation. When the γ value is less than 1, the area with lower grayscale value in the image will be stretched, and the area with higher grayscale value will be compressed. When the γ value is greater than 1, the opposite is true. The gamma grayscale transformation formula in the present invention is as follows:
[0064] CT out =2CT in 0.65 (2)
[0065] Among them, CT in is the gray value of the original image, CT out is the gray value after gamma grayscale transformation, and the grayscale scaling factor is 2. In order to improve the brightness of the aortic dissection CT image and highlight the aortic dissection area, the gamma factor value is 0.65. Through gamma grayscale transformation, the overall brightness of the CT image is greatly improved, the contrast is also greatly improved, the aortic area is highlighted, and useless information such as background is suppressed, laying a good foundation for subsequent segmentation tasks.
[0066] (3) Data geometric transformation and augmentation processing
[0067] Deep learning neural networks generally require a large amount of training data. Increasing the amount of data can enable the network to learn different features of the image, which can usually greatly improve the effect of the algorithm. However, in real life, valuable medical data is often difficult to obtain, and it takes experts several hours to manually segment an image. Therefore, data augmentation is an indispensable part of medical image segmentation. In order to improve the training accuracy of the network, the present invention uses geometric transformations such as translation, rotation, flipping, and mirroring of the image to increase the training data.
[0068] (4) Resolution and grayscale value standardization
[0069] The two-dimensional image is converted into a three-dimensional image through the data format conversion operation in (1). First, the image resolution is standardized. The resolution of the aortic dissection CT image can be expressed as 512×512×s, where s is the number of slices. The number of slices for different patient data may be different. Considering the CPU graphics card memory size, the CT image is scaled to 256×256×256, and then the center is cropped to highlight the aortic dissection area in the image. Finally, the aortic dissection data is cropped to a size of 128×128×256. At the same time, in order to make the distribution of image data as consistent as possible, the grayscale value of the CT image needs to be standardized. The grayscale value standardization formula is shown in (3), where Image in is the gray value of the input image, Image max ,Image min are the maximum and minimum grayscale values in the input image, respectively. out is the grayscale value of the output image.
[0070]
[0071] The resolution of aortic dissection CT images was normalized to 256×128×128 through the above standardization process, and the grayscale values were all between 0 and 1. The above preprocessed data was divided into a training set and a validation set in a ratio of 5:1, which were used for network training and validation in subsequent steps, respectively.
[0072] A deep edge response three-dimensional cascade segmentation model for aortic dissection is established; the deep edge response three-dimensional cascade segmentation model for aortic dissection includes a first subnetwork and a second subnetwork; the first subnetwork is used to segment the entire aortic region of aortic dissection; the second subnetwork is used to segment the true lumen and false lumen of aortic dissection; the first subnetwork and the second subnetwork both include an encoder, a decoder, and a plurality of skip-layer connections. The first subnetwork and the second subnetwork both also include a deep edge response supervision module; the deep edge response supervision module is used to process the output feature map sampled on each layer in the first subnetwork and the second subnetwork.
[0073] The method for processing the output feature map of each upsampled layer includes:
[0074] Calculate the gradient amplitude of each pixel in the feature map; calculate the edge response coefficient matrix of the feature map; obtain a processed output feature map based on the gradient amplitude and the edge response coefficient matrix, and perform deep supervision on the processed output feature map.
[0075] In this embodiment, the established deep edge response three-dimensional cascade segmentation model for aortic dissection includes two sub-networks. The first sub-network is used to segment the entire aortic region of aortic dissection, and the second sub-network is used to segment the true lumen and false lumen of aortic dissection. Each sub-network includes an encoder, a decoder, a plurality of skip-layer connections and a deep edge response supervision module. The network structures of the first sub-network and the second sub-network are shown in the attached Figure 2 shown.
[0076] (1) Structure of the three-dimensional cascade segmentation model for aortic dissection depth edge response
[0077] When performing the segmentation of the true and false lumens of aortic dissection, the frequency of occurrence of the false lumen, true lumen and background categories in the CT image is quite different, which causes the network model to not converge well on all categories at the same time. Therefore, the segmentation effects on each category are inconsistent and the segmentation effect is poor. This is the category imbalance problem in the segmentation of the true and false lumens of aortic dissection. In order to alleviate the category imbalance problem in the whole aorta segmentation process of aortic dissection, the present invention adopts a cascade segmentation model. There are two subnetworks in total. The first subnetwork is used to segment the whole aorta area of aortic dissection, and the second subnetwork is used to segment the true lumen and false lumen areas of aortic dissection.
[0078] Compared with a single segmentation model, the cascade segmentation model can gradually reduce the region of interest based on the prediction results of the previous sub-network, thereby reducing overfitting segmentation. At the same time, because the first sub-network segments the entire aorta area, it eliminates the interference of irrelevant areas such as the background, reduces the calculation area, and enables the second sub-network to focus more on the true lumen and false lumen areas rather than the background area. This alleviates the imbalance problem of the true and false lumens in aortic dissection to a certain extent and improves the segmentation accuracy.
[0079] Both sub-networks are improved based on the 3D U-Net network. Their basic architecture consists of three parts: an encoder for feature extraction, a symmetrical decoder for restoring spatial resolution to generate segmentation results, and a skip-layer connection. The overall network is in the shape of a capital letter U.
[0080] (2) Deep Edge Response Supervision Module
[0081] The main function of the deep edge response supervision module is to enhance the segmentation model's attention to the edge area of the aortic dissection, suppress the attention to useless background information, and alleviate the problem of rough edge segmentation caused by discontinuous edge areas. This module is used to process the output feature maps sampled at each layer of the subnetwork.
[0082] Define the input feature map set as F in , F in =[f1, f2, f3, ..., f c ], Fin ∈R H×W×C , f i ∈R H×W , i = 1, 2, 3, ..., C, C represents the total number of channels of the input feature map, H represents the height of the feature map, W represents the width of the feature map, f i Represents the i-th feature map.
[0083] Here are the steps:
[0084] 1) Calculate the gradient magnitude of each pixel in the input feature map
[0085] g c (i, j) is the grayscale value of the pixel at the (i, j)th position in the cth feature map, g c i (i, j) is the gradient value of the pixel in the i direction, g c j (i, j) is the gradient value of the pixel in the j direction, A C (i, j) is the gradient amplitude of the pixel. The calculation formula is as follows:
[0086]
[0087]
[0088]
[0089] 2) Calculate the edge response coefficient matrix of the feature map
[0090] Because the gradient and amplitude of the pixels in the edge area are much larger than those in the non-edge area, the normalized gradient amplitude of the pixel can be used as the edge response coefficient of the pixel, giving the pixels in the edge area of the aortic dissection a larger edge response coefficient. The edge response coefficient r of the pixel at the (i, j)th position in the cth feature map is c (i, j) can be obtained by the following formula:
[0091]
[0092] Among them, A c min is the minimum gradient amplitude among all pixels in the cth feature map, A c max is the maximum gradient magnitude among all pixels in the cth feature map.
[0093] By calculating the edge response coefficient of each pixel in the feature map c, we can get the edge response coefficient matrix r of the cth feature map c Similarly, according to the edge response coefficient matrix of each feature map, the total edge response coefficient matrix R can be obtained, R∈RH×W×C , R = [r 1 ,r 2 ,r 3 ,…,r c ],r i ∈R H×W ,i=1,2,3,…,C,r i Represents the edge response coefficient matrix of the i-th feature map.
[0094] 3) Obtain the processed output feature map and perform deep supervision.
[0095] R is the edge response coefficient matrix of the C feature maps that describe the input feature image. The final output feature map F is obtained by multiplying R with the pixels at the corresponding position in the input feature map. out , the formula is as follows:
[0096] F out =F in ·*R (8)
[0097] Finally, deep supervision is performed on the output feature map to provide additional supervision for the edge area of the aortic dissection. Deep supervision can learn the transparency and directness of the edge area of the aortic dissection, achieve faster convergence of the model, and improve segmentation accuracy.
[0098] Set a combined loss function based on edge response loss and adaptive logarithmic loss;
[0099] The Dice coefficient is often used to evaluate the segmentation effect in medical image segmentation tasks. Its range is [0,1]. The larger the value, the more similar it is. The mathematical formula for its definition is as follows:
[0100]
[0101] A represents the sample label manually annotated by the expert, B represents the prediction result generated by the segmentation model, and Dice is the ratio of the intersection and union of these two regions. The larger the ratio, the closer the prediction result is to the label, and the better the segmentation effect. DiceLoss is a loss function based on the Dice coefficient, and its formula is shown in formula (10). In order to avoid the situation where the numerator or denominator is 0, a very small number s is often added to the numerator and denominator, and s can usually be taken as 0.0001.
[0102]
[0103] The first subnetwork performs segmentation of the entire aorta in aortic dissection, which is a two-classification problem. It includes two categories: background and entire aorta. The frequencies of occurrence of the entire aorta and background categories are very different. In most CT images, the entire aorta area only occupies a small part, resulting in the segmentation model not being able to converge well on all categories at the same time, which will lead to poor segmentation results on the entire aorta category. At the same time, there is also the problem of discontinuity in the edge area during the segmentation of aortic dissection. Discontinuous pixels near the edge area are easily misclassified, resulting in boundary misalignment between the segmentation result and the true result.
[0104] The second sub-network is used to segment the true and false lumens of aortic dissection, which is also a binary classification task. It includes two categories: true lumen and false lumen. The frequency of occurrence of true and false lumen categories in aortic dissection CT images is also unbalanced, and there is also a certain degree of imbalance. At the same time, due to the special structure of aortic dissection, there is a thin film between the true lumen and the false lumen, which leads to the discontinuity of the internal area of the aortic dissection.
[0105] Therefore, to address the regional discontinuity and category imbalance problems in the aortic dissection segmentation process, an edge response-based loss function and an adaptive logarithmic loss function were proposed respectively. A combined loss function was obtained based on the two loss functions and applied as the final total loss function in the sub-network of the segmentation model.
[0106] (1) Loss function based on edge response
[0107] Traditional CNN networks rely heavily on the texture features of images, but tend to ignore the shape features of objects. When experts manually segment aortic dissection CT images, they rely on the recognition of the boundary shapes of categories such as the true and false lumens of aortic dissection and the entire aortic area. That is, they first track the edge of the aortic dissection area, and then based on the prior knowledge of the edge of the aortic dissection area, it will be easy to derive its segmentation mask later.
[0108] Inspired by the above ideas, a loss function based on edge response is proposed to solve the problem of discontinuity in the edge area of aortic dissection. That is, the pixels in the edge area of the image are first detected by the canny edge detection operator, and then additional losses are calculated for these pixels to increase additional supervision of the discontinuous area. The main steps are as follows:
[0109] 1) Use the canny edge detection operator to detect the input image img_in and obtain a black and white image img_bw that highlights the edge area. The white part of the image is the edge area, and the black part is the other area.
[0110] 2) The image img_bw obtained in step 1) is superimposed with the predicted image img_p obtained by segmentation and the real label image img_t. The calculation formulas are shown in (11) and (12), and the predicted image img_pb and the real label image img_tb with prominent edge areas are obtained. Where [img_bw] ij Represents the grayscale value at the i-th row and j-th column in the image img_bw. [img_pb] ij , [img_tb] ij Same as above.
[0111]
[0112]
[0113] 3) Calculate the loss again for the predicted image img_pb containing only the edge area and the true label image img_tb to add additional supervision to the edge area; the loss calculation formula is as follows:
[0114]
[0115] Among them, DC loss is the additional loss value of the edge area, n c Indicates the total number of categories, N c is the number of pixels in the image that belong to category c. img_pb cn is the predicted value of the nth pixel of category c in image img_pb, img_tb cn is the true value of the nth pixel of category c in image img_tb.
[0116] (2) Adaptive logarithmic loss function
[0117] Secondly, an adaptive logarithmic loss function is proposed to address the imbalance problem of the whole aorta area and the true and false lumen categories in aortic dissection segmentation. When the sample is very difficult to identify, the nonlinear operation of the network can achieve a higher loss. In this way, the network with nonlinear loss may obtain more effective gradient updates than when using a linear loss function, and can truly identify the difficult and easy categories in the image. The formula of the adaptive logarithmic loss function of the present invention is as follows:
[0118]
[0119] In the formula, HE loss is the adaptive logarithmic loss function value, n c Indicates the total number of categories, because the first sub-network and the second sub-network both deal with 2-classification problems, so here n c The value of w is 2, cThe value of is the loss weight coefficient corresponding to category c, DL c is the Dice Loss value corresponding to category c, and its formula is as follows:
[0120]
[0121] In the formula, r cn is the true value of the nth pixel of category c, p cn is the predicted value of the nth pixel of category c, N c is the number of pixels belonging to category c in the image.
[0122] In the present invention, the true lumen, false lumen or whole aorta is a small category randomly. Therefore, a fixed weight cannot be applied to each category. The present invention sets the inverse of the square of the sum of the true values of all pixels of category c in the image as the loss weight coefficient w of the category. c The formula is as follows:
[0123]
[0124] Finally, we get the HE-based loss and DC loss The combined loss function is as follows:
[0125] T loss =DC loss +HE loss
[0126] The three-dimensional cascade segmentation model of the depth edge response of aortic dissection is trained based on the combined loss function and the training set, and the aortic dissection segmentation is completed using the trained three-dimensional cascade segmentation model of the depth edge response of aortic dissection. The method for training the three-dimensional cascade segmentation model of the depth edge response of aortic dissection is as follows: input the training set into the first sub-network, calculate the loss using the combined loss function, and adjust the hyperparameters using the validation set to complete the training of the first sub-network and obtain the segmentation result of the first sub-network; perform an intersection operation on the segmentation result of the first sub-network and the training set, input the operation result into the second sub-network as the training set of the second sub-network, calculate the loss using the combined loss function, and adjust the hyperparameters using the validation set to complete the training of the second sub-network.
[0127] In the training of the cascade segmentation model of this embodiment, training multiple networks simultaneously will result in limited computing power. Therefore, a step-by-step training strategy is often adopted, that is, first training the first sub-network and then training the second sub-network.
[0128] First, set the hyperparameters of the segmentation model, including batch size, optimizer, initial learning rate, maximum number of iterations, etc. Then train the first sub-network, input the full aorta training set of aortic dissection into the first sub-network, use the combined loss function to calculate the loss, and use the validation set to adjust the hyperparameters, complete the training, and save the network model. Intersect the segmentation result of the first sub-network with the full aorta training set of aortic dissection, and use the result as the training set of the second sub-network. Train the second sub-network in the same way and save the network model.
[0129] Practical use of the three-dimensional cascade segmentation model for aortic dissection. First, the original aortic dissection CT image test set is processed using the data preprocessing method, and then the saved trained segmentation network model parameters are loaded and input into the segmentation model to obtain the segmentation result mask images of the whole aorta, true lumen, and false lumen of aortic dissection. Finally, the mask images of the whole aorta, true lumen, and false lumen of aortic dissection are mapped to the original aortic dissection CT image, and the corresponding predicted segmentation results of the whole aorta, true lumen, and false lumen of aortic dissection can be obtained.
[0130] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for segmenting aortic dissection based on edge response and nonlinear loss, characterized in that: The following steps are included: Preprocessing the aortic dissection CT images, and dividing the preprocessed aortic dissection CT images into a training set and a validation set; Establish a three-dimensional cascade segmentation model of aortic dissection depth edge response; Set a combined loss function based on edge response loss and adaptive logarithmic loss; Based on the combined loss function and the training set, the three-dimensional cascade segmentation model of the depth edge response of aortic dissection is trained, and the segmentation of aortic dissection is completed using the trained three-dimensional cascade segmentation model of the depth edge response of aortic dissection; The aortic dissection depth edge response three-dimensional cascade segmentation model includes a first sub-network and a second sub-network; The first sub-network is used to segment the entire aorta area of aortic dissection; The second sub-network is used to segment the true lumen and false lumen of aortic dissection; The first sub-network and the second sub-network both include an encoder, a decoder, and a plurality of hop connections; The first sub-network and the second sub-network both further include a deep edge response supervision module; The deep edge response supervision module is used to process the output feature map sampled from each layer in the first sub-network and the second sub-network; The method for processing the output feature map of each layer upsampled includes: Calculate the gradient magnitude of each pixel in the feature map; Calculate the edge response coefficient matrix of the feature map; Obtaining a processed output feature map based on the gradient amplitude and the edge response coefficient matrix, and performing deep supervision on the processed output feature map; The edge response loss function formula is: Among them, DC loss is the additional loss value of the edge area, n c Indicates the total number of categories, N c is the number of pixels belonging to category c in the image, img_pb cn is the predicted value of the nth pixel of category c in image img_pb, img_tb cn is the true value of the nth pixel of category c in image img_tb, img_pb is the predicted image containing only the edge area, and img_tb is the true label image containing only the edge area; The adaptive logarithmic loss function formula is: Among them, HE loss is the adaptive logarithmic loss function value, n c is the total number of categories, w c is the loss weight coefficient corresponding to category c, DL c is the Dice Loss value corresponding to category c; The DL c The formula is: Among them, r cn is the true value of the nth pixel of category c, p cn is the predicted value of the nth pixel of category c, N c is the number of pixels belonging to category c in the image; The w c The formula is:
2. The aortic dissection segmentation method based on edge response and nonlinear loss according to claim 1, characterized in that: The preprocessing includes data format conversion processing, gamma enhancement processing, data geometric transformation augmentation processing and standardization processing.
3. The aortic dissection segmentation method based on edge response and nonlinear loss according to claim 2, characterized in that: The data format conversion process is to convert the aortic dissection CT image from a two-dimensional format to a three-dimensional format; The gamma enhancement process is to perform gamma grayscale transformation on the three-dimensional aortic dissection CT image, and the gamma grayscale transformation formula is: CT out =2CT in 0.65 Among them, CT in is the gray value of the image before gamma grayscale transformation, CT out is the gray value of the image after gamma grayscale transformation, 2 is the grayscale scaling factor, and 0.65 is the gamma factor value; The data geometric transformation augmentation process is to use image translation, rotation, flipping, and mirroring geometric transformations to increase the data volume of the aortic dissection CT image after the gamma enhancement process; The standardization process is to standardize the resolution and grayscale value of the aortic dissection CT image after the data geometric transformation and augmentation process, and process the resolution of the CT image to 256×128×128 and the grayscale value to between 0-1.
4. The aortic dissection segmentation method based on edge response and nonlinear loss according to claim 1, characterized in that: The method for training the aortic dissection depth edge response three-dimensional cascade segmentation model is: Inputting the training set into the first sub-network, calculating the loss using the combined loss function, and adjusting the hyperparameters using the validation set to complete the training of the first sub-network and obtain the segmentation result of the first sub-network; An intersection operation is performed on the segmentation result of the first sub-network and the training set, and the operation result is input into the second sub-network as the training set of the second sub-network, the loss is calculated using the combined loss function, and the hyperparameters are adjusted using the validation set to complete the training of the second sub-network.
